Substation safety control method and device based on camera following

Abstract

The invention provides a transformer substation safety management and control method and device based on camera following, and the method comprises the steps: capturing a target object entering a safety fence region based on rotatable cameras, obtaining the spatial position information, controlling at least two rotatable cameras to synchronously steer and lock the target object based on the spatial position information, and carrying out the safety management and control of a transformer substation. Obtaining a first multi-view real-time video stream; based on the first multi-view real-time video stream, performing continuous tracking on a motion track of the target object in a preset security fence area to obtain continuous space track data; judging whether the target object is continuously located in the preset security fence area based on the continuous space trajectory data to obtain a final space state, and judging whether the target object has a border crossing behavior trend based on the final space state to obtain a border crossing risk judgment result; and executing a corresponding alarm operation or an alarm release operation based on the border crossing risk judgment result to obtain a safety management and control response instruction. The reliability of safety management and control of the transformer substation is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and device for substation safety management based on camera tracking. Background Technology

[0002] In the current field of substation safety management, a common method is a safety monitoring method based on fixed cameras and static electronic fences. This method involves deploying video monitoring equipment with fixed viewing angles in key areas of the substation and pre-setting the boundaries of the electronic fence. When a target is detected entering the fenced area, an alarm is triggered.

[0003] However, existing methods, due to the fixed camera viewpoint, cannot continuously track targets entering the fenced area. Once the target shifts, is obstructed by equipment, or is located at the intersection of multiple fences, the system struggles to maintain continuous observation, making it impossible to accurately determine whether the target has truly crossed the boundary or remains within the danger zone. This results in frequent false alarms or missed alarms. Especially in complex operating environments, the lack of dynamic and coordinated perception of target movement and spatial location weakens the reliability of substation safety management. Summary of the Invention

[0004] This invention provides a substation safety management method and device based on camera tracking, aiming to improve the reliability of substation safety management.

[0005] In a first aspect, the present invention provides a substation safety management and control method based on camera tracking, comprising: Based on the rotatable cameras deployed at multiple different locations in the substation, the target object entering the preset safety fence area is captured to obtain spatial location information. Based on the spatial location information, at least two rotatable cameras are controlled to turn synchronously and lock onto the target object to obtain a first multi-view real-time video stream. Based on the first multi-view real-time video stream, the motion trajectory of the target object within the preset safety fence area is continuously tracked to obtain continuous spatial trajectory data; Based on the continuous spatial trajectory data, it is determined whether the target object is continuously located inside the preset safety fence area to obtain the final spatial state. Based on the final spatial state, it is determined whether the target object has a tendency to cross the boundary to obtain the boundary risk assessment result. Based on the boundary risk assessment result, the substation safety alarm device is controlled to perform the corresponding alarm operation or alarm cancellation operation to obtain a safety control response command.

[0006] Secondly, the present invention also provides a substation safety management and control device based on camera tracking, for implementing the substation safety management and control method based on camera tracking as described in the first aspect; the substation safety management and control device based on camera tracking includes: The multi-view fusion module is used to capture target objects entering a preset safety fence area based on rotatable cameras deployed at multiple different locations in the substation, obtain spatial location information, and control at least two rotatable cameras to synchronously turn and lock the target object based on the spatial location information, thereby obtaining a first multi-view real-time video stream. The motion trajectory tracking module is used to continuously track the motion trajectory of the target object within the preset safety fence area based on the first multi-view real-time video stream, and obtain continuous spatial trajectory data. The boundary crossing behavior monitoring module is used to determine whether the target object is continuously located inside the preset safety fence area based on the continuous spatial trajectory data, to obtain the final spatial state, and to determine whether the target object has a boundary crossing behavior trend based on the final spatial state, so as to obtain the boundary crossing risk judgment result. The safety control response module is used to control the substation safety alarm device to perform corresponding alarm operations or alarm cancellation operations based on the boundary risk assessment result, and obtain safety control response instructions.

[0007] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing computer software programs; and a processor for reading and executing the computer software programs, thereby realizing the substation safety management method based on camera tracking as described above.

[0008] Fourthly, the present invention also provides a non-transitory computer-readable storage medium storing a computer software program, which, when executed by a processor, implements the substation safety management method based on camera tracking as described above.

[0009] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the substation safety management method based on camera tracking as described above.

[0010] The substation safety management method based on camera tracking provided in this invention captures the spatial position information of a target object entering a preset safety fence area using multiple rotatable cameras located at different positions within the substation. Based on this information, at least two rotatable cameras are controlled to synchronously turn and lock onto the target object, obtaining a first multi-view real-time video stream. This solves the problem of fixed camera perspectives being immobile and achieves synchronous multi-view locking and observation of the target object. The movement trajectory of the target object within the preset safety fence area is continuously tracked based on the first multi-view real-time video stream, obtaining continuous spatial trajectory data. This avoids situations where continuous observation is impossible when the target is displaced, obstructed, or located at the intersection of multiple fences, achieving continuous trajectory capture of the target. Based on the continuous spatial trajectory data, it determines whether the target object remains continuously within the preset safety fence area to obtain the final spatial state, and determines whether the target object exhibits a tendency to cross the boundary to obtain a boundary risk assessment result. This solves the problem of accurately determining whether the target has actually crossed the boundary or is still within the danger zone, effectively avoiding judgment bias caused by discontinuous observation. Based on the boundary risk assessment results, the substation safety alarm device is controlled to perform corresponding alarm operations or alarm cancellation operations, and a safety management response command is obtained. This solves the problem of the inability to continuously track targets and the frequent false alarms and missed alarms caused by the fixed camera's fixed viewing angle. It improves the dynamic collaborative perception capability of the target's movement status and spatial position in complex operating environments, reduces the false alarm rate and missed alarm rate of safety alarms, and improves the reliability of substation safety management. Attached Figure Description

[0011] Figure 1 This is a schematic flowchart of the substation safety management and control method based on camera tracking provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the substation safety management and control device based on camera tracking provided in an embodiment of the present invention; Figure 3 An embodiment diagram of the electronic device provided in this invention; Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0014] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0015] See Figure 1 , Figure 1 This is a flowchart illustrating the substation safety management method based on camera tracking provided by the present invention. In this embodiment, the executing entity of the substation safety management method based on camera tracking is a safety management device. Therefore, the substation safety management method based on camera tracking includes: Step 10: Based on the rotatable cameras deployed at multiple different locations in the substation, the target object entering the preset safety fence area is captured to obtain spatial location information. Based on the spatial location information, at least two rotatable cameras are controlled to turn synchronously and lock onto the target object to obtain a first multi-view real-time video stream.

[0016] Optionally, the safety control device utilizes multiple rotatable cameras deployed at different locations within the substation. These rotatable cameras are monitoring devices capable of 360-degree horizontal rotation and ±90-degree vertical rotation, supporting real-time image acquisition and spatial positioning. Each rotatable camera undergoes pre-calibration to determine its fixed coordinates within the substation's physical space. All rotatable cameras establish wired communication connections with the safety control device to ensure stable and real-time data transmission. The preset safety fence area is a physical area pre-defined by the safety control device based on the substation's safety management needs. This area is precisely defined using the substation's actual geographical boundary coordinates and equipment layout range, and is stored in the safety control device's local database as a baseline range for target object capture and judgment.

[0017] Optionally, the safety control device controls each rotatable camera to perform a full-area scan of the preset safety fence area at a preset frequency (30 frames per second). When a target object (a person, vehicle, or other non-preset allowed moving object entering the preset safety fence area) appears in the image captured by any rotatable camera, the target object is immediately captured.

[0018] During the capture process, the safety control device uses the positioning module built into the rotatable camera to calculate the spatial position information of the target object by combining the fixed coordinates of the camera itself and the pixel position of the target object in the image. The spatial position information is the three-dimensional coordinate data of the target object in the physical space of the substation. The three-dimensional coordinates correspond to the X-axis, Y-axis and Z-axis values ​​of the substation's preset coordinate system, where the X-axis and Y-axis correspond to the horizontal ground direction and the Z-axis corresponds to the vertical ground height direction.

[0019] Furthermore, the security control device, based on spatial location information, selects at least two rotatable cameras that are closest to the target object and have unobstructed viewing angles, and sends synchronous turning commands to these two or more rotatable cameras. These synchronous turning commands contain the spatial location information of the target object. Upon receiving the command, the rotatable cameras drive their built-in rotation mechanisms to adjust their horizontal and vertical angles, synchronously turning towards the target object until image recognition confirms and locks onto the target object. In the locked state, the cameras track the target object's movement in real time, keeping the target object always in the center of the image frame. After locking onto the target object, each rotatable camera synchronously acquires real-time video data, forming a first multi-view real-time video stream. Therefore, the first multi-view real-time video stream is a collection of multiple real-time video data streams obtained by multiple rotatable cameras simultaneously capturing images of the target object from different directions, with each video data stream containing timestamp information.

[0020] In one embodiment, at a 220 kV substation, the safety control device pre-defines a safety fence area within a 10-meter radius around the No. 3 main transformer. The three-dimensional coordinates of this area are 50-60 meters on the X-axis, 30-40 meters on the Y-axis, and 0-5 meters on the Z-axis. Six rotatable cameras are deployed around this area, mounted on the top of the fence walls to the east, south, west, and north of the No. 3 main transformer, and on the frames of two adjacent devices. Each camera undergoes position calibration to determine its fixed coordinates; for example, the coordinates of the camera on the top of the east fence are 45 meters on the X-axis, 35 meters on the Y-axis, and 6 meters on the Z-axis, while the coordinates of the camera on the top of the south fence are 55 meters on the X-axis, 25 meters on the Y-axis, and 6 meters on the Z-axis. The safety control device controls the six rotatable cameras to scan the entire safety fence area at a frequency of 30 frames per second. When a person (target object) appears in the image captured by the camera on the top of the west fence, that person is immediately captured.

[0021] The security control device, using the built-in positioning module of the west-side camera and combining the camera's fixed coordinates (X-axis 55 meters, Y-axis 45 meters, Z-axis 6 meters) with the pixel position of the person in the image, calculates the person's spatial location information as X-axis 56 meters, Y-axis 38 meters, and Z-axis 1.7 meters. The device then selects the two cameras at the top of the east and south walls, which are closest to the person and have an unobstructed view, and sends synchronized turning commands to these two cameras. These commands contain the person's spatial location information: X-axis 56 meters, Y-axis 38 meters, and Z-axis 1.7 meters.

[0022] Upon receiving the command, the rotatable cameras on the east and south sides respectively drive their rotation mechanisms to adjust their angles. The east camera rotates 30 degrees westward from its initial scanning direction and adjusts vertically downward by 15 degrees; the south camera rotates 20 degrees northward from its initial scanning direction and adjusts vertically downward by 12 degrees, simultaneously turning towards the person's location. Image recognition confirms that both cameras have successfully locked onto the person, keeping them centered in the image frame. Subsequently, the two cameras synchronously acquire real-time video data at a frequency of 30 frames per second, forming the first multi-view real-time video stream, with each video data stream accompanied by a timestamp.

[0023] Step 20: Based on the first multi-view real-time video stream, continuously track the movement trajectory of the target object within the preset safety fence area to obtain continuous spatial trajectory data.

[0024] Optionally, the safety control device continuously tracks the movement of the target object within the preset safety fence area based on multi-view image fusion and target tracking combined with the first multi-view real-time video stream. By associating the target object features and timestamp information in multiple video streams, it integrates them to form continuous spatial trajectory data that can reflect the spatial location of the target object at different time points. The continuous spatial trajectory data is an ordered set of spatial location information of the target object within a preset time range, as specifically in steps 201 to 204.

[0025] Step 30: Based on continuous spatial trajectory data, determine whether the target object is continuously located inside the preset safety fence area to obtain the final spatial state, and based on the final spatial state, determine whether the target object has a tendency to cross the boundary to obtain the boundary risk assessment result.

[0026] Optionally, the safety control device determines whether the target object is within a preset safety fence area at each spatial location within a preset time range based on the area boundary judgment using continuous spatial trajectory data. It then integrates all judgment results to obtain the final spatial state of the target object, as described in steps 301 to 305. The final spatial state includes a compliant state and a non-compliant state. A compliant state indicates the target object is located inside the preset safety fence area, while a non-compliant state indicates the target object is located outside the preset safety fence area. Furthermore, based on trend prediction and the final spatial state, as well as the movement direction and speed characteristics of the target object, the safety control device determines whether the target object has a tendency to cross the boundary outside the preset safety fence area, and obtains the boundary crossing risk determination result, as in steps 306 to 309. The boundary crossing risk determination result includes whether there is a boundary crossing risk or not.

[0027] Step 40: Based on the boundary risk assessment result, control the substation safety alarm device to perform the corresponding alarm operation or alarm cancellation operation to obtain the safety control response command.

[0028] Optionally, the safety management and control device controls the substation safety alarm device to perform corresponding operations based on preset risk response rules and boundary crossing risk assessment results. The substation safety alarm device is a collection of equipment with audible and visual alarms, SMS alarms, and local linkage control functions. It includes audible and visual alarms installed in the substation duty room, on-site audible and visual alarms deployed in various areas, an SMS alarm module linked to the duty personnel's mobile phone, and associated access control modules, video pop-up modules, etc. The safety alarm device establishes a communication connection with the safety management and control device and receives control commands sent by the safety management and control device.

[0029] If the boundary crossing risk assessment result indicates the existence of a boundary crossing risk, an alarm operation command is sent to the substation safety alarm device. The alarm operation command corresponds to different alarm methods according to the risk level. The risk levels are divided into Level 1 risk (the target object has approached the boundary of the preset safety fence area and has an imminent tendency to cross the boundary) and Level 2 risk (the target object has partially crossed the preset safety fence area and is exhibiting continuous boundary crossing behavior). For Level 1 risk, the safety control device controls the on-site audible and visual alarm to be activated, issuing an audible and visual alert once per second. At the same time, it controls the duty room video pop-up module to display the first-view multi-angle real-time video stream of the target object on the duty room monitoring screen. For Level 2 risk, in addition to executing the Level 1 risk alarm operation, it controls the SMS alarm module to send alarm information to the duty personnel's mobile phone. The alarm information includes the spatial location information of the target object, the boundary crossing risk level, and the alarm time. At the same time, it controls the access control module of the area where the target object is located to close the access control of surrounding passages to prevent unauthorized personnel from entering or the target object from escaping.

[0030] If the boundary risk assessment result indicates that there is no boundary risk, the system checks whether the current substation safety alarm device is in an alarm state. If it is in an alarm state, the system sends an alarm cancellation command to the safety alarm device, controlling all alarm devices in operation to stop working, including turning off the audible and visual alarms, terminating SMS alarm sending, closing video pop-ups, and restoring the access control module to normal status. If the current device is not in an alarm state, the safety control device does not send any additional commands, and the substation safety alarm device remains in standby mode.

[0031] After the safety management and control device completes the control operation of the substation safety alarm device, it generates a safety management and control response instruction. The safety management and control response instruction is an instruction file that records the risk assessment result, alarm operation content (or alarm cancellation operation content), operation time, and equipment number involved. This instruction file is stored in the local database of the safety management and control device as a traceability basis for substation safety management and control.

[0032] In one embodiment, preset risk response rules are established: Level 1 risk triggers on-site audible and visual alarms and a video pop-up window in the duty room; Level 2 risk triggers on-site audible and visual alarms, a video pop-up window in the duty room, an SMS alarm, and access control linkage closure. For example, the security control device obtains the boundary crossing risk determination result from step 30 as indicating the existence of Level 1 risk, meaning the target object has approached the boundary of the preset safety fence area (e.g., 59.5 meters on the X-axis, 35 meters on the Y-axis, and 1.7 meters on the Z-axis), and there is a tendency for it to cross the boundary.

[0033] For example, the safety control device sends a level-one risk alarm operation command to the substation safety alarm device according to preset rules. The command includes the location of the target object, the risk level, and the alarm method.

[0034] After receiving the instruction, the substation safety alarm device activates the on-site audible and visual alarm in the area where the target object is located (around the No. 3 main transformer), emitting a red audible and visual alert once per second; at the same time, the duty room video pop-up module responds to the instruction and displays the first multi-view real-time video stream of the target object in a pop-up window on the left side of the main monitoring screen in the duty room, so that the duty personnel can observe the dynamics of the target object in real time.

[0035] If, in subsequent steps, the safety control device updates the boundary risk assessment result to indicate that there is no boundary risk (the target object has moved into the preset safety fence area, currently positioned at 55 meters X-axis, 36 meters Y-axis, and 1.7 meters Z-axis), and the safety alarm device is in a Level 1 risk alarm state, the safety control device immediately sends an alarm deactivation command to the safety alarm device. Upon receiving the command, the safety alarm device deactivates the on-site audible and visual alarms, stops the audible and visual prompts, and simultaneously closes the video pop-up window on the duty room monitoring screen. The access control module remains in normal operation (access control linkage is not activated due to the Level 1 risk in this case).

[0036] The safety control device recorded the two operations and generated two safety control response instructions. The first instruction recorded "There is a Level 1 boundary crossing risk. Activate the on-site audible and visual alarm and the duty room video pop-up window. The operation time is [date], [hour], [minute], and [second]. The equipment involved is the on-site audible and visual alarm around the No. 3 main transformer and the duty room video pop-up window module." The second instruction recorded "There is no boundary crossing risk. Deactivate the Level 1 risk alarm. The operation time is [date], [hour], [minute], and [second]. The equipment involved is the on-site audible and visual alarm around the No. 3 main transformer and the duty room video pop-up window module."

[0037] The embodiments of the present invention solve the problems of continuous target tracking and frequent false alarms and missed alarms caused by the fixed camera's fixed viewing angle. It improves the dynamic collaborative perception capability of the target's motion state and spatial position, reduces the false alarm rate and missed alarm rate of safety alarms, and improves the reliability of substation safety management.

[0038] Optionally, the process of steps 201 to 205 includes: Step 201: Based on the visual feature information of the target object contained in each frame of the first multi-view real-time video stream and the spatial orientation parameters of the corresponding rotatable camera, determine the two-dimensional image coordinate position of the target object in each view.

[0039] Optionally, the security control device performs frame-by-frame analysis on each video data in the first multi-view real-time video stream, extracting the visual feature information of the target object in each frame image. The visual feature information includes the shape outline, texture features, grayscale distribution features, and feature point coordinate set of the target object. The feature point coordinate set is the pixel coordinates of key parts of the target object (such as the head and shoulders of a person, the front and rear of a vehicle) in a single frame image. The pixel coordinates are determined with the upper left corner of the image as the origin, the horizontal axis to the right as the horizontal axis, and the vertical axis downward as the vertical axis.

[0040] Furthermore, the security control device, through communication with each rotatable camera, obtains the spatial orientation parameters of the rotatable camera corresponding to each video stream. These parameters are the current horizontal and vertical rotation angles of the rotatable camera. The horizontal rotation angle is based on the camera's initial calibration direction (e.g., due north), with clockwise rotation being positive and ranging from 0 to 360 degrees. The vertical rotation angle is based on the horizontal direction, with upward rotation being positive and downward rotation being negative, ranging from -90 to 90 degrees. Based on the visual feature information of the corresponding frame of each video stream, the security control device locates the core region of the target object in that frame. Combining this with the image resolution parameter (the total number of pixels in each frame, e.g., 1920 pixels * 1080 pixels), it calculates the two-dimensional image coordinates of the target object from that viewpoint. These two-dimensional image coordinates are the pixel coordinates of the center point of the target object's core region in the image, composed of horizontal and vertical pixel values.

[0041] Step 202: Based on the two-dimensional image coordinates of the target object from each viewpoint and the spatial installation position and orientation parameters of each rotatable camera, determine the spatial coordinates of the target object in three-dimensional space.

[0042] Optionally, the spatial installation position of the rotatable camera is based on the fixed three-dimensional coordinate data of the camera in the physical space of the substation, which has been calibrated and stored in step 10. The three-dimensional coordinates are respectively relative to the horizontal X-axis, horizontal Y-axis, and vertical Z-axis values ​​of the substation's preset coordinate system. The safety control device performs coordinate transformation calculations based on the two-dimensional image coordinate position of each viewpoint, combined with the image parameters of the corresponding camera (image parameters include lens focal length and imaging ratio; the lens focal length is the fixed focal length value of the camera lens; the imaging ratio is the ratio between the image pixel size and the actual physical size).

[0043] Optionally, in the calculation process of that embodiment, the relative attitude relationship between the camera's imaging plane and the substation's preset coordinate system is determined based on the spatial orientation parameters of the rotatable camera. Then, based on the two-dimensional image coordinate position, lens focal length, and imaging ratio, the horizontal and vertical offset distances of the target object relative to the camera are calculated. The horizontal offset distance is the left-right offset distance of the target object in the horizontal orientation direction of the camera, and the vertical offset distance is the up-down offset distance of the target object in the vertical orientation direction of the camera. Subsequently, using the spatial installation position of the camera as a reference point, and combining the horizontal and vertical offset distances and the spatial orientation parameters, the preliminary spatial coordinates of the target object in three-dimensional space are determined through spatial geometric calculations. The safety control device fuses and corrects the preliminary spatial coordinates obtained from multiple perspectives, eliminates abnormal data caused by perspective obstruction and measurement errors, and takes the average value of multiple valid preliminary spatial coordinates as the final three-dimensional spatial coordinates of the target object. The three-dimensional spatial coordinates are the actual position coordinates of the target object in the physical space of the substation, corresponding to the X-axis, Y-axis, and Z-axis values ​​of the substation's preset coordinate system.

[0044] Step 203: Based on the spatial coordinates of the target object in three-dimensional space and the timestamp information of the first multi-view real-time video stream corresponding to the current frame, determine the first spatial state of the target object at the current moment.

[0045] Optionally, the timestamp information is the precise time when the rotatable camera captured the image frame, in the format of year-month-day-hour-minute-second-millisecond, used to identify the time node corresponding to the spatial coordinates of the target object.

[0046] The safety control device associates and binds the three-dimensional spatial coordinates of the target object with the corresponding timestamp information to form the first spatial state of the target object at the current moment. Therefore, the first spatial state is a dataset containing the real-time location of the target object and the corresponding time information, which can accurately reflect the position of the target object in the physical space of the substation at a specific moment. If it is the spatial coordinates corresponding to the first frame of video data, the first spatial state is stored as the initial spatial state in the local database of the safety control device; if it is the spatial coordinates corresponding to the subsequent frame of video data, the first spatial state is used as the current moment state and associated with the previously stored moment state. At the same time, in order to ensure the continuity and accuracy of the timestamps, if a timestamp anomaly occurs (such as time jump or duplication), the spatial coordinate data of the corresponding frame is discarded.

[0047] Step 204: Based on the first spatial state at the current moment and the second spatial state at the previous moment, determine the spatial displacement vector of the target object between adjacent moments.

[0048] Optionally, the second spatial state is the spatial state of the target object corresponding to the previous acquisition time adjacent to the current time, which also includes three-dimensional spatial coordinates and timestamp information. The safety control device calculates the time interval between the current time and the previous time, which is the difference between the timestamp of the current time and the timestamp of the previous time, in seconds, to reflect the time span between the two location acquisitions.

[0049] Furthermore, the safety control device calculates the position difference of the target object in the X-axis, Y-axis, and Z-axis directions in three-dimensional space. The position difference in the X-axis direction is the difference between the current X-axis coordinate and the previous X-axis coordinate. The position difference in the Y-axis direction is the difference between the current Y-axis coordinate and the previous Y-axis coordinate. The position difference in the Z-axis direction is the difference between the current Z-axis coordinate and the previous Z-axis coordinate. The positive or negative value of the position difference is used to indicate the direction of movement of the target object in the corresponding axis (positive value is the positive direction of the coordinate axis, and negative value is the negative direction of the coordinate axis).

[0050] Furthermore, the safety control device determines the spatial displacement vector of the target object between adjacent moments based on the position difference and time interval in the X-axis, Y-axis, and Z-axis directions. The spatial displacement vector is vector data that includes displacement magnitude and displacement direction. The displacement magnitude is the straight-line distance between two positions in three-dimensional space, and the displacement direction is jointly determined by the movement trend in the X-axis, Y-axis, and Z-axis directions, which can reflect the movement trajectory segment of the target object in adjacent moments.

[0051] Step 205: Continuously track the spatial displacement vector and the first spatial state at the current moment to obtain continuous spatial trajectory data.

[0052] Optionally, the safety control device continuously tracks the target object based on the spatial displacement vector and the first spatial state at the current moment, combined with a preset tracking period (the preset tracking period is the time interval for continuously acquiring the spatial state of the target object, which matches the video stream frame rate).

[0053] In one embodiment, the safety control device can integrate the spatial state and spatial displacement vector of adjacent time moments frame by frame to form an ordered spatial position sequence of the target object within a preset time range, eliminate abnormal displacement data to ensure trajectory continuity, and finally obtain continuous spatial trajectory data, as in steps 2051 to 2054. Therefore, the continuous spatial trajectory data can completely reflect the movement process of the target object within the preset safety fence area.

[0054] The embodiments of the present invention obtain continuous spatial trajectory data that can fully reflect the movement process of the target object within the preset safety fence area, effectively making up for the problems of trajectory tracking breakage and inaccurate positioning caused by the limited field of view of fixed cameras, improving the continuity and accuracy of the target object's movement trajectory in complex working environments, thereby improving the dynamic collaborative perception capability of the target's movement state and spatial position, reducing the false alarm rate and missed alarm rate of safety alarms, and improving the reliability of substation safety management and control.

[0055] Optionally, the process of steps 2051 to 2054 includes: Step 2051: Based on the spatial displacement vector and the first spatial state at the current moment, determine the direction of movement and the result of the motion continuity judgment of the target object within the preset safety fence area.

[0056] Optionally, the spatial displacement vector is vector data of the target object between adjacent time points, including displacement magnitude and displacement direction, and the first spatial state is a dataset containing the target object's current three-dimensional spatial coordinates and corresponding timestamp information.

[0057] The safety control device determines the direction of movement of the target object based on the positive and negative values ​​of the position differences in the X, Y, and Z axes of the spatial displacement vector. The direction of movement refers to the direction in which the target object moves within the three-dimensional space of the substation's preset coordinate system. Specifically, it is determined by the combination of movement trends along each axis. For example, when the difference between the X-axis coordinates is positive and the difference between the Y-axis coordinates is negative, the direction of movement is determined to be the composite direction along the positive X-axis and the negative Y-axis. At the same time, the proportion of the movement components along each axis is marked to clarify the main direction of movement.

[0058] Furthermore, the safety control device judges the continuity of motion. The result of the motion continuity judgment is the conclusion of whether the motion state of the target object remains continuous. The judgment basis is the degree of fit between the current spatial displacement vector and the historical spatial displacement vector. The historical spatial displacement vector is the set of spatial displacement vectors of the previous several adjacent moments (the preset number is 3).

[0059] Optionally, the specific process of this embodiment of the invention is as follows: calculate the difference in displacement magnitude between the current spatial displacement vector and the historical spatial displacement vector, and the deviation angle of the motion direction, wherein the difference in displacement magnitude is the difference between the current displacement magnitude and the average historical displacement magnitude, and the deviation angle of the motion direction is the angle between the current motion direction and the average historical motion direction. If the difference in displacement magnitude is within a preset threshold (the preset threshold is 30% of the average historical displacement magnitude), and the deviation angle of the motion direction is less than a preset angle (the preset angle is 15 degrees), then the motion continuity judgment result is determined to be continuous; if either item exceeds the corresponding threshold, then the motion continuity judgment result is determined to be discontinuous.

[0060] Step 2052: Based on the judgment results of motion direction and motion continuity, the spatial neighborhood of the target object appearing at the next moment is limited to obtain the spatial search constraint region of the target object.

[0061] Optionally, the safety control device, based on the judgment results of the movement direction and continuity, and combined with the boundary parameters of the preset safety fence area, limits the spatial neighborhood of the target object's next location. The spatial neighborhood is a limited range defined based on the target object's current position and movement characteristics, used to narrow the search range for the target object in the next moment, improving positioning efficiency and accuracy. First, using the three-dimensional spatial coordinates in the first spatial state at the current moment as a reference point, a preset distance is extended according to the movement direction (the preset distance is a reasonable range determined based on the current displacement magnitude; 1.5 times the current displacement magnitude is used for continuous movement, and 0.8 times the current displacement magnitude is used for discontinuous movement) to determine the initial neighborhood range.

[0062] If the motion continuity judgment result is continuous, based on the stability of the historical motion trajectory, the boundary of the initial neighborhood range is shrunk by a preset ratio (the preset ratio is 20%) to obtain the spatial search constraint region. At this time, the region range is relatively concentrated, which is suitable for the continuous motion characteristics of the target object.

[0063] If the motion continuity judgment result is discontinuous, considering that the target object may turn or stop suddenly, the initial neighborhood range is expanded by a preset ratio (preset ratio is 50%) in both the motion direction and the vertical direction. At the same time, it is ensured that the spatial search constraint area does not exceed the boundary of the preset safety fence area. If the initially expanded range exceeds the fence boundary, the area range is adjusted with the fence boundary as the limit. Finally, the spatial search constraint area of ​​the target object is obtained. The spatial search constraint area is a cubic area in three-dimensional space, and the specific range is defined by the maximum and minimum values ​​of the X-axis, Y-axis and Z-axis.

[0064] Step 2053: Based on the spatial search constraint region and the second multi-view real-time video stream at the next moment, the two-dimensional image coordinate position of the target object at the next moment is matched and located to obtain the updated two-dimensional image coordinate position of the target object at the next moment.

[0065] Optionally, the second multi-view real-time video stream at the next moment is received. The second multi-view real-time video stream is a collection of multiple real-time video data collected synchronously by at least two rotatable cameras that have locked the target object in step 10. Each video data is accompanied by the spatial orientation parameters and timestamp information of the corresponding rotatable camera, and the timestamp is continuous with the current timestamp. Therefore, each video data in the second multi-view real-time video stream is frame parsed to extract the visual feature information of each frame image. The visual feature information is the shape outline, texture features, grayscale distribution features and feature point coordinate set of the target object.

[0066] Subsequently, based on the spatial search constraint region, and combined with the spatial installation position, spatial orientation parameters, and image parameters (lens focal length, imaging ratio, and image resolution parameters) of the corresponding rotatable camera, the three-dimensional spatial search constraint region is converted into a two-dimensional search range in the video images from each viewpoint. The two-dimensional search range is a specific pixel region in the corresponding image. Within the two-dimensional search range of each viewpoint, the security control device matches and compares the extracted visual feature information with the visual feature information of the target object at the current moment. The comparison includes the similarity of the outline and the overlap of feature points. The similarity of the outline is the overlap ratio of the outlines at two different times, and the overlap of feature points is the proportion of successfully matched feature points out of the total number of feature points.

[0067] Optionally, if the feature matching result from a certain viewpoint meets preset conditions (outline similarity not less than 80%, and feature point overlap not less than 70%), then the core region of the target object from that viewpoint is locked, and the pixel coordinates of the center point of the core region are calculated as candidate 2D image coordinate positions of the target object from that viewpoint. If multiple viewpoints are successfully matched, then the consistency of each candidate 2D image coordinate position is checked, and coordinate positions with excessive deviation (deviation exceeding 50 pixels) are removed. The average value of the remaining candidate coordinate positions is taken as the final coordinates of that viewpoint. Finally, the coordinate positions of all valid viewpoints are integrated to obtain the updated 2D image coordinate position of the target object at the next moment. The updated 2D image coordinate position is the set of pixel coordinates of the center point of the core region of the target object from each viewpoint.

[0068] Step 2054: Based on the updated two-dimensional image coordinate position, the spatial orientation parameters of the corresponding rotatable camera, and the spatial installation position, determine the updated spatial coordinates of the target object in the three-dimensional space at the next moment, and connect the updated spatial coordinates with the historical spatial state in sequence to obtain continuous spatial trajectory data.

[0069] Optionally, the spatial orientation parameters are the horizontal and vertical rotation angles of the rotatable camera corresponding to the second multi-view real-time video stream, and the spatial installation position is the calibrated and stored fixed three-dimensional coordinate data of the camera. Following the coordinate transformation calculation method in step 202, for each viewpoint, the updated two-dimensional image coordinate position is determined by combining the image parameters of the corresponding camera to determine the horizontal and vertical offset distances of the target object relative to that camera. Then, using the camera's spatial installation position as a reference point, preliminary updated spatial coordinates for each viewpoint are obtained through spatial geometric calculations. The preliminary updated spatial coordinates of multiple viewpoints are fused and corrected, eliminating abnormal data caused by viewpoint occlusion and measurement errors (abnormal data includes coordinate data exceeding the spatial search constraint area and data with a deviation exceeding 0.5 meters from most coordinates). The average of the valid preliminary updated spatial coordinates is taken as the updated spatial coordinates of the target object in three-dimensional space at the next moment.

[0070] Furthermore, the safety control device will associate and bind the updated spatial coordinates with the corresponding timestamp information to form the updated spatial state at the next moment. Then, the updated spatial state will be connected to the historical spatial states (including the first spatial state at the current moment and the spatial states at all previous moments) in the order of timestamps. The connection method is to connect the three-dimensional spatial coordinates at each moment by connecting them with line segments to form an ordered trajectory sequence that can completely reflect the movement process of the target object, and finally obtain continuous spatial trajectory data.

[0071] The embodiments of the present invention optimize the stability of multi-view tracking, effectively avoid the problems of trajectory breakage and positioning failure caused by changes in the target's motion state, provide high-quality trajectory data support for subsequent boundary risk assessment, improve the dynamic collaborative perception capability of target motion state and spatial position in complex operating environments, reduce the probability of false alarms and missed alarms, and improve the reliability of substation safety management and control.

[0072] Optionally, the processes of steps 301 to 305 include: Step 301: Based on the spatial coordinates of the target object in the three-dimensional space at each moment in the continuous spatial trajectory data and the three-dimensional spatial boundary of the preset safety fence area, determine the spatial position relationship of the target object relative to the preset safety fence area at each moment.

[0073] Optionally, the safety control device analyzes the three-dimensional spatial coordinates of the target object at each moment in the continuous spatial trajectory data, extracting the corresponding X-axis, Y-axis, and Z-axis coordinate values. Then, it compares each axial coordinate value with the extreme values ​​of the three-dimensional spatial boundaries of the corresponding axes of the preset safety fence area. The spatial positional relationship of the target object relative to the preset safety fence area at each moment is determined through the comparison results. The spatial positional relationship is a determination result used to characterize the relative orientation of the target object and the fence area. Optionally, the determination result in this embodiment of the invention specifically includes: The first determination result is: the coordinate values ​​of each axis of the target object are all between the extreme values ​​of the corresponding axis fence boundary, that is, the X-axis coordinate value is between the minimum and maximum values ​​of the fence X-axis, the Y-axis coordinate value is between the minimum and maximum values ​​of the fence Y-axis, and the Z-axis coordinate value is between the minimum and maximum values ​​of the fence Z-axis. At this time, the spatial position relationship is completely located inside the area. The second determination result is: if the target object has at least one axial coordinate value that exceeds the extreme value of the corresponding axial fence boundary, then the spatial position relationship is that it is completely located outside the area. The third determination result is: some axial coordinate values ​​of the target object are between the boundary extreme values, and some axial coordinate values ​​coincide with the boundary extreme values. In this case, the spatial position relationship is that it is located at the boundary of the region.

[0074] Step 302: Based on the spatial positional relationship of the target object relative to the preset safety fence area at each moment, determine whether the target object is located inside the preset safety fence area at each moment, and obtain the spatial compliance judgment result at each moment.

[0075] Optionally, the safety control device, based on spatial location relationships and a preset area attribution determination rule, determines whether the target object is located within the preset safety fence area at each moment, generating a spatial compliance determination result for each moment. Therefore, the spatial compliance determination result is a binary determination conclusion characterizing whether the target object's position at that moment meets the fence area control requirements, including only two results: "located within the area" and "not located within the area." Optionally, the preset area attribution determination rule in this embodiment of the invention is specifically as follows: if the target object's spatial location relationship at a certain moment is completely within the area, then the spatial compliance determination result for that moment is determined to be "located within the area"; if the target object's spatial location relationship at a certain moment is completely outside the area, or its spatial location relationship is at the area boundary, then the spatial compliance determination result for that moment is determined to be "not located within the area." During the determination process, the safety control device accurately matches the spatial location relationship at each moment, without omitting any data, while ensuring that the spatial compliance determination results corresponding to the same spatial location relationship remain consistent, avoiding determination contradictions.

[0076] Step 303: Based on the spatial compliance judgment result at each moment and the timestamp information of the corresponding moment, determine the spatial compliance sequence arranged in chronological order.

[0077] Optionally, the security control device sorts and verifies the timestamp information for all moments, organizing the data from earliest to latest time, and removing abnormal data such as duplicate timestamps and time jumps to ensure the continuity and uniqueness of the time series. Then, the spatial compliance judgment result for each moment is linked and bound to the corresponding sorted timestamp information, forming a location-time data pair for each moment. Each data pair contains the precise time and the corresponding spatial compliance judgment result. Finally, all location-time data pairs are arranged sequentially from earliest to latest timestamps, forming a spatial compliance sequence arranged in chronological order. This spatial compliance sequence is an ordered data set that can fully reflect the continuous process of the target object's spatial compliance status changing over time throughout the entire tracking period.

[0078] Step 304: Based on the spatial compliance judgment results of a preset number of consecutive moments in the spatial compliance sequence, identify whether there is a spatial compliance judgment result at least once that is outside the preset safety fence area, and obtain the boundary crossing event result.

[0079] Optionally, the preset threshold for the number of consecutive moments is the minimum continuous time window pre-set by the safety control device for determining whether a valid boundary crossing event has been constituted. The value is determined according to the safety control accuracy requirements of the substation, and is usually set to 5 moments. The corresponding continuous time length within the tracking period matches the video stream frame rate (e.g., when the frame rate is 30 frames per second, 5 moments correspond to approximately 0.17 seconds).

[0080] Optionally, the safety control device uses a sliding window method to traverse and analyze the spatial compliance sequence. The length of the sliding window is a preset threshold for the number of consecutive moments. The window starts from the first moment of the spatial compliance sequence and slides forward one moment at a time until it covers the last moment of the sequence. The spatial compliance judgment results are then determined for each preset number of consecutive moments within each sliding window.

[0081] The safety control device checks each sliding window to see if there is at least one instance where the judgment result is "not within the area". If a sliding window has at least one instance where the judgment result is "not within the area", then that window is marked as having an out-of-bounds indication; if all sliding windows have judgment results of "within the area", then it is marked as having no out-of-bounds indication. Finally, based on the inspection results of all sliding windows, the out-of-bounds event occurrence result is obtained. The out-of-bounds event occurrence result is a judgment conclusion characterizing whether the target object has shown an out-of-bounds indication during the tracking period, specifically including two types: "out-of-bounds event exists" and "out-of-bounds event does not exist". Where "out-of-bounds event exists" corresponds to at least one sliding window showing an out-of-bounds indication, and "out-of-bounds event does not exist" corresponds to all sliding windows showing no out-of-bounds indication.

[0082] Step 305: Based on the result of the boundary crossing event, determine whether the target object continues to be located inside the preset safety fence area to obtain the final spatial state.

[0083] Optionally, the safety control device determines whether the target object remains within the preset safety fence area throughout the entire tracking period based on the results of the boundary crossing event and the overall distribution characteristics of the spatial compliance sequence.

[0084] For example, the safety control device can integrate the frequency of boundary crossing events, duration, and location distribution in the spatial compliance sequence to obtain a final spatial state that can comprehensively reflect the spatial location status of the target object, as described in steps 3051 to 3053. The final spatial state includes "continuously located within the area" and "not continuously located within the area".

[0085] The embodiments of the present invention effectively filter out interference caused by accidental position shifts, accurately identify whether the target object is continuously within the fenced area, improve the dynamic perception capability of the target spatial position in complex operating environments, reduce false alarms and missed alarms caused by inaccurate state judgment, and improve the reliability of substation safety management and control.

[0086] Optionally, the processes of steps 3051 to 3053 include: Step 3051: Based on the fact that all spatial compliance judgment results when the boundary crossing event occurs are all within the preset safety fence area, a continuous compliance confirmation result is obtained that the target object is continuously within the preset safety fence area during the observation period. Based on the continuous compliance confirmation result, a first spatial state indicating that the target object is currently in a compliant state is generated.

[0087] Optionally, the observation time period is the time range corresponding to the spatial compliance sequence, that is, the complete duration from the first time stamp to the last time stamp of the continuous spatial trajectory data. The duration is determined according to the tracking period and the video stream frame rate to ensure coverage of the entire time when the target object is continuously tracked.

[0088] When the boundary crossing event is determined not to have occurred, the safety control device performs a secondary verification of all spatial compliance judgment results in the spatial compliance sequence. It confirms that the judgment result at each moment is within the preset safety fence area, ensuring no data is missed during the verification process and eliminating interference from judgment results corresponding to abnormal timestamps. After verifying that all spatial compliance judgment results are within the preset safety fence area, a continuous compliance confirmation result is generated. This continuous compliance confirmation result is conclusive data indicating that the target object's position consistently meets the fence area control requirements throughout the entire observation period, clearly recording the start and end times of the observation period and the verification results for compliance at all moments.

[0089] Furthermore, based on the continuous compliance confirmation result, the safety control device generates a first spatial state. The first spatial state is a specific data carrier used to identify the current location status of the target object. It includes the observation time period, the core information of the continuous compliance confirmation result, and the three-dimensional spatial coordinates at the current moment, clearly indicating that the target object is currently in a compliant state, that is, it is still continuously located inside the preset safety fence area.

[0090] Step 3052: Based on the fact that the spatial compliance determination result at least once when the boundary crossing event has occurred is that the target object is located outside the preset safety fence area, a non-continuous compliance confirmation result is obtained that the target object is not continuously located inside the preset safety fence area during the observation period. Based on the non-continuous compliance confirmation result, a second spatial state is generated indicating that the target object is currently in a non-compliant state.

[0091] Optionally, when the boundary crossing event is determined to have occurred, the safety control device combines the spatial compliance sequence to extract the time data where the judgment result is not within the preset safety fence area, clarifies the timestamp information of such time and the corresponding spatial compliance judgment result, confirms that there is at least one time where the spatial compliance judgment result is outside the preset safety fence area, and records the number, distribution location and duration of such time. The distribution location is the time sorting position of such time in the spatial compliance sequence, and the duration is the total time span corresponding to multiple adjacent judgment results of being outside the area.

[0092] Furthermore, based on the above confirmation results, the safety control device generates non-continuous compliance confirmation results. The non-continuous compliance confirmation results are conclusive data that characterizes the location of the target object within the observation period as not meeting the requirements for fenced area control. They clearly record core information such as the start and end times of the observation period, the number of times the target object is located outside the area, its distribution location, and its duration, and intuitively reflect the specific situation of the target object not being continuously inside the fenced area.

[0093] Furthermore, based on the non-continuous compliance confirmation result, the safety control device generates a second spatial state. The second spatial state is a specific data carrier used to identify the current location status of the target object. It includes the observation time period, the core information of the non-continuous compliance confirmation result, the three-dimensional spatial coordinates of the current moment, and the time information of the last time the target object was outside the area. It clearly indicates that the target object is currently in a non-compliant state, that is, it has not been continuously located inside the preset safety fence area during the observation time period.

[0094] Step 3053: Determine the final spatial state based on the first spatial state and the second spatial state.

[0095] Optionally, the safety control device determines the final spatial state based on preset state priority rules. The final spatial state is a conclusion that comprehensively and accurately reflects the overall spatial location of the target object within the observation period. Optionally, the preset state priority rules in this embodiment are as follows: the first spatial state and the second spatial state are mutually exclusive; only one state can be generated within the same observation period, and there is no situation where both states are generated simultaneously. Therefore, the safety control device detects the state data type stored in the local database. If the first spatial state is stored, the final spatial state is determined to be the compliant state corresponding to the first spatial state, i.e., "continuously located within the preset safety fence area"; if the second spatial state is stored, the final spatial state is determined to be the non-compliant state corresponding to the second spatial state, i.e., "not continuously located within the preset safety fence area".

[0096] The embodiments of the present invention effectively filter out interference caused by occasional data fluctuations and verification deviations, accurately output the final state of whether the target object is continuously located within the fenced area, provide high-quality core data support for subsequent boundary risk assessment, improve the dynamic collaborative perception capability of target spatial location in complex operating environments, avoid potential risks of false alarms and missed alarms from the state assessment stage, reduce false alarms and missed alarms caused by inaccurate state assessment, and thus improve the reliability of substation safety management and control.

[0097] Optionally, the process of steps 306 to 309 includes: Step 306: Based on the spatial coordinates of the target object at a preset number of time points in the continuous spatial trajectory data when the final spatial state is compliant, determine whether the target object is continuously lingering near the boundary area within the preset safety fence area, and obtain the boundary lingering behavior recognition result.

[0098] Optionally, when the final spatial state is compliant, that is, the target object is continuously located inside the preset safety fence area during the observation period, the safety control device extracts the spatial coordinates of the target object at a preset number of times from the continuous spatial trajectory data. The preset number is the minimum number of time samples for behavior recognition pre-set by the safety control device, which is determined according to the substation safety control accuracy requirements and is usually set to 10 times. These 10 times are the latest times in the continuous spatial trajectory data that are closest to the current time, to ensure that the recent movement status of the target object is reflected.

[0099] Furthermore, the safety control device simultaneously extracts the three-dimensional spatial boundary of the preset safety fence area, calculates the area range after offsetting the boundary inward by a preset distance, and obtains the boundary region. The boundary region is a specific area inside the preset safety fence area near the boundary. The preset distance is a fixed value set based on the size of the fence area, usually set to 1 meter. The three-dimensional spatial range of the boundary region is defined by the new extreme value after offsetting the extreme values ​​of each axial boundary by 1 meter inward. Subsequently, it is determined whether the extracted spatial coordinates at a preset number of time points are all within the boundary region, and the total spatial displacement of the target object within the preset number of time points is calculated. The total spatial displacement is the straight-line distance between the spatial coordinates at the first time point and the last time point.

[0100] If the spatial coordinates at a preset number of time points are all within the boundary area, and the total spatial displacement is less than a preset displacement threshold (the preset displacement threshold is 0.5 meters), then it is determined that the target object is exhibiting a behavior of continuously wandering in the boundary area, and the boundary wandering behavior recognition result is "Boundary wandering behavior exists".

[0101] If the spatial coordinates at any given time are not within the boundary area, or the total spatial displacement is greater than or equal to the preset displacement threshold, then the target object is determined to have no boundary wandering behavior, and the boundary wandering behavior identification result is "no boundary wandering behavior exists". Therefore, the boundary wandering behavior identification result is conclusive data characterizing whether the target object has been continuously staying or slowly moving in the fence boundary area recently.

[0102] Step 307: Based on the timestamp information corresponding to the final non-compliant state and the spatial coordinates of the target object at adjacent times before and after the time point in the continuous spatial trajectory data, determine the spatial movement direction of the target object before and after the non-compliant state occurs.

[0103] Optionally, when the final spatial state is non-compliant, the safety control device extracts the timestamp information of the moment when the non-compliant state occurs in the second spatial state, that is, the timestamp corresponding to the moment when the target object's spatial compliance determination result is not located within the preset safety fence area, and simultaneously clarifies the sorting position of this timestamp in the spatial compliance sequence. Subsequently, the safety control device extracts the spatial coordinates of the target object at adjacent moments before and after this time point from the continuous spatial trajectory data. Adjacent moments before and after specifically refer to the moment before and after the moment when the non-compliant state occurs. If the moment when the non-compliant state occurs is the first moment in the spatial compliance sequence, then only the spatial coordinates of the next adjacent moment are extracted; if it is the last moment, then only the spatial coordinates of the previous adjacent moment are extracted.

[0104] Furthermore, the safety control device calculates the spatial coordinate difference between the moment the non-compliance state occurred and the immediately preceding moment to obtain a forward displacement vector. This forward displacement vector includes the position differences along the X, Y, and Z axes and their corresponding displacement directions. Similarly, it calculates the spatial coordinate difference between the moment the non-compliance state occurred and the immediately following moment to obtain a backward displacement vector, which also includes the position differences along the three axes and their displacement directions. Combining the forward and backward displacement vectors, the device integrates the target object's movement pattern before and after the non-compliance state occurred to determine the spatial movement direction. This spatial movement direction is the overall direction of movement of the target object during the period of the non-compliance state. The dominant direction of both displacement vectors is considered. If the directions of the forward and backward displacement vectors are consistent, that direction is used as the spatial movement direction; otherwise, the direction of the backward displacement vector from the moment the non-compliance state occurred to the immediately following moment is used as the spatial movement direction.

[0105] Step 308: Based on the three-dimensional spatial boundary of the preset safety fence area, determine whether the spatial movement direction points to the outside of the safety fence area, obtain the boundary crossing direction judgment result, and based on the boundary crossing direction judgment result, generate risk warning information that the target object has a tendency to cross the boundary.

[0106] Optionally, the safety control device analyzes the component directions of spatial movement along the X, Y, and Z axes one by one. For each axis, it determines whether the component direction of that axis points towards the outside of the fence corresponding to that axis. The specific process of this embodiment is as follows: If the X-axis component points in the direction of increasing X-axis coordinates, and this direction corresponds to the maximum boundary extreme value of the fence on the X-axis, then the X-axis component is determined to point outside the fence; if the X-axis component points in the direction of decreasing X-axis coordinates, and this direction corresponds to the minimum boundary extreme value of the fence on the X-axis, then the X-axis component is determined to point outside the fence; the Y-axis and Z-axis directions are determined in the same way.

[0107] If at least one axial component points outward from the fence, and this axial direction is the one that caused the target object to exceed the boundary when it was in a non-compliant state, then the boundary directional judgment result is "the spatial movement direction points outward from the safety fence area"; if all axial components point inward from the fence, or the axial direction pointing outward is not the one that caused the target object to exceed the boundary when it was in a non-compliant state, then the boundary directional judgment result is "the spatial movement direction does not point outward from the safety fence area".

[0108] Furthermore, based on the boundary crossing direction judgment result and combined with the details of the non-compliance status, risk warning information is generated, as in steps 3081 to 3084. The risk warning information is warning data that indicates whether the target object has a further tendency to cross the boundary, and clearly records the boundary crossing direction judgment result and the corresponding movement direction information.

[0109] Step 309: Based on the boundary wandering behavior identification result indicating the existence of boundary wandering behavior or the risk warning information indicating the existence of boundary crossing behavior trend, generate a boundary crossing risk judgment result indicating that the target object has a boundary crossing behavior trend.

[0110] Optionally, the safety control device generates a boundary crossing risk assessment result according to a preset risk assessment rule. The boundary crossing risk assessment result is the final conclusion characterizing whether the target object has a tendency to cross the boundary outside the preset safety fence area. For example, the preset risk assessment rule in this embodiment of the invention is as follows: as long as the boundary wandering behavior identification result is "there is boundary wandering behavior", regardless of the status of the risk warning information, it is determined that the target object has a tendency to cross the boundary; if the boundary wandering behavior identification result is "there is no boundary wandering behavior", it is further determined whether the risk warning information is "there is a tendency to cross the boundary". If so, it is determined that there is a tendency to cross the boundary; otherwise, it is determined that there is no tendency to cross the boundary. Based on the above rules, if any condition for the existence of a tendency to cross the boundary is met, the safety control device generates a boundary crossing risk assessment result indicating that the target object has a tendency to cross the boundary; if none of the conditions are met, a boundary crossing risk assessment result indicating that the target object does not have a tendency to cross the boundary is generated. At the same time, this result is associated and bound with the boundary wandering behavior identification result and the risk warning information to form a complete risk assessment data package.

[0111] This invention, through precise analysis of target behavior and motion characteristics in different scenarios, extends the prediction from "exceeding the boundary" to "potential boundary crossing trends," improving the dynamic collaborative perception capability of target motion status and spatial position in complex operating environments. It effectively reduces frequent false alarms and missed alarms caused by inaccurate trend prediction, provides accurate basis for subsequent alarm operations, and enhances the reliability of substation safety management.

[0112] Optionally, the process of steps 3081 to 3084 includes: Step 3081: Based on the boundary crossing direction judgment result that the target object is pointing to the outside of the safety fence area, extract the spatial displacement change characteristics of the target object toward the safety fence boundary within a consecutive preset number of time moments to obtain the boundary crossing approach trend characteristics.

[0113] Optionally, when the boundary crossing direction determination result indicates that the target object is pointing outside the safety fence area, the safety control device extracts the three-dimensional spatial coordinate sequence of the target object within the corresponding time period. The three-dimensional spatial coordinate sequence is a set of spatial coordinates arranged in chronological order from early to late according to the timestamps. Specifically, it includes spatial coordinates of a preset number of times before the non-compliant state occurs, the time when the non-compliant state occurs, and a preset number of times after the time when the non-compliant state occurs. The preset number is a pre-set minimum sample size of times used to extract trend features, which is set to 5 times according to the control accuracy requirements to ensure coverage of the complete time period during which the target object moves towards the outside of the fence.

[0114] Optionally, the safety control device determines the specific location of the fence boundary that the target object is currently facing, i.e., the maximum or minimum boundary extreme value of the corresponding axis, based on the three-dimensional spatial boundary of the preset safety fence area.

[0115] Furthermore, the safety control device calculates the spatial displacement data of two adjacent moments in the three-dimensional spatial coordinate sequence one by one. The spatial displacement data includes the displacement distance and displacement direction in three directions: X-axis, Y-axis, and Z-axis. The displacement distance is the absolute value of the difference between the spatial coordinates at corresponding axes between two adjacent moments, and the displacement direction is the direction in which the coordinate value increases or decreases. Optionally, by integrating the displacement data of each axis, the spatial displacement change characteristics of the target object towards the safety fence boundary within a consecutive preset number of moments are extracted to obtain the boundary approach trend characteristics. The boundary approach trend characteristics are a comprehensive set of features characterizing the changes in the magnitude and direction of displacement over time when the target object moves towards the outside of the fence. The core features include the magnitude of the displacement distance change at each moment and the degree of alignment between the displacement direction and the direction pointing outwards from the fence.

[0116] Step 3082: Based on the consistency of the spatial displacement change direction at consecutive moments in the boundary approach trend characteristics, determine whether the target object exhibits a behavior pattern of continuously moving towards the outside of the safety fence boundary, and obtain a confirmation result of continuous outward movement behavior.

[0117] Optionally, the safety control device compares the displacement direction of adjacent moments with the external direction of the preset safety fence area one by one. First, it determines the axial displacement direction corresponding to the external direction of the fence that the target object is facing. For example, when facing the outside of the maximum boundary of the X-axis, the corresponding X-axis displacement direction is the direction of increasing coordinate value; when facing the outside of the minimum boundary of the Y-axis, the corresponding Y-axis displacement direction is the direction of decreasing coordinate value.

[0118] Furthermore, the number of times within a consecutive preset number of time intervals where the displacement direction is consistent with the direction outside the fence is calculated, and the consistency ratio is the ratio of the number of times the direction is consistent to the total preset number. Simultaneously, the continuity of the displacement direction between adjacent time intervals is determined, i.e., whether the displacement direction of the later time interval is consistent with the displacement direction of the previous time interval, without any reverse fluctuation.

[0119] If the consistency rate reaches a preset threshold (the preset threshold is 80%), and there is no reverse fluctuation in the displacement direction between adjacent time points, or the reverse fluctuation time does not exceed 1 time, then it is determined that the target object exhibits a behavior pattern of continuously moving towards the outside of the safety fence boundary, and a continuous outward movement behavior confirmation result is generated as "continuous outward movement behavior exists"; if the consistency rate is lower than the preset threshold, or the reverse fluctuation time exceeds 1 time, then it is determined that there is no such behavior pattern, and a continuous outward movement behavior confirmation result is generated as "no continuous outward movement behavior exists". Therefore, the continuous outward movement behavior confirmation result is a conclusive data characterizing the stability of the target object's movement direction.

[0120] Step 3083: Based on the confirmation results of continuous outward movement and the minimum distance between the current spatial coordinates of the target object and the boundary of the safety fence, proximity information is obtained.

[0121] Optionally, the safety control device calculates the minimum distance between the current spatial coordinates of the target object and the boundary of the safety fence. The minimum distance is the absolute value of the difference between the current coordinates and the extreme value of the corresponding fence boundary in the direction towards the outside of the fence. For example, if the current coordinate X-axis value is 59 meters, the minimum distance is 1 meter when moving towards the maximum extreme value of the X-axis boundary of 60 meters. If the object moves towards multiple axial boundaries at the same time, the minimum value among the minimum distances of each axis is taken as the final minimum distance.

[0122] Furthermore, based on the confirmation result of continuous outward movement and the aforementioned minimum distance, proximity information is generated. This proximity information is comprehensive data characterizing the distance and movement pattern between the target object and the fence boundary. Optionally, the proximity information in this embodiment includes: If the confirmation result of continuous outward movement is "continuous outward movement exists", and the minimum distance is less than or equal to the preset distance level 1 (the preset distance level 1 is 1 meter), then the proximity information is "continuous outward movement at close range"; if the continuous outward movement exists, and the minimum distance is greater than 1 meter and less than or equal to the preset distance level 2 (the preset distance level 2 is 3 meters), then the proximity information is "continuous outward movement at medium distance"; if the confirmation result of continuous outward movement is "no continuous outward movement exists", then regardless of the minimum distance, the proximity information is "no continuous outward movement trend". Therefore, the proximity information clearly quantifies the proximity status and movement association characteristics between the target object and the fence boundary.

[0123] Step 3084: Based on the proximity information being less than the preset safe distance threshold and the confirmation result of continuous outward movement, generate risk warning information.

[0124] Optionally, the preset safety distance threshold is a critical distance value pre-set by the safety control device to trigger risk warnings. It is set to 1.5 meters according to the substation fence protection requirements. The preset safety distance threshold is lower than the boundary area offset distance to ensure early warning of boundary crossing risks.

[0125] The safety control device compares the minimum distance corresponding to the proximity information with a preset safe distance threshold, and simultaneously verifies the confirmation result of continuous outward movement. If the minimum distance is less than the preset safe distance threshold, and the confirmation result of continuous outward movement is "continuous outward movement exists," then the target object is determined to have a high risk of further boundary crossing. A risk warning message is generated according to the preset risk warning generation rules. The risk warning message includes warning data such as the target object's current spatial coordinates, minimum distance, characteristics of continuous outward movement, boundary crossing risk level (high risk), and the corresponding fence boundary position, clearly informing the target object of a tendency to continuously approach the outside of the fence. If the minimum distance is greater than or equal to the preset safe distance threshold, or the confirmation result of continuous outward movement is "no continuous outward movement exists," then no risk warning message indicating a tendency to cross the boundary is generated; only the current state is recorded to ensure the accuracy of the risk warning and avoid false triggering.

[0126] The embodiments of the present invention accurately capture the trend characteristics and risk level of the target object moving towards the outside of the fence, improve the dynamic collaborative perception capability of the target movement state in complex working environments, optimize the risk identification accuracy from the trend prediction stage, effectively reduce the frequent false alarms and missed alarms caused by trend misjudgment, provide high-quality basis for subsequent safety alarm operations, and thus improve the reliability of substation safety management and control.

[0127] Furthermore, the substation safety management and control device based on camera tracking provided by the present invention will be described below. The substation safety management and control device based on camera tracking described below can be referred to in correspondence with the substation safety management and control method based on camera tracking described above.

[0128] Optionally, refer to Figure 2 , Figure 2 This is a schematic diagram of the substation safety management and control device based on camera tracking provided by the present invention. The substation safety management and control device based on camera tracking includes: The multi-view fusion module 210 is used to capture target objects entering the preset safety fence area based on multiple rotatable cameras deployed at different locations in the substation, obtain spatial location information, and control at least two rotatable cameras to synchronously turn and lock the target object based on the spatial location information, thereby obtaining a first multi-view real-time video stream. The motion trajectory tracking module 220 is used to continuously track the motion trajectory of the target object within the preset safety fence area based on the first multi-view real-time video stream, and obtain continuous spatial trajectory data. The boundary crossing behavior monitoring module 230 is used to determine whether the target object is continuously located inside the preset safety fence area based on continuous spatial trajectory data, obtain the final spatial state, and determine whether the target object has a boundary crossing behavior trend based on the final spatial state, and obtain the boundary crossing risk judgment result. The safety control response module 240 is used to control the substation safety alarm device to perform corresponding alarm operations or alarm cancellation operations based on the boundary risk assessment results, and obtain safety control response instructions.

[0129] The embodiments of the present invention solve the problems of continuous target tracking and frequent false alarms and missed alarms caused by the fixed camera's fixed viewing angle. It improves the dynamic collaborative perception capability of the target's motion state and spatial position, reduces the false alarm rate and missed alarm rate of safety alarms, and improves the reliability of substation safety management.

[0130] Please see Figure 3 , Figure 3 An embodiment diagram of an electronic device provided in accordance with the present invention. For example... Figure 3 As shown, an embodiment of the present invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it implements the processes of steps 10 to 40.

[0131] Please see Figure 4 , Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with an embodiment of the present invention is shown. Figure 4 As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 311 is stored. When the computer program 311 is executed by a processor, it implements the processes of steps 10 to 40.

[0132] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the substation safety management method based on camera tracking provided by the above methods, which includes steps 10 to 40.

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A substation safety management and control method based on camera tracking, characterized in that, include: Based on the rotatable cameras deployed at multiple different locations in the substation, the target object entering the preset safety fence area is captured to obtain spatial location information. Based on the spatial location information, at least two rotatable cameras are controlled to turn synchronously and lock onto the target object to obtain a first multi-view real-time video stream. Based on the first multi-view real-time video stream, the motion trajectory of the target object within the preset safety fence area is continuously tracked to obtain continuous spatial trajectory data. Based on the continuous spatial trajectory data, it is determined whether the target object is continuously located inside the preset safety fence area to obtain the final spatial state. Based on the final spatial state, it is determined whether the target object has a tendency to cross the boundary to obtain the boundary risk assessment result. Based on the boundary risk assessment result, the substation safety alarm device is controlled to perform the corresponding alarm operation or alarm cancellation operation to obtain a safety control response command.

2. The substation safety management method based on camera tracking according to claim 1, characterized in that, The steps for determining the boundary risk assessment result include: Based on the spatial coordinates of the target object at a preset number of time moments in the continuous spatial trajectory data when the final spatial state is compliant, it is determined whether the target object is continuously lingering near the boundary area within the preset safety fence area, and the boundary lingering behavior recognition result is obtained. Based on the timestamp information corresponding to the final non-compliant state and the spatial coordinates of the target object at adjacent times before and after the time point in the continuous spatial trajectory data, the spatial movement direction of the target object before and after the non-compliant state occurs is determined. Based on the three-dimensional spatial boundary of the preset safety fence area, it is determined whether the spatial movement direction points to the outside of the safety fence area, and the boundary crossing direction judgment result is obtained. Based on the boundary crossing direction judgment result, a risk warning message is generated that the target object has a tendency to cross the boundary. Based on the boundary wandering behavior identification result indicating the existence of boundary wandering behavior or the risk warning information indicating the existence of boundary crossing behavior trend, a boundary crossing risk judgment result is generated indicating that the target object has a boundary crossing behavior trend.

3. The substation safety management method based on camera tracking according to claim 2, characterized in that, The step of generating risk warning information about the target object's tendency to cross boundaries based on the boundary crossing direction judgment result includes: Based on the boundary directional judgment result that the target object is pointing to the outside of the safety fence area, the spatial displacement change characteristics of the target object toward the safety fence boundary within a continuous preset number of time moments are extracted to obtain the boundary approach trend characteristics. Based on the consistency of the spatial displacement change direction at consecutive moments in the boundary approach trend characteristics, it is determined whether the target object exhibits a behavior pattern of continuously moving towards the outside of the safety fence boundary, and a continuous outward movement behavior confirmation result is obtained. Based on the confirmation result of the continuous outward movement behavior and the minimum distance between the current spatial coordinates of the target object and the boundary of the safety fence, proximity information is obtained; The risk warning information is generated based on the proximity information being less than a preset safe distance threshold and the confirmation of continuous outward movement.

4. The substation safety management method based on camera tracking according to claim 1, characterized in that, The step of determining whether the target object remains continuously within the preset safety fence area based on the continuous spatial trajectory data to obtain the final spatial state includes: Based on the spatial coordinates of the target object in three-dimensional space and the three-dimensional spatial boundary of the preset safety fence area at each moment in the continuous spatial trajectory data, the spatial position relationship of the target object relative to the preset safety fence area at each moment is determined. Based on the spatial positional relationship of the target object relative to the preset safety fence area at each moment, it is determined whether the target object is located inside the preset safety fence area at each moment, and the spatial compliance determination result at each moment is obtained. Based on the spatial compliance judgment result at each moment and the corresponding timestamp information, a spatial compliance sequence arranged in chronological order is determined. Based on the spatial compliance judgment results of a preset number of consecutive moments in the spatial compliance sequence, identify whether there is at least one moment where the spatial compliance judgment result is outside the preset safety fence area, and obtain the boundary crossing event result. Based on the result of the boundary crossing event, it is determined whether the target object remains within the preset safety fence area, thus obtaining the final spatial state.

5. The substation safety management method based on camera tracking according to claim 4, characterized in that, The step of determining whether the target object remains within the preset safety fence area based on the result of the boundary crossing event, and obtaining the final spatial state, includes: Based on the fact that all spatial compliance judgment results when the boundary crossing event does not occur are all within the preset safety fence area, a continuous compliance confirmation result is obtained that the target object is continuously within the preset safety fence area during the observation period, and a first spatial state indicating that the target object is currently in a compliant state is generated based on the continuous compliance confirmation result. Based on the fact that the spatial compliance determination result at least once when the boundary crossing event has occurred is that the target object is located outside the preset safety fence area, a non-continuous compliance confirmation result is obtained that the target object is not continuously located inside the preset safety fence area during the observation period. Based on the non-continuous compliance confirmation result, a second spatial state indicating that the target object is currently in a non-compliant state is generated. The final spatial state is determined based on the first spatial state and the second spatial state.

6. The substation safety management and control method based on camera tracking according to claim 1, characterized in that, The step of continuously tracking the motion trajectory of the target object within the preset safety fence area based on the first multi-view real-time video stream to obtain continuous spatial trajectory data includes: Based on the visual feature information of the target object contained in each frame of the first multi-view real-time video stream and the spatial orientation parameters of the corresponding rotatable camera, the two-dimensional image coordinate position of the target object under each view is determined. Based on the two-dimensional image coordinates of the target object from various perspectives and the spatial installation position and orientation parameters of each rotatable camera, the spatial coordinates of the target object in three-dimensional space are determined. Based on the spatial coordinates of the target object in three-dimensional space and the timestamp information of the first multi-view real-time video stream corresponding to the current frame, the first spatial state of the target object at the current moment is determined. Based on the first spatial state at the current moment and the second spatial state at the previous moment, determine the spatial displacement vector of the target object between adjacent moments; The continuous spatial trajectory data is obtained by continuously tracking the spatial displacement vector and the first spatial state at the current moment.

7. The substation safety management method based on camera tracking according to claim 6, characterized in that, The continuous tracking based on the spatial displacement vector and the first spatial state at the current moment to obtain the continuous spatial trajectory data includes: Based on the spatial displacement vector and the first spatial state at the current moment, the movement direction and movement continuity judgment result of the target object within the preset safety fence area are determined; Based on the motion direction and the motion continuity judgment result, the spatial neighborhood of the target object appearing in the next moment is limited to obtain the spatial search constraint region of the target object; Based on the spatial search constraint region and the second multi-view real-time video stream at the next moment, the two-dimensional image coordinate position of the target object at the next moment is matched and located to obtain the updated two-dimensional image coordinate position of the target object at the next moment. Based on the updated two-dimensional image coordinates, the spatial orientation parameters of the corresponding rotatable camera, and the spatial installation position, the updated spatial coordinates of the target object in three-dimensional space at the next moment are determined, and the updated spatial coordinates are sequentially connected with the historical spatial state to obtain the continuous spatial trajectory data.

8. A substation safety management and control device based on camera tracking, characterized in that, Used to implement the substation safety management and control method based on camera tracking as described in any one of claims 1 to 7; The substation safety management and control device based on camera tracking includes: The multi-view fusion module is used to capture target objects entering a preset safety fence area based on rotatable cameras deployed at multiple different locations in the substation, obtain spatial location information, and control at least two rotatable cameras to synchronously turn and lock the target object based on the spatial location information, thereby obtaining a first multi-view real-time video stream. The motion trajectory tracking module is used to continuously track the motion trajectory of the target object within the preset safety fence area based on the first multi-view real-time video stream, and obtain continuous spatial trajectory data. The boundary crossing behavior monitoring module is used to determine whether the target object is continuously located inside the preset safety fence area based on the continuous spatial trajectory data, to obtain the final spatial state, and to determine whether the target object has a boundary crossing behavior trend based on the final spatial state, so as to obtain the boundary crossing risk judgment result. The safety control response module is used to control the substation safety alarm device to perform corresponding alarm operations or alarm cancellation operations based on the boundary risk assessment result, and obtain safety control response instructions.

9. An electronic device, comprising: Memory, used to store computer software programs; A processor for reading and executing the computer software program, characterized in that, when the processor executes the computer software program, it implements the substation safety management method based on camera tracking as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, wherein a computer software program is stored therein, characterized in that, When the computer software program is executed by the processor, it implements the substation safety management method based on camera tracking as described in any one of claims 1 to 7.

Images