Transformer substation construction safety distance monitoring method, device and equipment and storage medium

Abstract

According to the transformer substation construction safety distance monitoring method, device and equipment and the storage medium provided by the embodiment of the invention, the three-dimensional target detection model is established through the obtained point cloud data of the transformer substation, and the first three-dimensional target detection is carried out based on the three-dimensional target detection model so as to detect whether the electrified equipment exists in the to-be-monitored area; and under the condition of determining that the electrified equipment exists in the to-be-monitored area, obtaining three-dimensional position information corresponding to the electrified equipment, dividing a risk area based on the three-dimensional position information, and then performing secondary three-dimensional target detection based on the three-dimensional target detection model so as to detect whether an invader exists in the risk area. When the condition that an invader exists in the risk area is determined, the type of the invader is further judged, namely whether the invader is an invader or a construction machine is judged, and finally the Euclidean distance between the electrified equipment and the invader is calculated based on the boundary information; and if the Euclidean distance exceeds a preset distance threshold, outputting alarm information based on the category information.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of safety monitoring, and relate to but are not limited to a method, device, equipment and storage medium for monitoring safety distance during construction of a substation. Background Art

[0002] The direct cause of electric shock accidents in substations is often the lack of safe distance between construction vehicles, live equipment, and workers. At present, safe distance monitoring mostly uses a combination of cameras and lidar. The camera image is used for two-dimensional target detection, and the corresponding relationship between the camera and the lidar is established to generate a cone of vision. The three-dimensional information of the target is obtained inside the cone of vision for distance measurement. However, the image lacks depth information, and detection and positioning inside the cone of vision are often limited. In addition, the camera is sensitive to light and angle, which can easily affect the detection results.

[0003] At present, most of the existing substation safety distance monitoring methods use manual monitoring. However, the manual monitoring solution is not only time-consuming and labor-intensive, but also has poor reliability, stability and accuracy. In addition, for personnel or construction machinery that mistakenly enter the risk area of ​​the substation, it is impossible to issue an alarm in time, which poses a huge threat to the safety of the staff, resulting in uncertainty and danger in power safety management. Summary of the invention

[0004] Based on the problems in the related art, the embodiments of the present invention provide a substation construction safety distance monitoring method, device, equipment and storage medium.

[0005] The technical solution of the embodiment of the present invention is achieved as follows:

[0006] An embodiment of the present invention provides a method for monitoring a safe distance in substation construction, the method comprising:

[0007] Obtain point cloud data and image data of the substation;

[0008] Based on the point cloud data, a three-dimensional target detection model is established;

[0009] Based on the point cloud data and the three-dimensional target detection model, detecting whether there is a live device in the area to be monitored;

[0010] If the energized device exists in the area to be monitored, determining the risk area according to the acquired three-dimensional position information of the energized device;

[0011] Based on the three-dimensional target detection model, three-dimensional target detection is performed on the risk area to obtain a detection result; the detection result includes whether there is an intruder in the risk area, and the category information and boundary information of the intruder;

[0012] Based on the boundary information, determining a Euclidean distance between the powered device and the intruder;

[0013] If the Euclidean distance exceeds a preset distance threshold, an alarm message is output based on the category information.

[0014] An embodiment of the present invention provides a substation construction safety distance monitoring device, the device comprising:

[0015] An acquisition module, used to acquire point cloud data and image data of the substation;

[0016] An establishment module is used to establish a three-dimensional target detection model based on the point cloud data;

[0017] A detection module, used to detect whether there is a live device in the area to be monitored based on the point cloud data and the three-dimensional target detection model;

[0018] A determination module, configured to determine a risk area based on the acquired three-dimensional position information of the energized device if the energized device exists in the area to be monitored;

[0019] The detection module is further used to perform three-dimensional target detection on the risk area based on the three-dimensional target detection model to obtain a detection result; the detection result includes whether there is an intruder in the risk area, and the category information and boundary information of the intruder;

[0020] The determination module is further used to determine the Euclidean distance between the charged device and the intruder based on the boundary information;

[0021] The output module is used to output warning information based on the category information if the Euclidean distance exceeds a preset distance threshold.

[0022] In some embodiments, before acquiring the point cloud data and image data of the substation, the device also includes: a synchronization module for jointly calibrating the laser radar and the camera; based on a preset synchronization method, performing time soft synchronization on the jointly calibrated laser radar and the camera.

[0023] In some embodiments, the synchronization module is also used to determine the coordinate transformation matrix between the lidar coordinate system and the camera coordinate system using a preset joint calibration method; establish a correspondence between each corner point in the three-dimensional space and the projection point of the corner point on the two-dimensional image; calibrate the camera intrinsic parameters using the principle of nonlinear optimization to obtain the camera intrinsic parameter matrix; determine the camera extrinsic parameter matrix based on the camera intrinsic parameter matrix, the corresponding relationship and the camera scale factor.

[0024] In some embodiments, the establishment module is also used to introduce instance-aware and center-of-mass-aware downsampling into the original target detection model; perform sparse processing on the point cloud data to obtain sparse point cloud data; and train, evaluate and optimize the original target detection model based on the sparse point cloud data to obtain the three-dimensional target detection model.

[0025] In some embodiments, the determination module is also used to extract the target center of mass of the energized device based on the boundary information; calculate the corresponding distance from the target center of mass to each point cloud of the intruder; determine the corresponding point cloud with the minimum distance among multiple corresponding distances as the target point cloud; and calculate the Euclidean distance between the target point cloud and the target center of mass.

[0026] In some embodiments, the output module is also used to output a first alarm message if the Euclidean distance exceeds a preset distance threshold, and the intruder is close to the edge of the restricted area and the construction machinery deviates from the predetermined route. The first alarm message is used to remind the substation staff to observe the situation or the construction operator to immediately correct the machinery route; if the Euclidean distance exceeds the preset distance threshold, and the intruder crosses the boundary and enters the restricted area and the construction machinery enters the restricted area, a second alarm message is output. The second alarm message is used to notify the security personnel to rush to the scene to intervene immediately or the construction operator to stop working immediately; if the Euclidean distance exceeds the preset distance threshold, and multiple intruders are detected to enter the restricted area and the construction machinery operates abnormally, a third alarm message is output. The third alarm message is used to notify the security personnel to block the relevant area and request law enforcement agencies to intervene, and if necessary, turn off the power of the equipment to remove the abnormal operating machinery.

[0027] An embodiment of the present invention provides a substation construction safety distance monitoring device, comprising: a memory for storing executable instructions; a processor for implementing the above-mentioned substation construction safety distance monitoring method when executing the executable instructions stored in the memory.

[0028] An embodiment of the present invention provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned substation construction safety distance monitoring method.

[0029] The substation construction safety distance monitoring method, device, equipment and storage medium provided by the embodiments of the present invention establish a three-dimensional target detection model by acquiring point cloud data of the substation, and perform a first three-dimensional target detection based on the three-dimensional target detection model to detect whether there is a live device in the monitored area. When it is determined that there is a live device in the monitored area, the three-dimensional position information corresponding to the live device is obtained, and a risk area is divided based on the three-dimensional position information. Thereafter, a secondary three-dimensional target detection is performed based on the three-dimensional target detection model to detect whether there is an intruder in the risk area. When it is determined that there is an intruder in the risk area, the type of the intruder is further determined, that is, whether it is an intruder or a construction machine. Finally, based on the boundary information, the Euclidean distance between the live device and the intruder is calculated; if the Euclidean distance exceeds the preset distance threshold, an alarm message is output based on the category information. In this way, the present invention reduces the monitored area of ​​a larger range to a risk area of ​​a smaller range through the first three-dimensional target detection, thereby detecting intruders in the risk area, which improves the accuracy of target detection to a certain extent. Through the second three-dimensional target detection, the type of intruder and boundary information are determined, which improves the accuracy of target detection to a certain extent and avoids the existence of false detection and missed detection. In addition, according to the determined category information, different levels of alarm information are output to provide managers with more comprehensive and accurate alarm information. While improving the safety management and control capabilities of substation operations, it also improves the accuracy and precision of safety monitoring, reduces the risks of substation operators and the failure risks of substation construction machinery. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of a method for monitoring safe distance of substation construction provided by an embodiment of the present invention;

[0031] Figure 2 It is a schematic diagram of a process of monitoring the safe distance of substation construction based on three-dimensional target detection provided by an embodiment of the present invention;

[0032] Figure 3 It is a structural schematic diagram of a three-dimensional target detection model for substation construction safety distance monitoring provided by an embodiment of the present invention;

[0033] Figure 4 It is a schematic diagram of the safety distance calculation principle of the substation construction safety distance monitoring provided by the embodiment of the present invention;

[0034] Figure 5 A schematic diagram of the structure of a substation construction safety distance monitoring device provided by an embodiment of the present invention;

[0035] Figure 6 A schematic diagram of the structure of a substation construction safety distance monitoring device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0037] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present invention have the same meaning as those commonly understood by those skilled in the art to which the embodiments of the present invention pertain. The terms used in the embodiments of the present invention are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0038] The following describes an exemplary application of the substation construction safety distance monitoring device of an embodiment of the present invention. The substation construction safety distance monitoring device provided by the embodiment of the present invention can be implemented as a terminal or a server. In one implementation, the substation construction safety distance monitoring device provided by the embodiment of the present invention can be implemented as various types of terminals such as laptops, tablet computers, desktop computers, mobile devices, etc.; in another implementation, the substation construction safety distance monitoring device provided by the embodiment of the present invention can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present invention. Below, an exemplary application of the substation construction safety distance monitoring device when it is implemented as a server will be described.

[0039] The embodiment of the present invention provides a substation construction safety distance monitoring method, see Figure 1 , Figure 1 is a flow chart of a substation construction safety distance monitoring method provided by an embodiment of the present invention, which will be combined with Figure 1 The steps shown are explained.

[0040] Step S110, acquiring point cloud data and image data of the substation.

[0041] In some embodiments, point cloud data is acquired through lidar scanning, by emitting laser pulse signals to the object to be scanned and receiving reflected signals, and finally converting them into three-dimensional point cloud data information that can be identified and processed.

[0042] These point cloud data may include the following categories of data: people, aerial work vehicles, cranes, insulators, grading rings, bushings, and disconnectors. Here, the laser radar may be a low-cost, high-efficiency, non-repetitive scanning solid-state laser radar. Of course, the present invention may also use other types of laser radars as long as they can realize the collection of point cloud data. Therefore, the embodiments of the present invention do not specifically limit this.

[0043] In some embodiments, the image data is collected by an image acquisition device. Here, the image acquisition device may be a camera, and the camera may be a high-resolution CMOS industrial camera. Of course, the present invention may also use other types of cameras as long as they can achieve the collection of image data. Therefore, the embodiments of the present invention do not specifically limit this.

[0044] Step S120: establishing a three-dimensional target detection model based on the point cloud data.

[0045] In some embodiments, the three-dimensional target detection model refers to the optimal target detection model obtained by training, evaluating and optimizing the original target detection model through the obtained training set point cloud data. Here, the original target detection model is constructed by a point-based single-stage 3D target detection method. The three-dimensional target detection model refers to a target detection model that adds instance perception and centroid perception downsampling modules to the original target detection model. The three-dimensional target detection model can reduce the point cloud computing cost based on the complex background of the substation.

[0046] Step S130: Detect whether there is a powered device in the area to be monitored based on the point cloud data and the three-dimensional target detection model.

[0047] In the present invention, the point cloud data after sparse processing can be input into the three-dimensional target detection model to detect whether there is a live device in the point cloud data.

[0048] Step S140: If the powered device exists in the area to be monitored, determine the risk area according to the acquired three-dimensional position information of the powered device.

[0049] In some embodiments, three-dimensional position information refers to describing and locating objects or points in space through a three-dimensional coordinate system. This information not only includes the position of the object on a two-dimensional plane (such as the X-axis and Y-axis), but also adds height or depth information (such as the Z-axis), thereby providing a more comprehensive and accurate spatial positioning capability.

[0050] In some embodiments, risk areas refer to areas where there are potential dangers or safety hazards that may have adverse effects on personnel, equipment or the environment. Risk areas in substation safety monitoring mainly include high voltage areas, oil-immersed equipment areas, electrical equipment concentration areas and other possible risk areas.

[0051] In the present invention, after determining that there are energized devices in the area to be monitored, the three-dimensional position information of the energized devices can be obtained, and the risk area can be delineated with each energized device as the center.

[0052] Step S150, based on the three-dimensional target detection model, three-dimensional target detection is performed on the risk area to obtain a detection result; the detection result includes whether there is an intruder in the risk area, and the category information and boundary information of the intruder.

[0053] In some embodiments, the category information refers to information defining the types of intrusion objects. The categories of intrusion objects may include: intruders and construction machinery.

[0054] In some embodiments, the boundary information refers to information related to the bounding box of the intruder. The bounding box is usually used to indicate the position and size of an object in an image. In the present invention, each detected object is marked with a bounding box. The bounding box can be a rectangle (two-dimensional) or a cube (three-dimensional). In a two-dimensional image, the bounding box is usually defined by four parameters: the coordinates of the upper left corner or the lower right corner (x, y), and the width (w) and height (h) of the bounding box.

[0055] In the present invention, three-dimensional target detection can be performed on the risk area based on the three-dimensional target detection model to detect whether there is an intruder in the risk area. When it is determined that there is an intruder in the risk area, the category information and boundary information of the intruder are further obtained.

[0056] Step S160: determining the Euclidean distance between the powered device and the intruder based on the boundary information.

[0057] In some embodiments, Euclidean distance is used to represent straight line segment distance in two-dimensional image data and three-dimensional point cloud data.

[0058] In the present invention, the detection result of the three-dimensional target detection model in the above step S150 searches for the minimum distance point between the live equipment and the intruder or construction machinery in the risk area, and calculates the distance of the minimum distance point to obtain the Euclidean distance.

[0059] Step S170: if the Euclidean distance exceeds a preset distance threshold, outputting a warning message based on the category information.

[0060] In some embodiments, the preset distance threshold is a distance threshold preset based on experience or construction safety requirements. The preset distance threshold is used to determine whether the distance between the intruder and the energized device is safe. The preset distance threshold is used to compare with the Euclidean distance to determine whether to issue an early warning for the intruder.

[0061] In some embodiments, a corresponding safety distance threshold is set for each security level area. When an intruder enters these areas and approaches the preset safety distance threshold, the system should trigger an alarm. It should be noted that it can also include multi-level alarm responses: Primary alarm: When the intruder enters the safety distance threshold, the system issues a primary alarm to alert the substation staff. Emergency alarm: If the intruder continues to approach or perform dangerous behaviors, the system will issue an emergency alarm and automatically trigger a linkage mechanism, such as closing the access control, activating the sound and light alarm, etc.

[0062] In some embodiments, different safety distance thresholds are set for different types of construction machinery and substation equipment. When the construction machinery approaches these thresholds, the system should issue an alarm.

[0063] In some embodiments, before acquiring the point cloud data and image data of the substation, the method may further include:

[0064] First, the laser radar and the camera are jointly calibrated. Second, based on a preset synchronization method, the laser radar and the camera after the joint calibration are soft-synchronized in time.

[0065] In some embodiments, joint calibration refers to automatically calibrating the laser radar and the camera. In the present invention, the ACSC joint calibration method can be used to perform spatial transformation calculations from the laser radar coordinate system to the camera coordinate system, and the corresponding 3D-2D corner points are extracted from the same frame data of the two sensors to solve the external parameter matrix.

[0066] In some embodiments, joint calibration can be performed by:

[0067] A preset joint calibration method is used to determine the coordinate transformation matrix between the laser radar coordinate system and the camera coordinate system; a corresponding relationship between each corner point in the three-dimensional space and the projection point of the corner point on the two-dimensional image is established; the camera intrinsic parameters are calibrated using the nonlinear optimization principle to obtain the camera intrinsic parameter matrix; the camera extrinsic parameter matrix is ​​determined based on the camera intrinsic parameter matrix, the corresponding relationship and the camera scale factor.

[0068] In some embodiments, the above step S120 can be implemented by following the steps S121 to S123:

[0069] Step S121, introducing instance-aware and centroid-aware downsampling into the original object detection model.

[0070] Step S122, performing sparse processing on the point cloud data to obtain sparse point cloud data.

[0071] Step S123: training, evaluating and optimizing the original target detection model according to the sparse point cloud data to obtain the three-dimensional target detection model.

[0072] In some embodiments, a point-based single-stage 3D target detection method is used to build an original target detection model, and instance-aware and centroid-aware downsampling modules are added to reduce the point cloud computing cost based on the complex background of the substation. The original target detection model is trained, evaluated and optimized using the obtained training set point cloud data to obtain a three-dimensional target detection model.

[0073] In some embodiments, time soft synchronization refers to the use of software algorithms to keep the timestamps of multiple devices or sensors consistent or nearly consistent without dedicated hardware synchronization devices. It usually involves the acquisition, calibration and synchronization of timestamps to ensure that each device can refer to a unified time reference when processing data.

[0074] In some embodiments, soft time synchronization between the lidar and the camera can be achieved through the following methods: unified clock source, software algorithm synchronization, offline synchronization, etc.

[0075] In practical applications, the implementation method of time soft synchronization generally includes the following steps: 1) Timestamp interception: Each device or sensor will be accompanied by a timestamp when generating data, indicating the precise time when the data was generated. 2) Time calibration: Obtain a unified time base through a certain mechanism (such as the Network Time Protocol NTP or the Precision Time Protocol PTP); compare the timestamp of each device or sensor with this unified time base to calculate the time deviation; adjust the local clock of each device or sensor according to the time deviation to eliminate or reduce the time deviation. 3) Synchronous processing: During the data processing process, sort, interpolate or resample the data according to the calibrated timestamp to ensure the synchronization of the data in time; for application scenarios that require real-time processing, software algorithms can be used to predict and compensate for time delays to achieve more accurate synchronization.

[0076] In some embodiments, the above step S160 can be implemented by the following steps S161 to S164:

[0077] Step S161: extracting the target centroid of the powered device according to the boundary information.

[0078] Step S162, calculating the corresponding distance from the target centroid to each point cloud of the intruder.

[0079] Step S163: determine the corresponding point cloud with the minimum distance among the multiple corresponding distances as the target point cloud.

[0080] Step S164, calculating the Euclidean distance between the target point cloud and the target centroid.

[0081] In some embodiments, based on the boundary information obtained above, a three-dimensional corner point frame is extracted, and the centroid of the live equipment is extracted from the three-dimensional corner point frame as a representative point, and the minimum distance point cloud from the intruder or construction machinery to the representative point is searched as the target point cloud, and the three-dimensional coordinates of the centroid representative point and the target point cloud are output. Based on the three-dimensional coordinates of the centroid and the target point cloud, the Euclidean distance between the two points is calculated.

[0082] In some embodiments, the above step S170 can be implemented by the following steps S171 to S172:

[0083] Step S171, if the Euclidean distance exceeds the preset distance threshold, and the intruder is close to the edge of the restricted area and the construction machinery deviates from the predetermined route, a first alarm message is output. The first alarm message is used to remind the substation staff to observe the situation or the construction operator to immediately correct the machinery route.

[0084] Step S172: If the Euclidean distance exceeds the preset distance threshold, and the intruder crosses the boundary and enters the restricted area, and the construction machinery enters the restricted area, a second alarm message is output, and the second alarm message is used to notify the security personnel to rush to the scene to intervene immediately or the construction operator to stop working immediately.

[0085] Step S173, if the Euclidean distance exceeds the preset distance threshold, and multiple intruders are detected entering the restricted area and the construction machinery is operating abnormally, a third alarm message is output. The third alarm message is used to notify the security personnel to block the relevant area and request law enforcement agencies to intervene, and if necessary, shut down the equipment power to remove the abnormal operating machinery.

[0086] Here, the abnormal operation of the construction machinery may include: the driving speed of the construction machinery exceeds a preset driving threshold, the construction machinery is moving without anyone operating it, and other abnormalities.

[0087] The substation construction safety distance monitoring method, device, equipment and storage medium provided by the embodiments of the present invention establish a three-dimensional target detection model by acquiring point cloud data of the substation, and perform a first three-dimensional target detection based on the three-dimensional target detection model to detect whether there is a live device in the monitored area. When it is determined that there is a live device in the monitored area, the three-dimensional position information corresponding to the live device is obtained, and a risk area is divided based on the three-dimensional position information. Thereafter, a secondary three-dimensional target detection is performed based on the three-dimensional target detection model to detect whether there is an intruder in the risk area. When it is determined that there is an intruder in the risk area, the type of the intruder is further determined, that is, whether it is an intruder or a construction machine. Finally, based on the boundary information, the Euclidean distance between the live device and the intruder is calculated; if the Euclidean distance exceeds the preset distance threshold, an alarm message is output based on the category information. In this way, the present invention reduces a larger area to be monitored to a smaller risk area through the first three-dimensional target detection, thereby detecting intruders in the risk area, which improves the accuracy of target detection to a certain extent. Through the second three-dimensional target detection, the type of intruder and boundary information are determined, which improves the accuracy of target detection to a certain extent. In addition, according to the determined category information, different levels of alarm information are output to provide managers with more comprehensive and accurate alarm information, which improves the safety management and control capabilities of substation operations while also improving the accuracy and precision of safety monitoring, reducing the risks of substation operators and the failure risks of substation construction machinery.

[0088] The following describes an exemplary application of an embodiment of the present invention in a practical application scenario.

[0089] This embodiment provides a substation construction safety distance monitoring method based on three-dimensional target detection. The specific process is as follows: Figure 2 As shown, the following steps are included:

[0090] Step 1: Establish a data acquisition system: Fix the lidar and camera and perform joint calibration to collect substation images and point cloud data, which includes 7 categories: people, aerial work vehicles, cranes, insulators, grading rings, bushings, and disconnectors. Construct a substation dataset of point cloud data and image data within a range of 0 to 50 meters.

[0091] Step 1 includes the following steps:

[0092] Step 1.1, build data acquisition equipment: LiDAR is a point cloud acquisition device, and a low-cost, high-efficiency non-repetitive scanning solid-state LiDAR is selected; the camera is an image acquisition device, and a high-resolution CMOS industrial camera is selected.

[0093] Step 1.2, joint calibration of lidar and camera: Use ACSC joint calibration method to perform spatial transformation calculation from lidar coordinate system to camera coordinate system, and extract corresponding 3D-2D corner points from the same frame data of the two sensors to solve the external parameter matrix.

[0094] The camera intrinsic parameter calibration is performed using the principle of nonlinear optimization, and the camera intrinsic parameter matrix M is obtained as follows:

[0095]

[0096] in, , is the camera focal length and pixel density , The product of .

[0097] 3D corner points are extracted by optimizing spatial geometric features and 3D corner point estimation architecture based on reflection intensity distribution, and 3D-2D corner point pairs are established. L , Y L , Z L ) is projected onto a two-dimensional image (u, v) to solve the external parameters, which can be calculated using the following formula:

[0098]

[0099] Where s is the camera scale factor, M is the camera intrinsic parameter matrix mentioned above, and [R|T] is the camera extrinsic parameter matrix.

[0100] Step 1.3, substation data collection: The LiDAR and camera are synchronized in time to collect substation images and point cloud data, which includes 7 categories: people, aerial work vehicles, cranes, insulators, grading rings, bushings, and disconnectors. A substation dataset of point cloud data and image data within the range of 0 to 50 meters is constructed.

[0101] Step 2: Construct a substation scene target recognition model: Based on the substation site cloud data, a 3D target detection model is established. First, identify the live equipment in the LiDAR monitoring area, delineate the risk area with each live equipment as the center, perform three-dimensional target detection on intruders or construction machinery in the risk area, accurately locate the intruder in three dimensions, and further judge the identified objects.

[0102] Step 2 includes the following steps:

[0103] Step 2.1, data labeling: SUSTechPOINTS labeling tool is used for point cloud labeling, Labelme is used for image labeling, and the two are sorted according to the label format of KITTI dataset. The obtained labels include 15 attributes, namely: object category, whether truncated, whether blocked, observation angle, 2D object bounding box (x min ,y min , x max ,y max ), three-dimensional object size (height, width, length), three-dimensional object position (x, y, z), and three-dimensional object global orientation angle; the training set, validation set, and test set are divided into 6:2:2.

[0104] Step 2.2, 3D target detection method adapted to construction work of substations with diverse shapes: Use the point-based single-stage 3D target detection method to build the original target detection model, add instance-aware and centroid-aware downsampling modules to reduce the point cloud computing cost based on the complex background of the substation, and use the training set point cloud data obtained in step 2.1 to train, evaluate and optimize the original target detection model to obtain the optimal target detection model. The specific method is as follows:

[0105] like Figure 3 As shown in the figure, the original point cloud data of the substation Input network, where is the 3D coordinate of the point cloud, and r is the reflectivity. The D-FPS and F-FPS methods are used to extract point-by-point features, and then instance-aware downsampling is performed to gradually reduce the computational cost while retaining the information-rich foreground points. The learned potential features are further input into the context centroid perception module, and the IA-SSD detection generates instance proposal regions and regresses the final bounding box. The loss function is used to optimize the detection performance of the model and obtain the final 3D object detection model.

[0106] Step 2.3, substation operation object identification: Input the substation original point cloud data into the above target detection model, and detect the live equipment in the original data based on the trained 3D target detection model. Obtain the 3D position information of the live equipment, delineate the risk area with the live equipment as the center, detect the intruders and construction machinery in the risk area, output the category of the intruder and the bounding box, and obtain the 3D position and boundary of the intrusion.

[0107] Step 3: Calculate the minimum spacing between substation construction operations selected by the optimal representative point cloud: Based on the recognition results of the three-dimensional detection model, search for the minimum spacing point between the detected targets, calculate the distance of the minimum spacing point, and alarm for targets that exceed the safety distance threshold.

[0108] Step 3 includes the following steps:

[0109] Step 3.1, selection of the optimal representative point cloud for substation construction operations: Based on the bounding box obtained in step 2.3, extract the three-dimensional corner point box, and extract the centroid of the live equipment from the three-dimensional corner point box as the representative point, search for the point cloud with the minimum distance from the intruder or construction machinery to the representative point as the optimal point cloud, and output the three-dimensional coordinates of the centroid representative point and the optimal representative point cloud.

[0110] The centroid of the three-dimensional target detection frame of the live equipment is ,The point cloud collection of operators or construction vehicles is , take the vertices of the three-dimensional detection box of the operator or construction vehicle as the initial candidate points V m , V m The coordinates of , First, calculate the distances d={d1, d2, d3, ..., d8} from point P to the eight vertices. The minimum distance is d1, and the nearest vertex is V1. The three faces connected to vertex V1 are F1, F2, and F2. Find the point with the minimum distance from the plane to point P.

[0111]

[0112] in, The nearest vertex The coordinates of is the coordinate of the center of mass P of the energized equipment.

[0113] like Figure 4 As shown, taking F1 as an example, the shortest distance from P to the plane is calculated. The specific method is as follows:

[0114] ,

[0115] First select the plane The four fixed points construct the two intersecting line segments as well as , then the plane Any point on With fixed point The connection can be used and Indicates that .

[0116] Solving for the maximum value

[0117] The above equation has a minimum solution, and the minimum distance point on F1 can be obtained. Similarly, the minimum distance point from P to F2 and F3 can be obtained, and the optimal representative point can be obtained. .

[0118] Step 3.2: Calculation of safe distance for substation construction work: Based on the centroid representative point obtained in step 3.1 And the optimal representative point cloud 3D coordinates , calculate the Euclidean distance between two points , the distance is visualized in 3D point clouds and 2D images, and an alarm is issued for targets that exceed the safety distance threshold, thus realizing safety distance monitoring of substation construction.

[0119] Minimum distance .

[0120] Figure 5 FIG. 1 is a schematic diagram of the structure of a substation construction safety distance monitoring device provided by an embodiment of the present invention. Figure 5 As shown, the substation construction safety distance monitoring device 500 includes: an acquisition module 501, which is used to acquire point cloud data and image data of the substation; an establishment module 502, which is used to establish a three-dimensional target detection model based on the point cloud data; a detection module 503, which is used to detect whether there is a live device in the monitored area based on the point cloud data and the three-dimensional target detection model; a determination module 504, which is used to determine the risk area according to the acquired three-dimensional position information of the live device if the live device exists in the monitored area; the detection module 503 is also used to perform three-dimensional target detection on the risk area based on the three-dimensional target detection model to obtain a detection result; the detection result includes whether there is an intruder in the risk area, and the category information and boundary information of the intruder; the determination module 504 is also used to determine the Euclidean distance between the live device and the intruder based on the boundary information; an output module 505 is used to output an alarm message based on the category information if the Euclidean distance exceeds a preset distance threshold.

[0121] In some embodiments, before acquiring the point cloud data and image data of the substation, the device also includes: a synchronization module for jointly calibrating the laser radar and the camera; based on a preset synchronization method, performing time soft synchronization on the jointly calibrated laser radar and the camera.

[0122] In some embodiments, the synchronization module is also used to determine the coordinate transformation matrix between the lidar coordinate system and the camera coordinate system using a preset joint calibration method; establish a correspondence between each corner point in the three-dimensional space and the projection point of the corner point on the two-dimensional image; calibrate the camera intrinsic parameters using the principle of nonlinear optimization to obtain the camera intrinsic parameter matrix; determine the camera extrinsic parameter matrix based on the camera intrinsic parameter matrix, the corresponding relationship and the camera scale factor.

[0123] In some embodiments, the establishment module 502 is also used to introduce instance-aware and centroid-aware downsampling into the original target detection model; perform sparse processing on the point cloud data to obtain sparse point cloud data; and train, evaluate and optimize the original target detection model based on the sparse point cloud data to obtain the three-dimensional target detection model.

[0124] In some embodiments, the determination module 504 is also used to extract the target center of mass of the charged device based on the boundary information; calculate the corresponding distance from the target center of mass to each point cloud of the intruder; determine the corresponding point cloud with the minimum distance among multiple corresponding distances as the target point cloud; and calculate the Euclidean distance between the target point cloud and the target center of mass.

[0125] In some embodiments, the output module 505 is also used to output a first alarm message if the Euclidean distance exceeds a preset distance threshold, and the intruder is close to the edge of the restricted area and the construction machinery deviates from the predetermined route. The first alarm message is used to remind the substation staff to observe the situation or the construction operator to immediately correct the machinery route; if the Euclidean distance exceeds the preset distance threshold, and the intruder crosses the boundary and enters the restricted area and the construction machinery enters the restricted area, a second alarm message is output. The second alarm message is used to notify the security personnel to rush to the scene to intervene immediately or the construction operator to stop working immediately; if the Euclidean distance exceeds the preset distance threshold, and multiple intruders are detected to enter the restricted area and the construction machinery operates abnormally, a third alarm message is output. The third alarm message is used to notify the security personnel to block the relevant area and request law enforcement agencies to intervene, and if necessary, turn off the power of the equipment to remove the abnormal operating machinery.

[0126] It should be noted that the description of the device of the embodiment of the present invention is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the device embodiment, please refer to the description of the method embodiment of the present invention for understanding.

[0127] It should be noted that in the embodiment of the present invention, if the above-mentioned substation construction safety distance monitoring method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a terminal to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), disk or optical disk, etc. Various media that can store program codes. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0128] Correspondingly, an embodiment of the present invention provides a substation construction safety distance monitoring device, Figure 6 Schematic diagram of the structure of the substation construction safety distance monitoring device provided by the embodiment of the present invention. Figure 6 As shown, the substation construction safety distance monitoring device 600 at least includes: a processor 601 and a computer-readable storage medium 602 configured to store executable instructions, wherein the processor 601 generally controls the overall operation of the substation construction safety distance monitoring device 600. The computer-readable storage medium 602 is configured to store instructions and applications executable by the processor 601, and can also cache data to be processed or processed by each module in the processor 601 and the substation construction safety distance monitoring device 600, which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0129] An embodiment of the present invention provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the method provided by the embodiment of the present invention, for example, Figure 1 The method shown.

[0130] In some embodiments, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM); it can also be various devices including one or any combination of the above memories.

[0131] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0132] As an example, executable instructions may, but need not correspond to, a file in a file system, may be stored as part of a file storing other programs or data, such as one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As an example, executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected by a communication network.

[0133] The above description is only an embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.

[0134] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0135] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, an element defined by the statement "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed.

[0136] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A substation construction safety distance monitoring method, characterized in that: The method comprises: Obtain point cloud data and image data of the substation; Based on the point cloud data, a three-dimensional target detection model is established; Based on the point cloud data and the three-dimensional target detection model, detecting whether there is a live device in the area to be monitored; If the energized device exists in the area to be monitored, determining the risk area according to the acquired three-dimensional position information of the energized device; Based on the three-dimensional target detection model, three-dimensional target detection is performed on the risk area to obtain a detection result; the detection result includes whether there is an intruder in the risk area, and the category information and boundary information of the intruder; Based on the boundary information, determining a Euclidean distance between the powered device and the intruder; If the Euclidean distance exceeds a preset distance threshold, an alarm message is output based on the category information.

2. The method according to claim 1, characterized in that Before acquiring the point cloud data and image data of the substation, the method further includes: Jointly calibrate the lidar and camera; Based on a preset synchronization method, the jointly calibrated laser radar and the camera are soft-synchronized in time.

3. The method according to claim 2, characterized in that The laser radar and camera are jointly calibrated, including: A preset joint calibration method is used to determine the coordinate transformation matrix between the laser radar coordinate system and the camera coordinate system; Establishing a correspondence between each corner point in the three-dimensional space and the projection point of the corner point on the two-dimensional image; The camera intrinsic parameters are calibrated using the nonlinear optimization principle to obtain the camera intrinsic parameter matrix; A camera extrinsic parameter matrix is ​​determined according to the camera intrinsic parameter matrix, the corresponding relationship and the camera scale factor.

4. The method according to claim 1, characterized in that: The step of establishing a three-dimensional target detection model based on the point cloud data comprises: Introducing instance-aware and centroid-aware downsampling into the original object detection model; Performing sparse processing on the point cloud data to obtain sparse point cloud data; The original target detection model is trained, evaluated and optimized according to the sparse point cloud data to obtain the three-dimensional target detection model.

5. The method according to claim 1, characterized in that The determining, based on the boundary information, the Euclidean distance between the charged device and the intruder includes: Extracting the target centroid of the energized device according to the boundary information; Calculating the corresponding distance from the target mass center to each point cloud of the intruder; Determine the corresponding point cloud with the minimum distance among the plurality of corresponding distances as the target point cloud; The Euclidean distance between the target point cloud and the target centroid is calculated.

6. The method according to claim 1, characterized in that The intruder includes an intruder and a construction machine. If the Euclidean distance exceeds a preset distance threshold, based on the category information, outputting an alarm message includes: If the Euclidean distance exceeds a preset distance threshold, and the intruder approaches the edge of the restricted area and the construction machinery deviates from the predetermined route, a first alarm message is output, and the first alarm message is used to remind the substation staff to observe the situation or the construction operator to immediately correct the machinery route; If the Euclidean distance exceeds the preset distance threshold, and the intruder crosses the boundary and enters the restricted area, or the construction machinery enters the restricted area, a second alarm message is output, and the second alarm message is used to notify the security personnel to rush to the scene immediately for intervention or the construction operator to stop the operation immediately; If the Euclidean distance exceeds the preset distance threshold, and multiple intruders are detected entering the restricted area and the construction machinery is operating abnormally, a third alarm message is output. The third alarm message is used to notify the security personnel to block the relevant area and request law enforcement agencies to intervene, and if necessary, shut down the equipment power and remove the abnormal operating machinery.

7. A substation construction safety distance monitoring device, characterized in that: The device comprises: An acquisition module, used to acquire point cloud data and image data of the substation; An establishment module is used to establish a three-dimensional target detection model based on the point cloud data; A detection module, used to detect whether there is a live device in the area to be monitored based on the point cloud data and the three-dimensional target detection model; A determination module, configured to determine a risk area based on the acquired three-dimensional position information of the energized device if the energized device exists in the area to be monitored; The detection module is further used to perform three-dimensional target detection on the risk area based on the three-dimensional target detection model to obtain a detection result; the detection result includes whether there is an intruder in the risk area, and the category information and boundary information of the intruder; The determination module is further used to determine the Euclidean distance between the charged device and the intruder based on the boundary information; The output module is used to output warning information based on the category information if the Euclidean distance exceeds a preset distance threshold.

8. A substation construction safety distance monitoring device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the substation construction safety distance monitoring method as described in any one of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the substation construction safety distance monitoring method according to any one of claims 1 to 6.

Images