A method and device for determining the degree of hidden danger of a power transmission line channel and a storage medium

Abstract

This application discloses a method, device, and storage medium for determining the degree of hidden dangers in transmission line corridors, solving the technical problems of high cost, low accuracy, and low efficiency in existing methods for determining the degree of hidden dangers in transmission line corridors. The method includes: acquiring a first monocular 2D image, a first 3D point cloud dataset, a second monocular 2D image, and a second 3D point cloud dataset in a first cycle and a second cycle, respectively; inputting the first and second 3D point cloud datasets into a point cloud difference recognition model to determine whether a first difference region exists; if a first difference region exists and intersects with the 3D corridor monitoring area, determining a second difference region in the second monocular 2D image using a spatial coordinate transformation model; and based on a hidden danger target recognition model, determining the first hidden danger target information within the second difference region and determining the degree of hidden danger. This application optimizes existing methods for determining the degree of hidden dangers in transmission line corridors through the above method.

Description

Technical Field

[0001] This application relates to the field of power transmission line technology, and in particular to a method, device and storage medium for determining the degree of hidden dangers in power transmission line channels. Background Technology

[0002] Monitoring the surrounding environment is crucial in the operation and maintenance of power transmission lines. When machinery is operating near transmission lines, if the distance between the machinery and the line is less than the safe distance, electrical discharges may occur, potentially causing injury or even power outages. Furthermore, large construction machinery, especially cranes and concrete pump trucks in a raised or extended boom position, can easily snag power lines during operation, posing a significant threat to conductors. Therefore, identifying these potential hazards and quantitatively and qualitatively assessing the degree of threat to conductors is essential to ensuring the safety of power transmission lines.

[0003] Currently, the assessment of potential hazards in power transmission line corridors primarily relies on drone photography and satellite remote sensing imagery. However, drone inspections are susceptible to flight-related factors such as radio, weather, and geographical conditions. Furthermore, drones have limited data acquisition capacity per shot, resulting in high operational costs, low data collection effectiveness, and the need for post-collection quality checks, supplementary photography for omissions or deficiencies, and post-processing for 3D reconstruction and measurement using 3D software to identify hazards. This approach lacks timeliness. Satellite remote sensing images require high resolution and are not sensitive enough to detect small targets within each span of the corridor. They can only provide a rough assessment of the presence of hazards, but cannot accurately measure the degree of threat posed to the conductor.

[0004] Therefore, there is an urgent need for a precise, efficient, and low-cost method for determining the degree of hidden dangers in power transmission line corridors. Summary of the Invention

[0005] This application provides a method, device, and storage medium for determining the degree of hidden dangers in power transmission line corridors, in order to solve the technical problems that existing methods for determining the degree of hidden dangers in power transmission line corridors are costly, and have low accuracy and efficiency.

[0006] In a first aspect, embodiments of this application provide a method for determining the degree of hidden danger in a power transmission line corridor. The method includes: responding to a periodic hidden danger determination instruction, acquiring a first monocular two-dimensional image and a first three-dimensional point cloud dataset of the power transmission line corridor in a first cycle, and acquiring a second monocular two-dimensional image and a second three-dimensional point cloud dataset of the power transmission line corridor in a second cycle; wherein the second cycle is the cycle following the first cycle; the monocular two-dimensional image is obtained based on a monocular camera mounted on a power pole, and the three-dimensional point cloud dataset is obtained based on a lidar mounted on a power pole; inputting the first three-dimensional point cloud dataset and the second three-dimensional point cloud dataset into a preset point cloud difference recognition model to determine whether a first difference region exists between the power transmission line corridor described by the second three-dimensional point cloud dataset and the power transmission line corridor described by the first three-dimensional point cloud dataset; wherein the first difference region is a three-dimensional region. If a first difference region is identified, and if, based on the third 3D point cloud dataset corresponding to the first difference region, it is determined that the first difference region intersects with a preset three-dimensional passage monitoring area, then, based on the third 3D point cloud dataset and through a preset spatial coordinate transformation model, a second difference region is determined in the second monocular 2D image; wherein, the second difference region is the 2D difference region corresponding to the first difference region in the second monocular 2D image; based on a preset hazard target identification model, information about a first hazard target within the second difference region is determined; wherein, the information about the first hazard target includes at least: type information of the first hazard target and first coordinate information of the first hazard target; the first coordinate information is the coordinate information of the highest point of the first hazard target within the three-dimensional passage monitoring area; based on the first coordinate information, the distance from the first hazard target to the power transmission line is determined, and the degree of hazard is judged based on the distance from the first hazard target to the power transmission line.

[0007] This application provides a method for determining the degree of hidden dangers in power transmission line corridors. By leveraging the dense information advantage of monocular two-dimensional images and the precision advantage of three-dimensional point cloud data, it optimizes existing methods for determining the degree of hidden dangers in power transmission line corridors. By constructing a spatial coordinate transformation model, it achieves the conversion between monocular two-dimensional images and three-dimensional point cloud data. After determining the first difference region, it can further detect the type of hidden danger within the first difference region and calculate the distance information between the hidden danger target and the conductor. This achieves hidden danger distance measurement function while saving costs, thereby further determining the danger level of the hidden danger and providing more timely, accurate, and low-cost early warning information for the safety of power transmission line corridors.

[0008] In one implementation of this application, before responding to a periodic hazard level determination instruction, the method further includes: acquiring an experimental monocular two-dimensional image using a monocular camera and acquiring an experimental three-dimensional point cloud dataset using a lidar; determining the parameters of the monocular camera based on the experimental monocular two-dimensional image and the experimental three-dimensional point cloud dataset; wherein the parameters include external parameters and internal parameters; determining the spatial coordinate transformation relationship between pixel coordinates in the experimental monocular two-dimensional image and discrete points in the experimental three-dimensional point cloud dataset through joint calculation of the internal parameters and external parameters; wherein the pixel coordinates are the positions of each pixel in the monocular two-dimensional image, determined with pixels as the distance unit; and determining the spatial coordinate transformation model according to the spatial coordinate transformation relationship.

[0009] In one implementation of this application, the parameters of the monocular camera are determined based on experimental monocular 2D images and experimental 3D point cloud datasets. Specifically, this includes: calibrating the monocular camera based on the experimental monocular 2D images to determine the internal parameters of the monocular camera; determining several experimental feature points in the experimental monocular 2D images and determining several experimental discrete points corresponding to the several experimental feature points in the experimental 3D point cloud dataset; and determining the external parameters of the monocular camera based on the several experimental feature points and the several experimental discrete points using a camera pose estimation algorithm.

[0010] In one implementation of this application, before determining that the first difference region intersects with the preset three-dimensional channel monitoring area based on the third three-dimensional point cloud dataset corresponding to the first difference region, the method further includes: determining the set of discrete points of the transmission line corresponding to the transmission line in the second three-dimensional point cloud dataset, and fitting the set of discrete points of the transmission line into a three-dimensional curve; generating the three-dimensional channel monitoring area based on the three-dimensional curve and the safety distance corresponding to the voltage level of the transmission line.

[0011] In one implementation of this application, after determining the degree of hazard based on the distance from the first hazard target to the conductor, the method further includes: determining the degree of hazard determination level corresponding to the first hazard target; adding first hazard target information to the first hazard target in the second monocular two-dimensional image to generate a first hazard target information image; and outputting an alarm notification to issue an alarm based on the first hazard target information image and the degree of hazard determination level corresponding to the first hazard target.

[0012] In one implementation of this application, the method further includes: determining the second coordinate information corresponding to the highest dangerous discrete point in the three-dimensional channel monitoring area in the second three-dimensional point cloud dataset when it is determined that there is no first difference region; determining the distance from the dangerous discrete point to the transmission line based on the second coordinate information; and judging the degree of hidden danger based on the distance from the dangerous discrete point to the transmission line.

[0013] In one implementation of this application, after determining the degree of hazard based on the distance from the hazardous discrete point to the conductor, the method further includes: determining the hazardous pixel coordinates corresponding to the hazardous discrete point in the second monocular two-dimensional image through a spatial coordinate transformation model; determining the type information of the second hazardous target corresponding to the hazardous pixel coordinates based on a hazard target recognition model; and determining the second hazard target information based on the type information and the second coordinate information of the second hazard target.

[0014] In one implementation of this application, before determining the information of hidden danger targets in the second difference region based on a preset hidden danger target identification model, the method further includes: acquiring several monocular two-dimensional images containing hidden danger targets to construct a hidden danger sample set; determining a hidden danger target identification algorithm and training the hidden danger target identification algorithm according to the hidden danger sample set to obtain a converged hidden danger target identification model; the hidden danger targets include at least one or more of the following: large construction machinery, poles, tower cranes, cranes, trees, and buildings.

[0015] Secondly, embodiments of this application also provide a device for determining the degree of hidden dangers in power transmission line channels, characterized in that the device includes: a processor; and a memory, on which executable code is stored, wherein when the executable code is executed, the processor performs a method as described in any one of claims 1-8.

[0016] Thirdly, embodiments of this application also provide a non-volatile computer storage medium for determining the degree of hidden danger in transmission line channels, storing computer-executable instructions. The computer-executable instructions are configured to: in response to periodic hidden danger determination instructions, acquire a first monocular two-dimensional image and a first three-dimensional point cloud dataset of the transmission line channel in a first period, and acquire a second monocular two-dimensional image and a second three-dimensional point cloud dataset of the transmission line channel in a second period; wherein the second period is the period following the first period; the monocular two-dimensional image is obtained based on a monocular camera mounted on a power pole, and the three-dimensional point cloud dataset is obtained based on a lidar mounted on a power pole; input the first three-dimensional point cloud dataset and the second three-dimensional point cloud dataset into a preset point cloud difference recognition model to determine whether a first difference region exists between the transmission line channel described by the second three-dimensional point cloud dataset and the transmission line channel described by the first three-dimensional point cloud dataset. The first difference region is a three-dimensional region. If, based on the third three-dimensional point cloud dataset corresponding to the first difference region, it is determined that the first difference region intersects with a preset three-dimensional passage monitoring area, then, based on the third three-dimensional point cloud dataset and a preset spatial coordinate transformation model, a second difference region is determined in the second monocular two-dimensional image. The second difference region is the two-dimensional difference region corresponding to the first difference region in the second monocular two-dimensional image. Based on a preset hazard target identification model, information about a first hazard target within the second difference region is determined. This first hazard target information includes at least: the type information of the first hazard target and the first coordinate information of the first hazard target. The first coordinate information is the coordinate information of the highest point of the first hazard target within the three-dimensional passage monitoring area. Based on the first coordinate information, the distance from the first hazard target to the conductor is determined, and the degree of hazard is judged based on this distance. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A flowchart illustrating a method for determining the degree of hidden dangers in a power transmission line corridor, as provided in this application embodiment;

[0019] Figure 2 This is a schematic diagram of the internal structure of a device for determining the degree of hidden dangers in a power transmission line corridor, provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] In the operation and maintenance of power transmission lines, monitoring the surrounding environment is a crucial task. This includes checking for excessively tall trees, illegal buildings, unauthorized construction, and so on. In particular, all construction machinery within the transmission line corridor must be kept at least 10 meters away from the conductors; within 3 meters, there is a high risk of electrical discharge causing injury or power outage. Large construction machinery, especially cranes and concrete pump trucks in raised or extended boom positions, pose a significant threat to the conductors. Therefore, it is essential to identify these hazards and quantitatively and qualitatively assess their degree of threat to the conductors.

[0022] Currently, the identification of potential hazards and the assessment of hazard levels along power transmission line corridors primarily rely on drone photography, satellite remote sensing images, and fixed-point camera monitoring. However, these three methods still have shortcomings, which will be discussed separately below:

[0023] (1) The inspection method of UAV is greatly affected by flight environment factors (radio environment, meteorological environment, geographical environment). The amount of data collected in a single time is limited, the collection effectiveness is low, the operation cost is high, and the data quality needs to be checked after collection. If there are omissions or deficiencies, supplementary photos should be taken, and then post-processing should be carried out for three-dimensional reconstruction. Data measurement should be carried out using 3D software to check for hidden dangers. The timeliness is not high.

[0024] (2) Satellite remote sensing images require very high resolution and are not sensitive to the detection of small targets within each span channel. They can only roughly determine whether there are hidden dangers, but cannot measure and judge the degree of threat to the conductor from the hidden dangers.

[0025] (3) In existing camera-based fixed-point monitoring technologies, for applications within the span of power transmission channels, detection methods based solely on visual images are easily affected by viewing angle and external lighting factors. Moreover, monocular vision cannot directly acquire the depth information of the target. In non-cooperative target pose measurement, it needs to be used in conjunction with multiple other sensors, which often limits its application in independent systems and results in low accuracy. The limitation of binocular stereo vision lies in the fact that its method of obtaining parallax images is limited by the baseline length and the matching accuracy of pixels between the left and right images; due to the limitations of the matching algorithm, distance calculation cannot be real-time. In practical applications, high installation accuracy is required, and the design accuracy of this system in the laboratory cannot be achieved under general application conditions.

[0026] This application provides a method, device, and storage medium for determining the degree of hidden dangers in power transmission line corridors, in order to solve the technical problems that existing methods for determining the degree of hidden dangers in power transmission line corridors are costly, and have low accuracy and efficiency.

[0027] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0028] Figure 1 This is a flowchart illustrating a method for determining the degree of hidden dangers in a power transmission line corridor, as provided in an embodiment of this application. Figure 1 As shown in the embodiment of this application, a method for determining the degree of hidden danger in a power transmission line corridor mainly includes the following steps:

[0029] Step 101: In response to the periodic hazard level determination command, acquire the first monocular two-dimensional image and the first three-dimensional point cloud dataset of the transmission line channel in the first cycle, and acquire the second monocular two-dimensional image and the second three-dimensional point cloud dataset of the transmission line channel in the second cycle.

[0030] It should be noted that the monocular 2D images are obtained from monocular cameras mounted on power poles, while the 3D point cloud dataset is obtained from lidar mounted on power poles. The 3D point cloud data refers to a set of vectors in a three-dimensional coordinate system, exhibiting good environmental robustness and high-precision depth data. The principle of monocular 2D image acquisition is that the wider the camera's field of view, the shorter the precise detection distance; conversely, the narrower the field of view, the longer the detection distance. This is similar to how the human eye perceives the world: the farther away the view, the narrower the coverage area; the closer the view, the wider the coverage area. Therefore, this embodiment of the application jointly analyzes monocular 2D images and 3D point cloud data, combining the dense information advantage of monocular 2D images with the precision advantage of 3D point cloud data. This improves the effectiveness and accuracy of data during power transmission line monitoring, thereby providing more timely and effective early warning information for power transmission line safety.

[0031] In one embodiment of this application, in order to enable joint analysis of monocular 2D images and 3D point cloud data, it is first necessary to determine the spatial coordinate transformation relationship between the pixel coordinates in the 3D point cloud data and the monocular 2D image. It should be noted that pixel coordinates are the positions of each pixel in the monocular 2D image, determined in pixels as the distance unit.

[0032] Specifically, firstly, a monocular camera acquires experimental monocular 2D images, and a LiDAR acquires experimental 3D point cloud datasets. These experimental monocular 2D images and 3D point cloud datasets provide data support for determining spatial coordinate transformation relationships. It should be noted that, to better map the pixel coordinates in the 3D point cloud data and the monocular 2D images, the data acquisition angles of the monocular camera and LiDAR can be adjusted as much as possible to ensure a better correspondence between the 3D point cloud data acquired by the two devices and the monocular 2D images. After determining the angles of the two devices, their positions and data acquisition angles are fixed, ensuring that the data acquired by the monocular camera and LiDAR in each instance corresponds to the same environment at the same location.

[0033] Then, based on the acquired experimental monocular 2D images and experimental 3D point cloud data, the parameters of the monocular camera are determined. This includes determining the internal and external parameters of the monocular camera. The internal parameters are determined by calibrating the monocular camera using the acquired experimental 2D images. It should be noted that there are various methods for calibrating a monocular camera, such as the Zhang Zhengyou calibration method and the OpenCV monocular camera calibration method.

[0034] In one embodiment of this application, the Zhang Zhengyou calibration method is preferred. The camera is calibrated using the Zhang Zhengyou calibration method to determine the internal parameters of the monocular camera. These internal parameters include the camera focal length parameters fx and fy, and the camera optical center parameters cx and cy.

[0035] After determining the internal parameters of the monocular camera, it is first necessary to determine several experimental feature points in the experimental monocular 2D image and several experimental discrete points corresponding to the experimental feature points in the experimental 3D point cloud dataset.

[0036] It should be noted that experimental feature points are points in the experimental monocular 2D image that highlight image features, such as the apex of a tower, the lowest point on a power transmission line formed by gravity, and the apex of a building. The experimental feature points can be determined through object detection models, image processing, or manual calibration; this application does not impose any limitations and the choice can be made based on the specific experimental environment. After determining several experimental feature points in the experimental monocular 2D image, several corresponding discrete experimental points are found in the experimental 3D point cloud dataset. To ensure the accuracy of the spatial coordinate transformation, the number of pairs of determined experimental feature points and discrete experimental points should be greater than or equal to four.

[0037] After determining several experimental feature points and several experimental discrete points, the coordinate information corresponding to the discrete points is used as input. A camera pose estimation algorithm is then used to combine the coordinate information of the discrete points with the pixel coordinates of the experimental feature points in the experimental monocular 2D image to determine the external parameters of the monocular camera. These external parameters include the rotation angle and translation distance of the LiDAR relative to the monocular camera.

[0038] After determining the parameters of the monocular camera, the spatial coordinate transformation relationship between discrete points in the experimental 3D point cloud data and pixel coordinates in the experimental monocular 2D image was determined through joint calculation of internal and external parameters. Based on this spatial coordinate transformation relationship, the corresponding spatial coordinate transformation model was determined. It is understood that the determined spatial transformation model can perform coordinate transformation on any set of acquired monocular 2D images and 3D point cloud datasets, not just on the experimental 3D point cloud data and experimental monocular 2D images.

[0039] In one embodiment of this application, after the spatial coordinate transformation model is completed, if a hazard level determination instruction is received, a monocular two-dimensional image is acquired using a monocular camera, and three-dimensional point cloud data is acquired using a lidar. Since this embodiment compares the monocular two-dimensional image and the three-dimensional point cloud data acquired in two consecutive acquisitions, the hazard level determination instruction in this application is a periodic hazard level determination instruction. Taking any two consecutive cycles as an example, the previous cycle is designated as the first cycle, and the subsequent cycle as the second cycle. In the first cycle, a first monocular two-dimensional image and a first three-dimensional point cloud dataset of the transmission line channel are acquired; in the second cycle, a second monocular two-dimensional image and a second three-dimensional point cloud dataset of the transmission line channel are acquired.

[0040] Step 102: Input the first three-dimensional point cloud dataset and the second three-dimensional point cloud dataset into the preset point cloud difference recognition model to determine whether there is a first difference region between the transmission line channel described by the second three-dimensional point cloud dataset and the transmission line channel described by the first three-dimensional point cloud dataset.

[0041] In one embodiment of this application, after the first 3D point cloud dataset and the second 3D point cloud dataset, the first 3D point cloud dataset and the second 3D point cloud dataset are input into a preset point cloud difference recognition model to determine whether there is a first difference region between the transmission line channel described by the second 3D point cloud dataset and the transmission line channel described by the first 3D point cloud dataset. It is understood that the first difference region is a 3D region. For example, if there is a tower crane in both a certain area of ​​the transmission line channel described by the first 3D point cloud dataset and a certain area of ​​the transmission line channel described by the second 3D point cloud dataset, it can be understood that these two tower cranes are actually the same entity, but were acquired at different times. If the tower crane in that area of ​​the transmission line channel described by the first 3D point cloud dataset is in a non-working state, its height is lower; if the tower crane in that area of ​​the transmission line channel described by the first 3D point cloud dataset is in a working state, its height is higher, and its boom span is larger. This is reflected in the 3D point cloud dataset, where the difference between the second 3D point cloud dataset and the first 3D point cloud dataset in that area is significant, thus constituting the first difference region.

[0042] It should be noted that the point cloud difference recognition model in this application includes, but is not limited to, models trained using the kd-tree algorithm or the octree algorithm. Furthermore, this application also considers the magnitude of the difference in determining the first difference region. A threshold can be set for the point cloud difference recognition model; only when the difference exceeds the threshold is the difference region determined as the first difference region, thus preventing excessively small difference regions from affecting the judgment. The threshold is primarily determined based on the volume corresponding to the first difference region in three-dimensional space. For example, if the difference between the second and first three-dimensional point cloud datasets in a certain region is negligible (only the discrete point corresponding to one person is different), then that region is not determined as the first difference region.

[0043] Step 103: If the existence of a first difference region is determined, and the first difference region intersects with the preset stereoscopic channel monitoring area based on the third three-dimensional point cloud dataset corresponding to the first difference region, then the second difference region is determined in the second monocular two-dimensional image based on the third three-dimensional point cloud dataset and through the preset spatial coordinate transformation model.

[0044] In one embodiment of this application, if it is determined that there is a first difference region between the transmission line channel described by the second 3D point cloud dataset and the transmission line channel described by the first 3D point cloud dataset, the system first determines whether the first difference region intersects with a preset 3D channel monitoring area based on the third 3D point cloud dataset corresponding to the first difference region. If it is determined that the first difference region intersects with the preset 3D channel monitoring area based on the third 3D point cloud dataset, the system then determines the second difference region in the second monocular 2D image using a preset spatial coordinate transformation model. It can be understood that the second difference region is the 2D difference region corresponding to the first difference region in 3D space in the second monocular 2D image.

[0045] It should be noted that before determining whether the first difference region intersects with the preset three-dimensional channel monitoring area based on the third three-dimensional point cloud dataset corresponding to the first difference region, it is also necessary to determine the three-dimensional channel monitoring area.

[0046] Specifically, firstly, the discrete point set of the transmission line corresponding to the transmission line in the second three-dimensional point cloud dataset is determined, and the discrete point set of the transmission line is fitted into a three-dimensional curve. It should be noted that the transmission line in this embodiment is the lowest layer of the transmission line, that is, the layer of transmission line closest to the ground. Then, a three-dimensional channel monitoring area is generated based on the three-dimensional curve and the safety distance corresponding to the voltage level of the transmission line.

[0047] It should be noted that the discrete point set of the transmission line can be determined by using a converged target detection model to identify the transmission line in the second-dimensional monocular image, and then inversely deriving the discrete point set of the transmission line based on the spatial coordinate transformation model. Alternatively, the discrete point set of the transmission line can be determined by using cluster analysis or trajectory extrapolation algorithms on the experimental discrete points corresponding to the lowest point on the transmission line determined when establishing the spatial coordinate transformation model.

[0048] It should also be noted that, in the embodiments of this application, the moving least squares method can be used to fit the discrete point set of the conductor into a three-dimensional curve.

[0049] Step 104: Based on the preset hazard target identification model, determine the first hazard target information within the second difference area.

[0050] In one embodiment of this application, after determining a second difference region in a second two-dimensional monocular image based on a first difference region, the second difference region is identified based on a preset hazard target recognition model to determine the first hazard target information within the second difference region. It should be noted that the first hazard target information includes at least: the type information of the first hazard target and the first coordinate information of the first hazard target; wherein the first coordinate information is the coordinate information of the highest point of the first hazard target within the three-dimensional passage monitoring area.

[0051] It should also be noted that before determining the information of the first hidden danger target in the second difference area based on the preset hidden danger target identification model, it is necessary to construct a hidden danger target identification model.

[0052] Specifically, several monocular 2D images containing potential hazards are acquired to construct a hazard sample set; a hazard target recognition algorithm is determined, and the algorithm is trained based on the hazard sample set to obtain a converged hazard target recognition model; the hazard targets include at least one or more of the following: large construction machinery, poles, tower cranes, cranes, trees, and buildings. The hazard target recognition algorithm may employ a Faster-RCNN-Res101 network.

[0053] Step 105: Based on the first coordinate information, determine the distance from the first hidden danger target to the transmission line, and determine the degree of hidden danger based on the distance from the first hidden danger target to the transmission line.

[0054] In one embodiment of this application, after determining the first hidden danger target information, the distance from the first hidden danger target to the transmission line is determined based on the first coordinate information contained in the first hidden danger target information. It can be understood that the calculation of the distance from the first hidden danger target to the transmission line is essentially the distance from the first coordinate information to the first three-dimensional curve. The calculation of the shortest distance from a point to a line can be implemented using any existing technology, and this application does not limit it; the appropriate technology can be selected based on the specific application environment.

[0055] After determining the distance from the first potential hazard target to the transmission line, the degree of hazard is judged based on the distance from the first potential hazard target to the transmission line.

[0056] In one embodiment of this application, if the distance from the first potential hazard target to the transmission line is less than the safety distance corresponding to the corresponding voltage level of the transmission line, it is determined that the first potential hazard target poses a threat and meets the requirements for hazard level determination, thus requiring hazard level determination for the first potential hazard target. If the distance from the first potential hazard target to the transmission line is less than a first distance, the hazard level of the first potential hazard target is determined to be urgent; if the distance from the first potential hazard target to the transmission line is greater than or equal to the first distance but less than a second distance, the hazard level of the first potential hazard target is determined to be serious; if the distance from the first potential hazard target to the transmission line is greater than or equal to the second distance but less than the safety distance, the hazard level of the first potential hazard target is determined to be moderate. Wherein, the first distance is less than the second distance, and the second distance is less than the safety distance. If the distance from the first potential hazard target to the transmission line is greater than or equal to the safety distance corresponding to the corresponding voltage level of the transmission line, it is determined that the first potential hazard target does not pose a threat and does not meet the requirements for hazard level determination, therefore, hazard level determination for the first potential hazard target is unnecessary.

[0057] In one embodiment of this application, after determining the degree of hazard based on the distance from the first potential hazard target to the transmission line, the method further includes: determining the hazard degree assessment level corresponding to the first potential hazard target, and adding first potential hazard target information to the second monocular two-dimensional image to generate a first potential hazard target information image. Based on the first potential hazard target information image and the hazard degree assessment level corresponding to the first potential hazard target, an alarm notification is output to issue an alarm.

[0058] It should be noted that since the first difference region is the area where the second 3D point cloud dataset differs significantly from the first 3D point cloud dataset within that region, the cumulative effect of regions with smaller differences may be overlooked if the interval between each hazard level assessment instruction is short. For example, if the interval between hazard level assessment instructions is several hours to a day, then when building within a transmission line corridor, the difference between the 3D point cloud datasets acquired in adjacent intervals will be small. Therefore, with accumulation, the highest point of the building may reach the hazard level assessment requirement. Therefore, it is also necessary to assess the hazard level in cases where the first difference region does not exist.

[0059] In one embodiment of this application, if it is determined that there is no first difference region, the second coordinate information corresponding to the highest dangerous discrete point within the three-dimensional channel monitoring area is first determined from the second three-dimensional point cloud dataset. Then, based on the second coordinate information, the distance from the dangerous discrete point to the transmission line is determined, and the degree of hazard is judged based on the distance from the dangerous discrete point to the transmission line.

[0060] In one embodiment of this application, after determining the degree of hazard based on the distance from the hazardous discrete point to the transmission line, the method further includes: determining the hazardous pixel coordinates corresponding to the hazardous discrete point in the second monocular two-dimensional image using a spatial coordinate transformation model; determining the type information of the second hazardous target corresponding to the hazardous pixel coordinates based on a hazard target recognition model; and determining the second hazardous target information based on the type information and the second coordinate information of the second hazardous target. Then, the hazard degree determination level corresponding to the second hazardous target is determined, and the second hazardous target information is added to the second hazardous target in the second monocular two-dimensional image to generate a second hazardous target information image. Based on the second hazardous target information image and the hazard degree determination level corresponding to the second hazardous target, an alarm notification is output to issue an alarm.

[0061] In one embodiment of this application, the method further includes: if, after determining the existence of a first difference region, it is determined, based on the third three-dimensional point cloud dataset corresponding to the first difference region, that the first difference region does not intersect with the preset three-dimensional channel monitoring area, the same steps are performed as when it is determined that the first difference region does not exist. That is: in the second three-dimensional point cloud dataset, the second coordinate information corresponding to the highest dangerous discrete point within the three-dimensional channel monitoring area is determined. Based on the second coordinate information, the distance from the dangerous discrete point to the transmission line is determined, and the degree of hazard is judged based on the distance from the dangerous discrete point to the transmission line. Then, through a spatial coordinate transformation model, the dangerous pixel coordinates corresponding to the dangerous discrete point in the second monocular two-dimensional image are determined; based on a hazard target recognition model, the type information of the second hazard target corresponding to the dangerous pixel coordinates is determined; based on the type information of the second hazard target and the second coordinate information, the second hazard target information is determined. Then, the hazard degree judgment level corresponding to the second hazard target is determined, and the second hazard target information is added to the second hazard target in the second monocular two-dimensional image to generate a second hazard target information image. Based on the second hazard target information image and the hazard degree judgment level corresponding to the second hazard target, an alarm notification is output to issue an alarm.

[0062] Based on the same inventive concept, this application also provides a device for determining the degree of hidden danger in power transmission line corridors, the internal structure of which is as follows: Figure 2 As shown.

[0063] Figure 2 This is a schematic diagram of the internal structure of a device for determining the degree of hidden dangers in a power transmission line corridor, provided as an embodiment of this application. Figure 2 As shown, the device includes: a processor 201; and a memory 202 storing executable instructions, which, when executed, cause the processor 201 to perform a method for determining the degree of hidden dangers in a power transmission line channel as described above.

[0064] In one embodiment of this application, the processor 201 is configured to respond to a periodic hazard level determination instruction, acquiring a first monocular two-dimensional image and a first three-dimensional point cloud dataset of the transmission line channel in a first cycle, and acquiring a second monocular two-dimensional image and a second three-dimensional point cloud dataset of the transmission line channel in a second cycle; wherein the second cycle is the cycle following the first cycle; the monocular two-dimensional image is obtained based on a monocular camera installed on the power pole, and the three-dimensional point cloud dataset is obtained based on a lidar installed on the power pole; the first three-dimensional point cloud dataset and the second three-dimensional point cloud dataset are input into a preset point cloud difference recognition model to determine whether there is a first difference region between the transmission line channel described by the second three-dimensional point cloud dataset and the transmission line channel described by the first three-dimensional point cloud dataset; wherein the first difference region is a three-dimensional region; upon determining the existence of a first difference region... In this case, if the first difference region is determined to intersect with the preset three-dimensional passage monitoring area based on the third three-dimensional point cloud dataset corresponding to the first difference region, then the second difference region is determined in the second monocular two-dimensional image based on the third three-dimensional point cloud dataset and through a preset spatial coordinate transformation model; wherein, the second difference region is the two-dimensional difference region corresponding to the first difference region in the second monocular two-dimensional image; based on the preset hidden danger target identification model, the first hidden danger target information within the second difference region is determined; wherein, the first hidden danger target information includes at least: the type information of the first hidden danger target and the first coordinate information of the first hidden danger target; the first coordinate information is the coordinate information of the highest point of the first hidden danger target in the three-dimensional passage monitoring area; based on the first coordinate information, the distance from the first hidden danger target to the power transmission line is determined, and the degree of hidden danger is judged based on the distance from the first hidden danger target to the power transmission line.

[0065] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium for determining the degree of hidden dangers in power transmission line corridors, storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0066] In response to the periodic hazard level assessment command, the first monocular two-dimensional image and the first three-dimensional point cloud dataset of the transmission line channel are acquired in the first cycle, and the second monocular two-dimensional image and the second three-dimensional point cloud dataset of the transmission line channel are acquired in the second cycle; wherein, the second cycle is the cycle following the first cycle; the monocular two-dimensional image is obtained based on a monocular camera installed on the power pole, and the three-dimensional point cloud dataset is obtained based on a lidar installed on the power pole;

[0067] The first 3D point cloud dataset and the second 3D point cloud dataset are input into a preset point cloud difference recognition model to determine whether there is a first difference region between the transmission line channel described by the second 3D point cloud dataset and the transmission line channel described by the first 3D point cloud dataset; wherein, the first difference region is a 3D region.

[0068] If the existence of a first difference region is determined, and if the first difference region intersects with the preset stereoscopic channel monitoring area based on the third three-dimensional point cloud dataset corresponding to the first difference region, then the second difference region is determined in the second monocular two-dimensional image based on the third three-dimensional point cloud dataset and through the preset spatial coordinate transformation model; wherein, the second difference region is the two-dimensional difference region corresponding to the first difference region in the second monocular two-dimensional image.

[0069] Based on a pre-set hazard target identification model, the information of the first hazard target within the second difference area is determined; wherein, the information of the first hazard target includes at least: the type information of the first hazard target and the first coordinate information of the first hazard target; the first coordinate information is the coordinate information of the highest point of the first hazard target within the three-dimensional passage monitoring area;

[0070] Based on the first coordinate information, the distance from the first potential hazard target to the power transmission line is determined, and the degree of hazard is judged based on the distance from the first potential hazard target to the power transmission line.

[0071] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0072] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0073] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0078] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0079] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0080] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0081] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for determining the degree of hidden danger in a power transmission line corridor, characterized in that, The method includes: In response to the periodic hazard level assessment command, a first monocular two-dimensional image and a first three-dimensional point cloud dataset of the transmission line channel are acquired in the first cycle, and a second monocular two-dimensional image and a second three-dimensional point cloud dataset of the transmission line channel are acquired in the second cycle; wherein, the second cycle is the cycle following the first cycle; the monocular two-dimensional image is obtained based on a monocular camera installed on the power pole, and the three-dimensional point cloud dataset is obtained based on a lidar installed on the power pole; The first 3D point cloud dataset and the second 3D point cloud dataset are input into a preset point cloud difference recognition model to determine whether there is a first difference region between the transmission line channel described by the second 3D point cloud dataset and the transmission line channel described by the first 3D point cloud dataset; wherein, the first difference region is a 3D region; wherein, the point cloud difference recognition model is trained by the kd-tree algorithm or the octree algorithm. If a first difference region is determined to exist, and if, based on the third 3D point cloud dataset corresponding to the first difference region, it is determined that the first difference region intersects with a preset 3D channel monitoring area, then, based on the third 3D point cloud dataset, a second difference region is determined in the second monocular 2D image through a preset spatial coordinate transformation model; wherein, the second difference region is the 2D difference region corresponding to the first difference region in the second monocular 2D image. Based on a preset hazard target identification model, the information of the first hazard target within the second difference area is determined; wherein, the information of the first hazard target includes at least: the type information of the first hazard target and the first coordinate information of the first hazard target; the first coordinate information is the coordinate information of the highest point of the first hazard target within the three-dimensional passage monitoring area; Based on the first coordinate information, the distance from the first potential hazard target to the transmission line is determined, and the degree of hazard is judged based on the distance from the first potential hazard target to the transmission line. Prior to responding to periodic hazard level determination instructions, the method further includes: The monocular camera acquires experimental monocular two-dimensional images, and the lidar acquires experimental three-dimensional point cloud datasets. Based on the experimental monocular 2D image and the experimental 3D point cloud dataset, the parameters of the monocular camera are determined; wherein, the parameters include external parameters and internal parameters. By jointly calculating the internal parameters and the external parameters, the spatial coordinate transformation relationship between the pixel coordinates in the experimental monocular two-dimensional image and the discrete points in the experimental three-dimensional point cloud dataset is determined; wherein, the pixel coordinates are the positions of each pixel in the monocular two-dimensional image, determined with pixels as the distance unit. Based on the spatial coordinate transformation relationship, determine the spatial coordinate transformation model; Before determining the hazard target information within the second difference area based on a preset hazard target identification model, the method further includes: Acquire several monocular two-dimensional images containing potential hazards to construct a hazard sample set; A hazard target identification algorithm is determined, and the algorithm is trained based on the hazard sample set to obtain a converged hazard target identification model; The potential hazards include at least one or more of the following: large construction machinery, poles, tower cranes, cranes, trees, and buildings.

2. The method for determining the degree of hidden danger in a power transmission line corridor according to claim 1, characterized in that, Based on the experimental monocular 2D images and the experimental 3D point cloud dataset, the parameters of the monocular camera are determined, specifically including: Based on the experimental monocular 2D images, the monocular camera is calibrated to determine the internal parameters of the monocular camera. Several experimental feature points are determined in the experimental monocular two-dimensional image, and several experimental discrete points corresponding to the several experimental feature points are determined in the experimental three-dimensional point cloud dataset; Based on the aforementioned experimental feature points and discrete experimental points, the external parameters of the monocular camera are determined using a camera pose estimation algorithm.

3. The method for determining the degree of hidden danger in a power transmission line corridor according to claim 1, characterized in that, Before determining, based on the third 3D point cloud dataset corresponding to the first difference region, that the first difference region intersects with a preset 3D channel monitoring area, the method further includes: Determine the discrete point set of the transmission line corresponding to the transmission line in the second three-dimensional point cloud dataset, and fit the discrete point set of the transmission line into a three-dimensional curve; The three-dimensional channel monitoring area is generated based on the safe distance between the three-dimensional curve and the corresponding voltage level of the transmission line.

4. The method for determining the degree of hidden danger in a power transmission line corridor according to claim 1, characterized in that, After determining the degree of hazard based on the distance from the first hazard target to the conductor, the method further includes: Determine the hazard level corresponding to the first hazard target; Add first hidden danger target information to the first hidden danger target in the second monocular two-dimensional image to generate a first hidden danger target information image; Based on the first hidden danger target information image and the degree of hidden danger corresponding to the first hidden danger target, an alarm notification is output to issue an alarm.

5. The method for determining the degree of hidden danger in a power transmission line corridor according to claim 1, characterized in that, The method further includes: If it is determined that there is no first difference region, the second coordinate information corresponding to the highest dangerous discrete point in the three-dimensional point cloud dataset is determined; Based on the second coordinate information, the distance from the dangerous discrete point to the transmission line is determined, and the degree of hazard is judged based on the distance from the dangerous discrete point to the transmission line.

6. The method for determining the degree of hidden danger in a power transmission line corridor according to claim 5, characterized in that, After determining the degree of hazard based on the distance from the discrete hazardous points to the conductor, the method further includes: The spatial coordinate transformation model is used to determine the coordinates of the dangerous discrete points in the second monocular two-dimensional image. Based on the aforementioned hazard target identification model, the type information of the second hazard target corresponding to the dangerous pixel coordinates is determined; Based on the type information of the second hidden danger target and the second coordinate information, the information of the second hidden danger target is determined.

7. A device for determining the degree of hidden danger in a power transmission line corridor, characterized in that, The device includes: processor; and a memory having executable code stored thereon, which, when executed, causes the processor to perform a method as described in any one of claims 1-6.

8. A non-volatile computer storage medium for determining the degree of hidden dangers in power transmission line corridors, storing computer-executable instructions, characterized in that, The computer-executable instructions are set as follows: In response to the periodic hazard level assessment command, a first monocular two-dimensional image and a first three-dimensional point cloud dataset of the transmission line channel are acquired in the first cycle, and a second monocular two-dimensional image and a second three-dimensional point cloud dataset of the transmission line channel are acquired in the second cycle; wherein, the second cycle is the cycle following the first cycle; the monocular two-dimensional image is obtained based on a monocular camera installed on the power pole, and the three-dimensional point cloud dataset is obtained based on a lidar installed on the power pole; The first 3D point cloud dataset and the second 3D point cloud dataset are input into a preset point cloud difference recognition model to determine whether there is a first difference region between the transmission line channel described by the second 3D point cloud dataset and the transmission line channel described by the first 3D point cloud dataset; wherein, the first difference region is a 3D region; wherein, the point cloud difference recognition model is trained by the kd-tree algorithm or the octree algorithm. If a first difference region is determined to exist, and if, based on the third 3D point cloud dataset corresponding to the first difference region, it is determined that the first difference region intersects with a preset 3D channel monitoring area, then, based on the third 3D point cloud dataset, a second difference region is determined in the second monocular 2D image through a preset spatial coordinate transformation model; wherein, the second difference region is the 2D difference region corresponding to the first difference region in the second monocular 2D image. Based on a preset hazard target identification model, the information of the first hazard target within the second difference area is determined; wherein, the information of the first hazard target includes at least: the type information of the first hazard target and the first coordinate information of the first hazard target; the first coordinate information is the coordinate information of the highest point of the first hazard target within the three-dimensional passage monitoring area; Based on the first coordinate information, the distance from the first potential hazard target to the transmission line is determined, and the degree of hazard is judged based on the distance from the first potential hazard target to the transmission line. Prior to responding to periodic hazard level assessment instructions, it also includes: The monocular camera acquires experimental monocular two-dimensional images, and the lidar acquires experimental three-dimensional point cloud datasets. Based on the experimental monocular 2D image and the experimental 3D point cloud dataset, the parameters of the monocular camera are determined; wherein, the parameters include external parameters and internal parameters. By jointly calculating the internal parameters and the external parameters, the spatial coordinate transformation relationship between the pixel coordinates in the experimental monocular two-dimensional image and the discrete points in the experimental three-dimensional point cloud dataset is determined; wherein, the pixel coordinates are the positions of each pixel in the monocular two-dimensional image, determined with pixels as the distance unit. Based on the spatial coordinate transformation relationship, determine the spatial coordinate transformation model; Before determining the hazard target information in the second difference area based on the preset hazard target identification model, the process also includes: Acquire several monocular two-dimensional images containing potential hazards to construct a hazard sample set; A hazard target identification algorithm is determined, and the algorithm is trained based on the hazard sample set to obtain a converged hazard target identification model; The potential hazards include at least one or more of the following: large construction machinery, poles, tower cranes, cranes, trees, and buildings.

Images