A method, device and medium for determining hidden danger targets based on power transmission channels

Abstract

The embodiments of this specification disclose a method, device and medium for determining hidden danger targets based on power transmission channels, and the method includes: obtaining the initial two-dimensional image and three-dimensional point cloud data of the power transmission channel collected in advance, marking the specified feature points in the initial two-dimensional image; collecting the real-time two-dimensional image of the power transmission channel, and determining the specified feature points at the same position in the real-time two-dimensional image; comparing the positions of the specified feature points in the real-time two-dimensional image with the specified feature points in the initial two-dimensional image, and determining the position deviation value of the same feature points; if the position deviation value exceeds the preset threshold, adjusting the pre-generated first mapping relationship to the second mapping relationship; if there is a hidden danger target in the real-time two-dimensional image, calculating the position information and distance information of the hidden danger target in the three-dimensional point cloud data according to the second mapping relationship. Timely update the mapping relationship between the two-dimensional image and the high-precision three-dimensional space scene to achieve accurate calculation and measurement of hidden danger targets.

Description

Technical Field

[0001] The present specification relates to the field of electric power technology, and in particular to a method, device and medium for determining hidden danger targets based on power transmission channels. Background Art

[0002] The transmission line channel refers to the strip area below the line with a specified width extending to both sides along the side conductors of the high-voltage overhead power line. It is crucial to fully understand the channel status of the transmission line map and promptly discover and eliminate potential safety hazards that may endanger the safe operation of the line in and outside the transmission line channel to ensure the safe and stable operation of the power grid.

[0003] In the operation and maintenance of transmission lines, it is an important task to monitor the environment around the transmission lines, such as checking whether there are super-high trees, illegal buildings, illegal construction, etc. In the transmission line channel, the construction machinery is required to be at least 10 meters away from the conductor. Discharge may occur within 3 meters, causing casualties or tripping. Large construction machinery in the transmission line channel hidden danger type, especially cranes and cement pump trucks in the raised or extended state, can easily pose a great threat to the conductor. It is necessary to identify such hidden dangers and make quantitative and qualitative judgments on the threat level of the line. The location and distance of the existing hidden danger targets are determined by real-time images collected by the camera. When the monitoring equipment is affected by strong winds and other factors that cause the posture of the transmission line to change, there will be errors in the two-dimensional image calculation alone, and the hidden danger cannot be accurately identified. Summary of the invention

[0004] One or more embodiments of the present specification provide a method, device and medium for determining hidden danger targets based on power transmission channels, which are used to solve the following technical problems: when the posture of the monitoring equipment changes due to strong winds or other factors, errors will occur only through two-dimensional image calculation, and hidden dangers cannot be accurately identified.

[0005] One or more embodiments of this specification adopt the following technical solutions:

[0006] One or more embodiments of the present specification provide a method for determining a hidden danger target based on a power transmission channel, the method comprising: obtaining an initial two-dimensional image and three-dimensional point cloud data of a power transmission channel collected in advance, and marking designated feature points in the initial two-dimensional image; collecting a real-time two-dimensional image of the power transmission channel, and determining designated feature points at the same position in the real-time two-dimensional image; comparing the positions of the designated feature points in the real-time two-dimensional image with the designated feature points in the initial two-dimensional image, and determining the position deviation value of the same feature points in the initial two-dimensional image and the real-time two-dimensional image; if the position deviation value of the same feature points in the initial two-dimensional image and the real-time two-dimensional image exceeds a preset threshold, adjusting a pre-generated first mapping relationship to a second mapping relationship; wherein the first mapping relationship represents a mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data, and the second mapping relationship represents a mapping relationship between the real-time two-dimensional image and the three-dimensional point cloud data; if there is a hidden danger target in the real-time two-dimensional image, calculating the position information and distance information of the hidden danger target in the three-dimensional point cloud data according to the second mapping relationship.

[0007] Furthermore, before adjusting the pre-generated first mapping relationship to the second mapping relationship, the method also includes: according to a manual point selection and calibration mechanism, forming feature point pairs corresponding to the same positions in the initial two-dimensional image data and the three-dimensional point cloud data; determining the internal parameters and external parameters of the monitoring equipment according to the feature point pairs, and associating the initial two-dimensional image data and the three-dimensional point cloud data to determine the first mapping relationship.

[0008] Furthermore, the internal parameters and external parameters of the monitoring device are determined based on the feature point pairs, and the initial two-dimensional image data and the three-dimensional point cloud data are associated to determine the first mapping relationship, which specifically includes: collecting the initial two-dimensional image of the transmission channel by the monitoring device, calibrating the monitoring device according to a specified calibration method, and obtaining the internal parameters of the monitoring device, wherein the internal parameters are used to determine the projection relationship of the monitoring device from three dimensions to two dimensions; determining the external parameters of the monitoring device based on the coordinate information of the feature point pairs in the three-dimensional point cloud data and the initial two-dimensional image data, wherein the external parameters are used to determine the relative position relationship between the monitoring device and the point cloud; determining the spatial coordinate conversion relationship between the three-dimensional point cloud data and the initial two-dimensional image based on the internal parameters and the external parameters; and determining the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data based on the spatial coordinate conversion relationship.

[0009] Furthermore, the external parameters of the monitoring device are determined based on the coordinate information of the feature point pairs in the three-dimensional point cloud data and the initial two-dimensional image data, specifically including: adjusting the viewing angle of the three-dimensional point cloud data according to the viewing angle of the initial two-dimensional image; obtaining the three-dimensional point cloud coordinates of the feature point pairs in the three-dimensional point cloud data, and obtaining the pixel coordinates of the feature point pairs in the initial two-dimensional image; and determining the external parameters of the monitoring device based on the three-dimensional point cloud coordinates and the pixel coordinates of the designated feature points.

[0010] Furthermore, determining the designated feature points at the same position in the real-time two-dimensional image specifically includes: taking the designated feature points in the initial two-dimensional image as the center, selecting a designated area as a first designated feature point area, and extracting target features in the first designated feature point area according to a preset algorithm; dividing the real-time two-dimensional image into multiple target areas, each of which has the same size as the first designated feature point area, and extracting target features of the multiple target areas in the real-time two-dimensional image according to a preset algorithm; determining the target area in the real-time two-dimensional image corresponding to the first designated feature point area according to the target features of the multiple target areas in the real-time two-dimensional image and the target features in the first designated feature point area, and determining the designated feature points at the same position in the target areas.

[0011] Furthermore, determining the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image specifically includes: determining the pixel position of the designated feature point in the initial two-dimensional image, and determining the pixel position of the feature point at the same position in the real-time two-dimensional image; calculating the pixel difference value based on the pixel position of the designated feature point in the initial two-dimensional image and the pixel position of the feature point at the same position in the real-time two-dimensional image, and using the pixel difference value as the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image.

[0012] Furthermore, the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image exceeds a preset threshold, specifically including: calculating the pixel difference between all specified feature points in the real-time two-dimensional image and the specified feature points in the initial two-dimensional image, and calculating the average value of the pixel difference values ​​of all specified feature points; if there is a feature point whose pixel difference value is higher than the preset deviation threshold, or the average value of the pixel difference value is higher than the preset average deviation threshold, it is determined that the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image exceeds the preset threshold.

[0013] Furthermore, after determining the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image, the method also includes: if the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image is within a preset threshold, then according to the first mapping relationship, calculating the position information and distance information of the hidden danger target in the real-time two-dimensional image in the three-dimensional point cloud data.

[0014] One or more embodiments of this specification provide a device for determining a hidden danger target based on a power transmission channel, including:

[0015] at least one processor; and,

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the above method.

[0018] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to:

[0019] Acquire an initial two-dimensional image and three-dimensional point cloud data of a power transmission channel collected in advance, and mark designated feature points in the initial two-dimensional image; collect a real-time two-dimensional image of the power transmission channel, and determine designated feature points at the same position in the real-time two-dimensional image; compare the positions of the designated feature points in the real-time two-dimensional image with those in the initial two-dimensional image, and determine the position deviation value of the same feature points in the initial two-dimensional image and the real-time two-dimensional image; if the position deviation value of the same feature points in the initial two-dimensional image and the real-time two-dimensional image exceeds a preset threshold, adjust the pre-generated first mapping relationship to a second mapping relationship; wherein the first mapping relationship represents the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data, and the second mapping relationship represents the mapping relationship between the real-time two-dimensional image and the three-dimensional point cloud data; if there is a hidden danger target in the real-time two-dimensional image, calculate the position information and distance information of the hidden danger target in the three-dimensional point cloud data according to the second mapping relationship.

[0020] At least one of the above-mentioned technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects: by calculating the position deviation value by comparing the real-time two-dimensional image with the specified feature points in the initial two-dimensional image, it is determined whether the posture of the monitoring equipment has changed, avoiding the need for manual on-site determination; in addition, the mapping relationship between the two-dimensional image and the three-dimensional point cloud data is used to calculate the position and distance of the hidden danger target, and the mapping relationship between the two-dimensional image and the high-precision three-dimensional space scene is updated in time, providing accuracy guarantee for the use of two-dimensional image information to realize the accurate calculation and measurement of hidden danger targets in the channel scene, providing more effective early warning information for the safety of the transmission line channel, and can greatly reduce the false alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0022] Figure 1 A schematic flow chart of a method for determining a hidden danger target based on a power transmission channel provided in an embodiment of this specification;

[0023] Figure 2 A flowchart of a manual point selection and calibration algorithm provided in an embodiment of this specification;

[0024] Figure 3 A schematic diagram of a two-dimensional image viewing angle provided in an embodiment of this specification;

[0025] Figure 4 A schematic diagram of a spatial coordinate transformation relationship provided in an embodiment of this specification;

[0026] Figure 5 A schematic flow chart of a dynamic adaptive calibration method for fusing a three-dimensional laser point cloud with a monocular vision image provided in an embodiment of this specification;

[0027] Figure 6 A schematic diagram of the structure of a hidden danger target determination device based on a power transmission channel provided in an embodiment of this specification. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0029] The transmission line channel refers to the strip area below the line with a specified width extending to both sides along the side conductor of the high-voltage overhead power line. It is crucial to fully grasp the channel status of the transmission line map, timely discover and eliminate the potential safety hazards that may endanger the safe operation of the line in and outside the transmission line channel, and ensure the safe and stable operation of the power grid. In the operation and maintenance of the transmission line, it is an important task to monitor the environment around the transmission line, such as checking whether there are super-high trees, illegal buildings, illegal construction, etc. The construction machinery in the transmission line channel is required to be at least 10 meters away from the conductor. Discharge may occur within 3 meters, causing casualties or tripping. Large construction machinery in the transmission line channel hidden danger type, especially cranes and cement pump trucks in the raised or extended state, can easily pose a great threat to the conductor. It is necessary to identify such hidden dangers and make quantitative and qualitative judgments on the threat level of the line. The location and distance of the existing hidden danger targets are determined by real-time images collected by the camera. When the monitoring equipment changes its posture due to other factors such as strong winds, there are errors in the two-dimensional image calculation alone, and the hidden danger cannot be accurately identified.

[0030] For the application scenario of overhead transmission channels, two-dimensional monocular images are collected by monitoring cameras installed on transmission towers, and three-dimensional point cloud data is constructed through point clouds collected by drones or helicopters. Without changing the installed online monitoring monocular camera, the monitoring system is reconstructed in three dimensions using the channel three-dimensional point cloud. It is necessary to calibrate the three-dimensional point cloud data and two-dimensional image data, establish a mapping relationship between the image and the high-precision three-dimensional spatial scene, and realize accurate calculation and measurement of hidden danger targets through the two-dimensional image position.

[0031] However, the point cloud acquisition equipment and the camera equipment are not integrated, resulting in the separation of point cloud data and image data. In most cases, they are not synchronous data and cannot be calibrated using customized calibration plates and calibration devices. It should be noted that in the image measurement process and machine vision applications, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, a geometric model of camera imaging must be established. These geometric model parameters are the camera parameters. Under most conditions, these parameters must be obtained through experiments and calculations. The process of solving the parameters is camera calibration. Through calibration, the world coordinates and pixel coordinates of the calibration control points are known to solve this mapping relationship. The world coordinates can be inferred from the pixel coordinates of the point, and other subsequent operations such as measurement can be performed based on the obtained world coordinates.

[0032] In addition, as time goes by, the monocular camera equipment fixedly installed at the monitoring point is easily affected by the external environment, such as strong wind vibration, loose screws, equipment maintenance and other factors, which cause the equipment's own posture to change. At this time, the initial calibration of the two-dimensional and three-dimensional spatial mapping relationship will not represent the real two-dimensional scene, resulting in a large deviation. During the actual operation of the equipment, it is impossible to calibrate at any time, and an automatic detection method is required to achieve the purpose of adaptive calibration. The existing calibration method uses metal balls as calibration objects to perform external parameter calibration methods for monocular cameras and millimeter-wave radars, and relies on the selected calibration objects and environment to obtain data from millimeter-wave radars and cameras, project point cloud data to the camera coordinate system, and adjust external parameters by dragging the slider. Most of them are suitable for scenes such as autonomous driving and drone mapping. Moreover, based on the matching method of three-dimensional features and two-dimensional features, the design of features and matching requires great care, and the calibration work depends on specially arranged specific calibration objects and environments, which is not universal.

[0033] The embodiment of this specification provides a method for determining hidden danger targets based on power transmission channels. It should be noted that the execution subject of the embodiment of this method can be a processor or other devices with processing capabilities. Figure 1 As shown, it mainly includes the following steps:

[0034] Step S101, obtaining an initial two-dimensional image and three-dimensional point cloud data of a power transmission channel collected in advance, and marking designated feature points in the initial two-dimensional image.

[0035] In one embodiment of the present specification, the initial two-dimensional image of the transmission channel is collected by a monitoring device, which can be installed at the transmission tower and can be a monocular camera; the point cloud of the transmission channel is collected by a drone or a helicopter to construct three-dimensional point cloud data. It should be noted that in the actual operation process, the monocular camera is in a real-time shooting state, and the drone or helicopter collects point clouds at a longer time interval, for example, once every six months.

[0036] The designated feature points are determined in the initial two-dimensional image. The designated feature points are strong feature points in the transmission channel, such as target corner points, inflection points, vertices and other special points of the transmission tower, and can also be target corner points, inflection points, vertices and the like of buildings in the image.

[0037] In one embodiment of the present specification, the initial two-dimensional image data and the three-dimensional point cloud data are associated according to the manual point selection calibration mechanism to determine the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data. In the existing calibration methods of three-dimensional point clouds and two-dimensional images, a specially customized calibration device is required, and it relies on reflective media, and the stability of the three-dimensional laser scanning data cannot be guaranteed; in addition, the embodiment of the present specification adopts a manual point selection calibration algorithm, which selects strong feature points, fuses the three-dimensional laser point cloud data with the two-dimensional image collected by the monocular camera, and performs joint calibration. The calibration process is as follows: Figure 2 shown.

[0038] First, the initial two-dimensional image and three-dimensional point cloud data corresponding to the transmission channel are obtained. The monitoring equipment can be calibrated by Zhang Zhengyou calibration method to determine the internal parameters of the monitoring equipment, where the internal parameters are parameters related to the characteristics of the monitoring equipment itself, and are used to determine the projection relationship between the monitoring equipment from three dimensions to two dimensions. The internal parameters of the monitoring equipment can include focal length parameters fx, fy and optical center parameters cx, cy, and can also be other related parameters that can determine the projection relationship between the monitoring equipment from three dimensions to two dimensions.

[0039] Secondly, read the 3D point cloud data and adjust the viewing angle in the 3D point cloud data according to the visual angle of the initial 2D image so that the viewing angles of the two are consistent. The viewing angle of the 2D image is as follows: Figure 3 As shown in , the perspective of the 3D point cloud data is adjusted to be consistent with the perspective of the 2D image. After adjusting the perspective, the feature points in the collected transmission channel scene are selected based on the initial 2D image. It should be noted that these feature points are strong feature points, such as the target corner points, inflection points, vertices and other special points of the transmission tower, and can also be the target corner points, inflection points, vertices, etc. of the buildings in the image, such as Figure 3 Point 1, point 2, point 3 and point 4 are four exemplary feature points, and the number and position of the feature points are not specifically limited. The feature point positions at the same position are found in the initial two-dimensional image and the three-dimensional point cloud data respectively, and the feature points corresponding to the same positions in the initial two-dimensional image data and the three-dimensional point cloud data are combined into feature point pairs, and the number of feature point pairs is not less than 4 groups. It should be noted that the same position refers to the same position in the scene, for example, the position of the vertex of the transmission tower in the initial two-dimensional image is found, and then the position of the vertex of the transmission tower in the three-dimensional point cloud data is found.

[0040] After determining the positions of the feature points at the same position in the initial two-dimensional image and the three-dimensional point cloud data, determine the three-dimensional point cloud coordinates of the feature points in the three-dimensional point cloud data, and determine the pixel coordinates of the feature points in the initial two-dimensional image, and combine the three-dimensional point cloud coordinates and pixel coordinates of the feature points to determine the external parameters of the camera. It should be noted that the external parameters of the camera can be referred to as camera extrinsics, which are the parameters of the camera in the world coordinate system, such as the position and rotation direction of the camera. The relative position relationship between the monitoring device and the point cloud can be determined through the external parameters.

[0041] According to the internal and external parameters of the camera, the spatial coordinate conversion relationship between the three-dimensional point cloud data and the initial two-dimensional image is determined. Among them, the spatial coordinate conversion relationship is described as follows: the origin coordinate system of the three-dimensional laser point cloud is used as the unified world coordinate system, which is defined as Xw, Yw, and Zw, and the unit is the length unit. The camera coordinate system uses the optical center as the origin of the camera coordinate system, and the x and y directions parallel to the two-dimensional image as the Xc axis and Yc axis. The Zc axis is parallel to the optical axis, and Xc, Yc, and Zc are perpendicular to each other. The unit is the length unit. The physical coordinate system of the image uses the intersection of the principal optical axis and the image plane as the coordinate origin. The x and y directions are as follows: Figure 4 As shown, the unit is the length unit. The image pixel coordinate system takes the vertex of the image as the coordinate origin, the u and v directions are parallel to the x and y directions, and the unit is in pixels. The final coordinate transformation relationship is:

[0042]

[0043] Among them, R is the rotation matrix with 3 degrees of freedom, t is the translation matrix, and the two form a 3×4 matrix, which is the external parameter matrix of the camera. fx, fy, cx, and cy are the internal parameters of the camera.

[0044] According to the spatial coordinate conversion relationship between the obtained 3D point cloud data and the initial 2D image, the mapping relationship between the initial 2D image and the 3D point cloud data is determined, so that the mapping relationship can represent the real 3D scene. Without changing the installed online monitoring equipment, the monitoring system is reconstructed in 3D using the channel 3D point cloud obtained by the point cloud acquisition device, and dynamic adaptive calibration is achieved for the 3D point cloud data and 2D image data. There is no need to calibrate using a customized calibration plate or calibration device. Only the 2D image and the corresponding 3D point cloud data information are needed. There is no need to calibrate on site, which saves calibration costs.

[0045] Step S102: collect a real-time two-dimensional image of the power transmission channel, and determine designated feature points at the same position in the real-time two-dimensional image.

[0046] In one embodiment of the present specification, a real-time two-dimensional image of a power transmission channel is captured in real time by a monitoring device. In the actual operation of the device, it is necessary to collect the real-time two-dimensional image of the power transmission channel by the monitoring device in real time to detect and calculate hidden dangers. After collecting the real-time two-dimensional image of the power transmission channel, the feature points at the same position are determined in the real-time two-dimensional image. It should be noted that the same position here refers to the same position in the scene. For example, the position of the vertex of the transmission tower in the initial two-dimensional image is determined, and the position of the vertex of the transmission tower in the real-time two-dimensional image is also determined.

[0047] In the actual operation of the equipment, due to strong winds, loose screws, etc., the monitoring equipment may change its posture, resulting in differences between the real-time pictures collected and the pictures taken during the initial calibration. In other words, the mapping relationship obtained during the initial calibration cannot represent the actual application scenario. Therefore, the embodiment of this specification adopts an adaptive calibration algorithm, taking the strong feature points in the initial manually selected two-dimensional image as the benchmark, and using the initial two-dimensional image used for calibration as the benchmark image, and matching the strong feature points with the real-time two-dimensional image regularly obtained from the monitoring equipment in real time, and using a feature matching algorithm with rotation, scale scaling, and affine transformation feature invariance to detect and obtain the new position of the feature points in the real-time two-dimensional image, and calculate the point deviation.

[0048] In one embodiment of the present specification, a designated area is selected as a first designated feature point area with a designated feature point in the initial two-dimensional image as the center, and target features in the first designated feature point area are extracted according to a preset algorithm. For example, a 50*50 area is selected as the first designated feature point area with the vertex of the transmission tower in the initial two-dimensional image as the center, and the target features in each area are extracted using ORB (Oriented FAST and Rotated BRIEF) or other algorithms. It should be noted that ORB is an algorithm for fast feature point extraction and description, with fast calculation speed, which not only saves storage space, but also greatly shortens the matching time.

[0049] The real-time two-dimensional image is divided into multiple target areas, each of which has the same size as the first designated feature point area, and the target features of the multiple target areas in the real-time two-dimensional image are extracted according to a preset algorithm. For example, the real-time two-dimensional image is divided into multiple 50*50 areas, and the target features of each area are extracted according to the ORB or other algorithms in each area. The first designated feature point area in the initial two-dimensional image is used as a reference to determine the target area that matches it, that is, the target area corresponding to the first designated feature point area in the real-time two-dimensional image is determined. Then, the designated feature points at the same position are determined in the target area. The target feature comparison is performed by selecting an area with the feature point as the center, which reduces the workload of calculation. The workload is reduced by first determining the target area and then determining the designated feature point.

[0050] Step S103 , comparing the positions of the designated feature points in the real-time two-dimensional image with the designated feature points in the initial two-dimensional image, and determining the position deviation values ​​of the same feature points in the initial two-dimensional image and the real-time two-dimensional image.

[0051] In one embodiment of the present specification, the pixel position of a designated feature point in an initial two-dimensional image is determined, and the pixel position of the feature point at the same position in a real-time two-dimensional image is determined. According to the pixel position of the designated feature point in the initial two-dimensional image and the pixel position of the feature point at the same position in the real-time two-dimensional image, a pixel difference is calculated, and the pixel difference is used as the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image. For example, the pixel position of the vertex of the transmission tower in the initial two-dimensional image is determined as the first position, and the pixel position of the vertex of the transmission tower in the real-time two-dimensional image is determined as the second position. The first position is used as the reference position, and the position deviation generated by the second position is calculated. The position deviation between the second position and the first position can be determined by the pixel deviation.

[0052] In one embodiment of the present specification, since there are multiple strong feature points in the initial two-dimensional image and the real-time two-dimensional image, there are two methods for determining the pixel difference. One is to calculate the pixel difference between each specified feature point in the real-time two-dimensional image and the specified feature point at the same position in the initial two-dimensional image. The other is to calculate the average of the pixel differences of all the specified feature points.

[0053] Step S104: if the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image exceeds a preset threshold, the pre-generated first mapping relationship is adjusted to a second mapping relationship.

[0054] It should be noted that the first mapping relationship represents the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data, and the second mapping relationship represents the mapping relationship between the real-time two-dimensional image and the three-dimensional point cloud data.

[0055] In one embodiment of the present specification, if there is a pixel difference of a feature point higher than a preset deviation threshold, or the average value of the pixel difference is higher than the preset average deviation threshold, it is determined that the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image exceeds the preset threshold. In actual application scenarios, when the mean deviation of all strong feature points is greater than 3 pixels, or the maximum deviation of the strong feature points is greater than 10 pixels, it means that there is a large change between the real-time two-dimensional image collected by the monitoring device and the initial two-dimensional image, that is, the position of the current monitoring device has changed significantly, and the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data can no longer reflect the real three-dimensional scene of the power transmission channel. Therefore, it is necessary to update the mapping relationship to the mapping relationship between the real-time two-dimensional image and the three-dimensional point cloud data.

[0056] In one embodiment of the present specification, if the position deviation value between the feature points in the real-time two-dimensional image and the feature points in the initial two-dimensional image does not exceed the preset threshold, it means that the position and posture of the monitoring device at this time are basically unchanged from the initial shooting, or in other words, the change is small and does not affect the calculation result. At this time, the hidden danger target is determined by the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud. In the actual operation of the equipment, the posture changes of the equipment itself caused by the influence of the external environment are eliminated, and the difficulty of matching three-dimensional feature points with two-dimensional feature points is converted into feature matching between two-dimensional and two-dimensional, realizing real-time automatic monitoring of the equipment position and automatic calibration and correction.

[0057] Step S105: If there is a hidden danger target in the real-time two-dimensional image, the position information and distance information of the hidden danger target are calculated in the three-dimensional point cloud data according to the second mapping relationship.

[0058] The monitoring equipment collects images of the transmission channel in real time to determine whether there are hidden danger targets. The images collected by the monitoring equipment are two-dimensional images. Depending on the differences in the shooting angle and shooting direction of the monitoring equipment, the collected images are also different. In addition, when determining the hidden danger target, it is necessary to determine whether the hidden danger target will cause damage to the transmission channel based on the location, distance and other information of the hidden danger target. There will be large errors in determining the location of the target and calculating the distance in the two-dimensional image.

[0059] In one embodiment of the present specification, if there are hidden danger targets in the real-time two-dimensional image, its position information and distance information can be calculated in the three-dimensional point cloud data according to the preset mapping relationship, and the two-dimensional image and three-dimensional data can be combined, and the mapping relationship between the two-dimensional image and the high-precision three-dimensional space scene can be updated in time, so as to provide accuracy guarantee for the accurate calculation and measurement of hidden danger targets in the channel scene using two-dimensional image information, thereby providing more effective early warning information for the safety of the transmission line channel, which can greatly reduce the false alarm rate.

[0060] Figure 5 The flowchart of the dynamic adaptive calibration method for fusion of 3D laser point cloud and monocular vision image provided in this manual is as follows: Figure 5 As shown, it mainly includes the following steps:

[0061] First, the manual point selection and calibration mechanism is used to perform initial manual point selection, and the strong feature points of the two-dimensional image and the three-dimensional image are used to form feature point pairs. The initial monocular camera two-dimensional image and the three-dimensional laser point cloud are calibrated to obtain the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud, and the initial two-dimensional image is used as the two-dimensional reference image. The feature point area is determined in the two-dimensional reference image. It should be noted that the range of the feature point area can be set according to the actual situation. For example, a 50*50 range can be selected as the feature point area with the strong feature point as the center.

[0062] During the actual operation of the equipment, the real-time collected two-dimensional images and the initial calibrated two-dimensional images are regularly registered to detect the changes in the installation posture of the monocular camera caused by environmental changes. The offset error is calculated using the first manually selected position points. When it exceeds the threshold, automatic calibration is started and the mapping relationship after calibration is updated. Based on the initial manually calibrated position point pairs, the difficult problem of matching 3D feature points with 2D feature points is converted into 2D and 2D feature matching to achieve automatic point selection and calibration.

[0063] By using the manually selected point area, the feature extraction algorithm is designed to achieve fast registration of two-dimensional images and extract the target point position of the same feature of the new two-dimensional image. Together with the initially selected three-dimensional points, a new 3D-2D matching point pair is formed and recalibrated.

[0064] The embodiment of this specification also provides a device for determining hidden danger targets based on power transmission channels, such as Figure 6 As shown, it includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method in any embodiment.

[0065] The embodiments of the present specification also provide a non-volatile computer storage medium, storing computer executable instructions, wherein the computer executable instructions are configured to: obtain an initial two-dimensional image and three-dimensional point cloud data of a pre-collected power transmission channel, and mark designated feature points in the initial two-dimensional image; collect a real-time two-dimensional image of the power transmission channel, and determine designated feature points at the same position in the real-time two-dimensional image; compare the positions of the designated feature points in the real-time two-dimensional image with the designated feature points in the initial two-dimensional image, and determine the position deviation value of the same feature points in the initial two-dimensional image and the real-time two-dimensional image; if the position deviation value of the same feature points in the initial two-dimensional image and the real-time two-dimensional image exceeds a preset threshold, adjust the pre-generated first mapping relationship to a second mapping relationship; wherein the first mapping relationship represents the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data, and the second mapping relationship represents the mapping relationship between the real-time two-dimensional image and the three-dimensional point cloud data; if there is a hidden danger target in the real-time two-dimensional image, calculate the position information and distance information of the hidden danger target in the three-dimensional point cloud data according to the second mapping relationship.

[0066] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0067] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0068] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A method for determining hidden danger targets based on power transmission channels, characterized in that: The method comprises: Acquire an initial two-dimensional image and three-dimensional point cloud data of a power transmission channel collected in advance, and mark designated feature points in the initial two-dimensional image, wherein the designated feature points are strong feature points in the power transmission channel; Collecting a real-time two-dimensional image of the power transmission channel, and determining designated feature points at the same position in the real-time two-dimensional image; Comparing the positions of the designated feature points in the real-time two-dimensional image with the designated feature points in the initial two-dimensional image, and determining the position deviation values ​​of the same feature points in the initial two-dimensional image and the real-time two-dimensional image; If the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image exceeds a preset threshold, adjusting the pre-generated first mapping relationship to a second mapping relationship; The first mapping relationship represents the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data, and the second mapping relationship represents the mapping relationship between the real-time two-dimensional image and the three-dimensional point cloud data; If there is a hidden danger target in the real-time two-dimensional image, calculating the position information and distance information of the hidden danger target in the three-dimensional point cloud data according to the second mapping relationship; Determining the designated feature points at the same position in the real-time two-dimensional image specifically includes: Taking the designated feature point in the initial two-dimensional image as the center, selecting a designated area as a first designated feature point area, and extracting target features in the first designated feature point area according to a preset algorithm; Dividing the real-time two-dimensional image into a plurality of target areas, each of which has the same size as the first designated feature point area, and extracting target features of the plurality of target areas in the real-time two-dimensional image according to a preset algorithm; According to the target features of multiple target areas in the real-time two-dimensional image and the target features in the first designated feature point area, the target area corresponding to the first designated feature point area in the real-time two-dimensional image is determined, and the designated feature points at the same position are determined in the target area.

2. A method for determining hidden danger targets based on power transmission channels according to claim 1, characterized in that: Before adjusting the pre-generated first mapping relationship to the second mapping relationship, the method further includes: According to the manual point selection and calibration mechanism, the feature points corresponding to the same positions in the initial two-dimensional image data and the three-dimensional point cloud data are combined into feature point pairs; The internal parameters and external parameters of the monitoring device are determined according to the feature point pairs, and the initial two-dimensional image data and the three-dimensional point cloud data are associated to determine the first mapping relationship.

3. A method for determining hidden danger targets based on power transmission channels according to claim 2, characterized in that: The step of determining the internal parameters and the external parameters of the monitoring device according to the feature point pair, and associating the initial two-dimensional image data with the three-dimensional point cloud data to determine the first mapping relationship specifically includes: The initial two-dimensional image of the power transmission channel is collected by a monitoring device, and the monitoring device is calibrated according to a specified calibration method to obtain internal parameters of the monitoring device, wherein the internal parameters are used to determine the projection relationship between the monitoring device from three dimensions to two dimensions; Determine the external parameters of the monitoring device according to the coordinate information of the feature point pair in the three-dimensional point cloud data and the initial two-dimensional image data, wherein the external parameters are used to determine the relative position relationship between the monitoring device and the point cloud; Determine a spatial coordinate transformation relationship between the three-dimensional point cloud data and the initial two-dimensional image according to the internal parameters and the external parameters; According to the spatial coordinate conversion relationship, a mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data is determined.

4. The method for determining hidden danger targets based on power transmission channels according to claim 3 is characterized in that: Determining the external parameters of the monitoring device according to the coordinate information of the feature point pair in the three-dimensional point cloud data and the initial two-dimensional image data specifically includes: Adjusting the viewing angle of the three-dimensional point cloud data according to the viewing angle of the initial two-dimensional image; Acquire the three-dimensional point cloud coordinates of the feature point pair in the three-dimensional point cloud data, and acquire the pixel coordinates of the feature point pair in the initial two-dimensional image; The external parameters of the monitoring device are determined based on the three-dimensional point cloud coordinates and the pixel coordinates of the designated feature points.

5. The method for determining hidden danger targets based on power transmission channels according to claim 1, characterized in that: The determining of the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image specifically includes: Determine the pixel position of the designated feature point in the initial two-dimensional image, and determine the pixel position of the feature point at the same position in the real-time two-dimensional image; A pixel difference is calculated based on the pixel position of the designated feature point in the initial two-dimensional image and the pixel position of the feature point at the same position in the real-time two-dimensional image, and the pixel difference is used as the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image.

6. A method for determining hidden danger targets based on power transmission channels according to claim 5, characterized in that: The position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image exceeds a preset threshold, specifically including: Calculating pixel differences between all designated feature points in the real-time two-dimensional image and designated feature points in the initial two-dimensional image, and calculating an average value of the pixel differences of all designated feature points; If there is a feature point whose pixel difference is higher than a preset deviation threshold, or the average value of the pixel difference is higher than a preset average deviation threshold, it is determined that the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image exceeds the preset threshold.

7. The method for determining hidden danger targets based on power transmission channels according to claim 1, characterized in that: After determining the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image, the method further includes: If the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image is within a preset threshold, the position information and distance information of the hidden danger target in the real-time two-dimensional image are calculated in the three-dimensional point cloud data according to the first mapping relationship.

8. A device for determining hidden danger targets based on power transmission channels, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

9. A non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to: Acquire an initial two-dimensional image and three-dimensional point cloud data of a power transmission channel collected in advance, and mark designated feature points in the initial two-dimensional image, wherein the designated feature points are strong feature points in the power transmission channel; Collecting a real-time two-dimensional image of the power transmission channel, and determining designated feature points at the same position in the real-time two-dimensional image; Comparing the positions of the designated feature points in the real-time two-dimensional image with the designated feature points in the initial two-dimensional image, and determining the position deviation values ​​of the same feature points in the initial two-dimensional image and the real-time two-dimensional image; If the position deviation value of the same feature point in the initial two-dimensional image and the real-time two-dimensional image exceeds a preset threshold, adjusting the pre-generated first mapping relationship to a second mapping relationship; The first mapping relationship represents the mapping relationship between the initial two-dimensional image and the three-dimensional point cloud data, and the second mapping relationship represents the mapping relationship between the real-time two-dimensional image and the three-dimensional point cloud data; If there is a hidden danger target in the real-time two-dimensional image, calculating the position information and distance information of the hidden danger target in the three-dimensional point cloud data according to the second mapping relationship; Determining the designated feature points at the same position in the real-time two-dimensional image specifically includes: Taking the designated feature point in the initial two-dimensional image as the center, selecting a designated area as a first designated feature point area, and extracting target features in the first designated feature point area according to a preset algorithm; Dividing the real-time two-dimensional image into a plurality of target areas, each of which has the same size as the first designated feature point area, and extracting target features of the plurality of target areas in the real-time two-dimensional image according to a preset algorithm; According to the target features of multiple target areas in the real-time two-dimensional image and the target features in the first designated feature point area, the target area corresponding to the first designated feature point area in the real-time two-dimensional image is determined, and the designated feature points at the same position are determined in the target area.

Images