Method, device, electronic device and storage medium for identifying hidden dangers of external damage to power transmission cables

Abstract

The present invention relates to the field of image processing technology, and provides a method, device, electronic device and storage medium for identifying hidden dangers of external damage to power transmission cables, the method comprising: obtaining a monitoring image for monitoring construction machinery and power transmission cables, the monitoring image comprising a first image area where the construction machinery is located and a second image area where the power transmission cables are located; if the first image area and the second image area are at the same relative depth in the monitoring image, then calculating the actual distance between the construction machinery and the power transmission cables according to the first image area and the second image area; the relative depth represents the front-to-back relationship between the construction machinery in the first image area and the power transmission cables in the second image area in the monitoring image; if the actual distance is less than a preset threshold, then determining that the power transmission cables have hidden dangers of external damage. The present invention can improve the accuracy of identifying hidden dangers of external damage to power transmission cables, and accurately warn of hidden dangers of external damage to power transmission cables.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for identifying hidden dangers of external damage to a power transmission cable. Background Art

[0002] In infrastructure construction sites, construction machinery such as cranes and hoists can easily cause physical damage to overhead transmission cables during construction, which is called external damage of transmission cables. In order to avoid external damage, transmission cables are usually divided into predefined danger zones. The target detection model is used to detect whether construction machinery appears in the monitoring image of the construction site, and to determine whether the construction machinery appears in the predefined danger zone. If construction machinery appears in the monitoring image and appears in the predefined danger zone, it is considered that there is a risk of external damage, which is reported to the remote monitoring center, and an audio-visual alarm is executed on site.

[0003] However, the target detection model can only distinguish the position and category of construction machinery in the two-dimensional image, and cannot accurately measure whether the construction machinery actually poses a threat to the transmission cables. Therefore, the false alarm rate of external damage alarm is high. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, electronic device and storage medium for identifying hidden dangers of external damage of transmission cables, which can improve the accuracy of identifying hidden dangers of external damage of transmission cables and accurately warn of hidden dangers of external damage of transmission cables.

[0005] The embodiments of the present invention can be implemented as follows:

[0006] In a first aspect, the present invention provides a method for identifying hidden dangers of external damage to a power transmission cable, the method comprising:

[0007] Acquire a monitoring image for monitoring a construction machine and a power transmission cable, wherein the monitoring image includes a first image area where the construction machine is located and a second image area where the power transmission cable is located;

[0008] If the first image area and the second image area are at the same relative depth in the monitoring image, the actual distance between the construction machine and the transmission cable is calculated based on the first image area and the second image area; the relative depth represents the front-to-back relationship between the construction machine in the first image area and the transmission cable in the second image area in the monitoring image;

[0009] If the actual distance is less than a preset threshold, it is determined that the power transmission cable has a risk of external damage.

[0010] In an optional implementation manner, after the step of acquiring monitoring images for monitoring the construction machinery and the power transmission cables, the method further includes:

[0011] Acquire a depth map of the surveillance image;

[0012] Obtaining a relative depth value of the first image area according to relative depth values ​​of all pixels in the first image area in the depth map;

[0013] Determine, from a plurality of pixels in the depth map, a target pixel whose relative depth value is within a preset range determined by the relative depth value of the first image area, and use an area covered by pixels corresponding to the target pixel in the monitoring image as a reference area;

[0014] If the second image area is within the reference area or the second image area is the same as the reference area, it is determined that the first image area and the second image area are at the same relative depth in the monitoring image.

[0015] In an optional implementation, the step of calculating an average value of relative depth values ​​of all pixels in the first image area in the depth map to obtain the relative depth value of the first image area includes:

[0016] Taking an area in the depth map that overlaps with the first image area and has the same size as the first image area as an intersection area;

[0017] An average value of the relative depth values ​​of all pixels in the intersection area is calculated to obtain a relative depth value of the first image area.

[0018] In an optional implementation, the step of calculating the actual distance between the construction machine and the power transmission cable according to the first image area and the second image area includes:

[0019] calculating a pixel distance between the first image area and the second image area;

[0020] Performing spatial transformation on the pixel distance to obtain the actual distance.

[0021] In an optional implementation, the step of calculating the pixel distance between the first image area and the second image area includes:

[0022] Acquire a first contour of the first image area and a second contour of the second image area respectively, wherein the first contour includes a plurality of first pixel points, and the second contour includes a plurality of second pixel points;

[0023] Calculating the distance between each of the first pixel points and each of the second pixel points to obtain a plurality of point pair distances between the first pixel points and the second pixel points;

[0024] The minimum distance among all point pair distances is taken as the pixel distance.

[0025] In an optional implementation manner, the step of performing spatial conversion on the pixel distance to obtain the actual distance includes:

[0026] Obtaining a spatial conversion relationship according to a pixel size of the first image area and an actual size of the construction machinery;

[0027] The pixel distance is converted into the actual distance according to the space conversion relationship.

[0028] In an optional implementation manner, the step of performing spatial conversion on the pixel distance to obtain the actual distance includes:

[0029] Acquire a first target pixel point and a second target pixel point corresponding to the pixel distance;

[0030] According to a preset camera intrinsic parameter matrix, coordinate conversion is performed on the first image coordinates of the first target pixel point and the second image coordinates of the second target pixel point to obtain the first spatial coordinates of the first target pixel point in a world coordinate system and the second spatial coordinates of the second target pixel point in the world coordinate system;

[0031] The distance between the first space coordinate and the second space coordinate is calculated to obtain the actual distance.

[0032] In a second aspect, the present invention provides a device for identifying hidden dangers of external damage to a power transmission cable, the device comprising:

[0033] An acquisition module, used to acquire a monitoring image for monitoring a construction machine and a power transmission cable, wherein the monitoring image includes a first image area where the construction machine is located and a second image area where the power transmission cable is located;

[0034] a calculation module, configured to calculate the actual distance between the construction machine and the power transmission cable according to the first image area and the second image area if the first image area and the second image area are at the same relative depth in the monitoring image; the relative depth represents the front-to-back relationship between the construction machine in the first image area and the power transmission cable in the second image area in the monitoring image;

[0035] The identification module is used to determine that the transmission cable has a hidden danger of external damage if the actual distance is less than a preset threshold.

[0036] In a third aspect, the present invention provides an electronic device, including a processor and a memory, wherein the memory is used to store a program, and the processor is used to implement a method for identifying hidden dangers of external damage to a transmission cable as described in any of the aforementioned embodiments when executing the program.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for identifying hidden dangers of external damage to a transmission cable as described in any one of the aforementioned embodiments.

[0038] The present invention monitors the construction machinery and the transmission cable to obtain a monitoring image. If the first image area where the construction machinery is located and the second image area where the transmission cable is located are at the same relative depth in the monitoring image, the actual distance between the construction machinery and the transmission cable is calculated based on the first image area and the second image area. If the actual distance is less than a preset threshold, it is determined that the transmission cable has a hidden danger of external damage. The present invention uses the relative depth that characterizes the front-to-back relationship between the construction machinery and the transmission cable in the monitoring image, and calculates the actual distance between the two when the two are at the same relative depth. The actual distance calculated at this time is more reference for whether the transmission cable has a hidden danger of external damage, and ultimately a more accurate identification result of the hidden danger of external damage of the transmission cable can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 This is an example flowchart of the method for identifying hidden dangers of external damage to power transmission cables provided in this embodiment.

[0041] Figure 2 This is a structural diagram of the binocular depth estimation model provided in this embodiment.

[0042] Figure 3 This is an example diagram of a monitoring image of an excavator construction site provided in this embodiment.

[0043] Figure 4 This is a block diagram of an example of a device for identifying hidden dangers of external damage to a transmission cable provided in this embodiment.

[0044] Figure 5 An example block diagram of an electronic device provided in this embodiment.

[0045] Icons: 10 - electronic device; 11 - processor; 12 - memory; 13 - bus; 100 - device for identifying hidden dangers of external damage to power transmission cables; 110 - acquisition module; 120 - calculation module; 130 - identification module. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0047] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0049] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0050] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0051] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0052] In order to accurately identify the hidden dangers of power transmission cable damage, one method is to first use the conductor semantic segmentation model and the monocular depth estimation model to reconstruct the dense 3D point cloud of the power transmission cable, and then use the construction machinery target detection model and the depth map to estimate the spatial coordinates of the dangerous construction machinery. Finally, the shortest Euclidean distance between the construction machinery and the power transmission cable is calculated in the 3D world coordinate system to identify the hidden dangers of power transmission cable damage. This method has the following defects:

[0053] (1) Due to the low accuracy and generalization ability of absolute depth estimation, the application is limited. Using a monocular depth estimation model to directly estimate the absolute pixel distance has low accuracy, poor precision, and lacks generalization performance. It can only be applied to specific models of cameras in specific installation locations. In addition, the production cost of the supervised dataset required for absolute depth estimation is high and difficult, and 3D reconstruction requires accurate internal calibration, which makes it difficult to deploy and apply widely.

[0054] (2) Although the target detection result includes the spatial coordinates of the target, it lacks the geometric shape of the target. Direct calculation will introduce background noise and increase the error of the three-dimensional spatial position calculation.

[0055] (3) When calculating the shortest distance between construction machinery and transmission cables, it is necessary to calculate the Euclidean distance of each sampling point in the corresponding target box to all conductor point clouds. This has high time complexity and large amount of calculation, which is not conducive to real-time application.

[0056] In view of this, the present embodiment provides a method, device, electronic device and storage medium for identifying hidden dangers of external damage of power transmission cables, which does not require three-dimensional reconstruction, avoids complex internal parameter calibration, does not use absolute depth, avoids low estimation accuracy, high cost and difficulty caused by absolute depth, and when calculating the shortest distance, it is not based on the cloud computing of each sampling point in the target box and all wire points, which reduces the amount of calculation and time complexity. It will be introduced in detail below.

[0057] Please refer to Figure 1 , Figure 1 The following is a flow chart of a method for identifying a potential risk of external damage to a power transmission cable provided in this embodiment. The method includes the following steps:

[0058] Step S101, obtaining a monitoring image for monitoring a construction machine and a power transmission cable, wherein the monitoring image includes a first image area where the construction machine is located and a second image area where the power transmission cable is located.

[0059] In this embodiment, the monitoring image may be an image taken by a camera of the construction machinery and the transmission cable, or a video frame in a video taken by a camera of the construction machinery and the transmission cable. The construction machinery includes but is not limited to excavators, cranes, bulldozers, loaders, etc. The transmission cable may be a cable hung in the air within the working range of the construction site.

[0060] In this embodiment, a plurality of construction machines and a plurality of sections of transmission cables may be monitored in one monitoring image. In this case, there will be a plurality of first image areas and a plurality of second image areas in the monitoring image.

[0061] As an implementation method, the first image area can be obtained by detecting and segmenting each construction machine using a construction machinery instance segmentation model, and the second image area can be obtained by detecting and segmenting each section of transmission cable using a transmission cable instance segmentation model. Both the construction machinery instance segmentation model and the transmission cable instance segmentation model can be based on MaskDINO, YOLOV8, and trained by pre-prepared corresponding high-quality annotated data sets.

[0062] For example, for construction machinery, each construction machinery appearing in the monitoring image is detected and segmented. The first image area corresponding to each construction machinery can be represented by the target constraint box Bm. The target constraint box Bm represents the minimum circumscribed rectangle of the machinery instance, which can be represented by the pixel coordinates of the upper left and lower right vertices. In addition, the construction machinery instance segmentation model can also identify the category of each construction machinery (represented by Cm) and its mask (represented by Mm). The value range of Cm is [0, 1,2, ..., N-1], where N is the total number of categories of construction machinery in the monitoring image. Each value corresponds to a specific category, and the order of each category is not mandatory. The mask Mm represents the binary segmentation map of the corresponding construction machinery. The pixel value 0 represents the background and 1 represents the target. The size is consistent with the input image.

[0063] For another example, for power transmission cables, each wire or each cluster of wires appearing in the monitoring image is detected and segmented to obtain the second image region where each power transmission cable is located. Like the first image region, the second image region can also be represented by a target constraint frame. In addition, the power transmission cable instance segmentation model can also identify the mask Mp of each power transmission cable or each cluster of power transmission cables. The mask Mp represents the binary segmentation map of each or each cluster of power transmission cables, with a pixel value of 0 representing the background and 1 representing the target, and the size is consistent with the input image.

[0064] Step S102: If the first image area and the second image area are at the same relative depth in the monitoring image, the actual distance between the construction machinery and the transmission cable is calculated based on the first image area and the second image area; the relative depth represents the front-to-back relationship between the construction machinery in the first image area and the transmission cable in the second image area in the monitoring image.

[0065] In this embodiment, the relative depth represents the front-to-back relationship between the construction machinery in the first image area and the transmission cables in the second image area in the monitoring image. In physical space, the construction machinery and the transmission cables may be one in front of the other, or the planes where certain surfaces of the two are located may be basically in the same plane. Although the relative positions of the two in physical space are quite different, the two appear to be on the same two-dimensional plane in the monitoring image. In fact, the relative positions of the construction machinery and the transmission cables in physical space are different, and the relative depth reflected in the monitoring image will also be different.

[0066] In this embodiment, the first image area and the second image area are at the same relative depth in the monitoring image, which means that the two-dimensional plane where the construction machinery is located in the monitoring image and the two-dimensional plane where the transmission cable is located in the monitoring image are the same plane, or the distance between the two two-dimensional planes is relatively close. The risk within the risk distance is relatively high, that is, the risk of external damage to the transmission cable by the construction machinery is also relatively high.

[0067] Step S103: If the actual distance is less than a preset threshold, it is determined that the power transmission cable has a risk of external damage.

[0068] In this embodiment, generally speaking, for high risk requirements, the preset threshold can be set larger, otherwise, the preset threshold can be set smaller, for high accuracy requirements, the preset threshold can be set smaller, otherwise, the preset threshold can be set larger. The preset threshold can also be set according to actual needs to balance risk requirements and accuracy requirements.

[0069] In this embodiment, after determining that the power transmission cable has a hidden danger of external damage, an alarm can be issued by means of sound, light, electricity, etc.

[0070] The above method provided in this embodiment calculates the actual distance between the construction machinery and the transmission cable when the two are at the same relative depth. The actual distance calculated at this time is more reference for whether there is a hidden danger of external damage to the transmission cable, and ultimately a more accurate identification result of whether the transmission cable has a hidden danger of external damage can be obtained.

[0071] In an optional implementation manner, in order to efficiently and accurately determine whether the first image area and the second image area are at the same relative depth in the monitoring image, this embodiment provides an implementation method:

[0072] First, obtain the depth map of the surveillance image;

[0073] In this embodiment, as an implementation method, the depth map can be a relative depth map, which is used to represent the relative depth relationship between different objects or surfaces in the scene of the monitoring image, where the objects include construction machinery and power transmission cables. Each pixel in the relative depth map has a relative depth value.

[0074] As a method of obtaining a relative depth map, a relative depth estimation model can be used to estimate the relative depth value of each pixel in the monitoring image to obtain the relative depth map of the monitoring image. The relative depth estimation model includes, but is not limited to, Depth Anything V2, ZoeDepth, etc. Depth Anything V2 is an upgraded version of Depth Anything V1. It expands the capacity and scale of the teacher model on its basis, and uses a large-scale image-depth data set generated by a virtual simulation engine instead of real scene data to train the teacher model. Then, the teacher model is used to pre-label the real scene image, and a large-scale pseudo-label training data set is established to guide the training of the student model. Thanks to the use of simulation data and semi-supervised training methods, Depth Anything V2 takes into account both the overall depth accuracy and local details, has strong generalization capabilities, and can handle mirrors, complex layouts, and transparent scenes. In addition, Depth Anything V2 also provides four sizes of models from large to small for practical application.

[0075] As another implementation method, the depth map can also be obtained through binocular depth estimation. In this case, a binocular camera can be used to obtain the main view for monitoring construction machinery and transmission cables and to perform binocular depth estimation from the view to obtain a depth map. The monitoring image is the main view, and the depth map is a depth map of the main view.

[0076] One way to get a depth map through binocular depth estimation is:

[0077] (1) Calibrate and correct the binocular camera to obtain the calibration parameters.

[0078] In this embodiment, the calibration process of the binocular camera includes single-target calibration and binocular calibration. Single-target calibration is to determine the internal parameters (focal length, optical center, etc.) and distortion coefficient of each camera, which is usually completed by taking multiple pictures of a known pattern (such as a chessboard). Binary-target calibration is to determine the relative position and rotation relationship between the two cameras, which usually requires taking the same calibration pattern in the overlapping area of ​​the field of view of the two cameras and calculating the spatial relationship between them.

[0079] Stereo correction of binocular cameras is also called binocular correction. The purpose of stereo correction is to make the imaging planes of the two cameras coplanar and the optical axes of the cameras parallel, so that the corresponding points in the two images will be on the same horizontal line, thereby simplifying the subsequent parallax calculation. The correction process usually includes image dedistortion and image reprojection to ensure that in the corrected image, the corresponding pixel rows are horizontally aligned.

[0080] As an implementation method, offline calibration can be used for calibration. The specific calibration process is: (1.1) Perform single-target calibration on the binocular camera in the stereo vision system, and obtain the intrinsic parameter matrices KL, KR, extrinsic parameter matrices RL, RR, tL, tR, distortions DL, DR of the binocular camera (taking the left and right cameras as examples), respectively. According to the calibration parameters of each camera, the captured images (the images captured by the left and right cameras are the left view and the right view, respectively) are subjected to distortion correction; (1.2) Perform stereo calibration on the binocular camera system, obtain the optical center position and calculate the optical center distance d through the relative posture relationship between the left and right cameras, determine the intrinsic and extrinsic parameters of the virtual camera, and then determine the homography matrix of the left and right cameras relative to the virtual camera to achieve stereo correction of the left and right views.

[0081] (2) The main view and the sub-view taken by the binocular camera are calibrated and preprocessed using the calibration parameters to obtain the calibrated main view and the sub-view.

[0082] Taking the left and right cameras as an example, the calibration parameters are used to perform distortion correction and stereo correction (epipolar correction) on the original images L' and R' of the left and right views taken by the binocular stereo vision system, respectively, to obtain the corrected left and right views L and R, where the left view L is the main view, that is, the monitoring image in this embodiment, and the right view R is the slave view.

[0083] (3) Use the corrected main view and slave view to perform binocular depth estimation and obtain a depth map.

[0084] In this embodiment, as a binocular depth estimation method, the LightStereo binocular depth estimation method is used to obtain the depth map D corresponding to the main view. LightStereo is the most cutting-edge stereo matching algorithm currently available, which can realize end-to-end disparity map estimation. The algorithm designs a lightweight 2D codec cost aggregation network that focuses on the disparity dimension of 3D cost voxels. Through the carefully designed network structure modeling semantic features and matching cost distribution, LightStereo achieves depth estimation while ensuring high precision, high real-time performance and low resource usage. The binocular depth estimation model (LightStereo) can be trained by a pre-prepared high-quality annotated dataset.

[0085] Please refer to Figure 2 , Figure 2 The structural diagram of the binocular depth estimation model provided in this embodiment inputs the corrected left and right views L and R into the pre-trained binocular depth estimation model, and the processing process is as follows:

[0086] (1) Multi-scale feature extraction: First, MobileNetV2 is used as the backbone network to extract multi-scale features of the binocular disparity image (the multi-scales are 1 / 4, 1 / 8, and 1 / 16 of the original image) and construct a feature pyramid.

[0087] (2) Cost calculation: Calculate the corresponding cost voxel V based on the left and right view features. The result is a three-dimensional tensor, where each dimension corresponds to the disparity level s, height h, and width w. The specific calculation method is shown in the following formula.

[0088] V(s,h,w) = 1 / C ∑ FL(h,w)·FR(h,ws)

[0089] Among them, C represents the total number of channels of multi-scale features, FL and FR represent the multi-scale features of the left and right views respectively, and “·” represents the vector inner product.

[0090] (3) Cost aggregation: The 2D cost aggregation network takes 3D cost voxels as input and inserts a multi-scale attention map calculated by the MSCA (Multi-Scale Convolutional Attention) module into the encoder-decoder network. It outputs an estimated initial disparity map whose size is 1 / 4 of the original input. MSCA is an attention mechanism module used in image processing. It can adaptively perform weighted fusion on feature maps of different scales, thereby improving the network’s expressiveness and generalization capabilities.

[0091] (4) Disparity improvement: Using the view features as input, the context network calculates the disparity map upsampling weights and improves the disparity estimate S from the low-resolution initial disparity map to the original resolution disparity map, taking into account both efficiency and accuracy.

[0092] (5) Depth estimation: After obtaining the parallax estimate S of the main view, the main view depth map D = d·fL / S can be obtained pixel by pixel based on the epipolar geometry, where d is the optical center distance and fL is the focal length of the main view camera after correction.

[0093] Secondly, according to the relative depth values ​​of all pixels in the first image area in the depth map, a relative depth value of the first image area is obtained;

[0094] In this embodiment, the average value of the relative depth values ​​of all pixels in the first image area in the depth map can be used as the relative depth value of the first image area, or all pixels in the first image area can be arranged from large to small according to the relative depth value, and a preset proportion of pixels are selected and their average value is calculated, and the calculated average value is used as the relative depth value of the first image area. The relative depth value of the pixel with the largest relative depth value can also be selected from all pixels in the first image area as the relative depth value of the first image area.

[0095] Taking the average value of the relative depth values ​​of all pixels in the first image area in the depth map as an example, one implementation method for obtaining the relative depth value of the first image area is:

[0096] (1) The area in the depth map that overlaps with the first image area and has the same size as the first image area is taken as the intersection area;

[0097] (2) Calculate the average relative depth value of all pixels in the intersection area to obtain the relative depth value of the first image area.

[0098] In this embodiment, as a specific implementation method, the intersection Mm∩D of the mask Mm of the construction machinery corresponding to the first image area and the depth map D is calculated. Then, the average of the non-zero elements in the intersection Mm is calculated to obtain the relative depth of the first image area, that is, Dm=AVG (Mm∩D>0), and its value range is between [0,1].

[0099] Third, determining a target pixel whose relative depth value is within a preset range determined by the relative depth value of the first image area from the multiple pixel points in the depth map, and taking an area covered by the pixel points corresponding to the target pixel point in the monitoring image as a reference area;

[0100] In this embodiment, the preset range determined by the relative depth value can be set according to actual needs. For example, the preset range determined by the relative depth value Dm is [Dm-1%, Dm+1%].

[0101] In this embodiment, the reference area is an area in the monitoring image within a preset range determined by the relative depth value of the first image area. The reference area may be larger than the second image area or may be exactly the same as the second image area.

[0102] Finally, if the second image area is within the reference area or the second image area is the same as the reference area, it is determined that the first image area and the second image area are at the same relative depth in the monitoring image.

[0103] In this embodiment, when there are multiple second image areas, another specific implementation method may be: first, with [Dm-1%, Dm+1%] as the threshold interval, the depth map D is binarized to obtain a binary segmentation map Dmb, where the pixel value 1 indicates that the relative depth is within the interval [Dm-1%, Dm+1%], and 0 indicates that the relative depth is outside the interval. Then, relative depth matching is performed on each transmission cable, and the relative depth matching method may be: find the intersection area Dmb∩Mp of Dmb and the mask Mp of each transmission cable. If the intersection areas corresponding to the masks of all transmission cables are empty, it means that there is no second image area at the same relative depth as the construction machinery, otherwise it means that the construction machinery and the second image area corresponding to the mask Mp are at the same relative depth.

[0104] As another implementation method for determining whether they are at the same relative depth, the relative depth value of the first image area and the relative depth value of the second image area may be calculated respectively, and by comparing the relative depth values ​​of the two, if the difference between the relative depth values ​​of the two is within a preset difference, it is determined that the two are at the same relative depth.

[0105] In this embodiment, in order to reduce the amount of calculation when calculating the actual distance and minimize the influence of background noise, this embodiment provides an implementation method for calculating the actual distance:

[0106] First, the pixel distance between the first image region and the second image region is calculated;

[0107] In this embodiment, the pixel distance between the first image area and the second image area can be determined by the center coordinates of the two areas. However, due to the lack of their corresponding geometric shapes, background noise will be introduced if it is directly calculated. In order to minimize the influence of background noise, this embodiment determines the pixel distance between the two areas by pixel points on the contours of the first image area and the second image area. One implementation method is:

[0108] (1) acquiring a first contour of a first image region and a second contour of a second image region respectively, wherein the first contour includes a plurality of first pixel points and the second contour includes a plurality of second pixel points;

[0109] (2) calculating the distance between each first pixel point and each second pixel point to obtain a plurality of point pair distances between the first pixel points and the second pixel points;

[0110] (3) The minimum distance among all point pairs is taken as the pixel distance.

[0111] In this embodiment, the first contour and the second contour can be the contours of the constraint box of the first image area and the constraint box of the second image area, respectively, or can be the contour of the construction machinery in the first image area and the contour of the power transmission line in the second image area, respectively. Since the pixel distance between the two areas is determined by the pixel points on the contours of the first image area and the second image area, the calculation of all pixel pairs between all pixel points in the first image area and all pixel points in the second image area is avoided, thereby greatly reducing the amount of calculation.

[0112] Secondly, the pixel distance is spatially transformed to obtain the actual distance.

[0113] In this embodiment, the manner of performing spatial conversion on the pixel distance includes at least the following two manners:

[0114] Method 1:

[0115] Obtaining a spatial conversion relationship according to the pixel size of the first image region and the actual size of the construction machinery;

[0116] The pixel distance is converted into the actual distance according to the spatial conversion relationship.

[0117] In this embodiment, the pixel size of the first image area can be obtained by the size of the corresponding constraint box identified by the construction machinery instance segmentation model, and the actual size of the construction machinery can be measured in advance. Specifically, the corresponding actual size can be stored in advance for each type of construction machinery, and the construction machinery instance segmentation model identifies the type of the corresponding construction machinery, and the actual size of the corresponding construction machinery is directly obtained according to the identified type. If the actual size of the construction machinery is expressed as: Lm×Wm×Hm, where Lm, Wm, and Hm represent the length, width, and height of the mth type of construction machinery, respectively, and the pixel size of the first image area bm is expressed as: Wbm×Hbm, then the calculation formula of the spatial conversion relationship can be: R=GMEAN(Lm,Wm,Hm) / GMEAN(Wbm,Hbm), unit: meter / pixel, GMEAN represents the geometric mean. The formula for calculating the actual distance is: Dmin=dmin×R, where Dmin is the actual distance and dmin is the pixel distance.

[0118] For the purpose of specific description, this embodiment takes the construction machinery as an excavator as an example for description. Figure 3 , Figure 3 This is an example diagram of a monitoring image of an excavator construction site provided in this embodiment. Figure 3 In the figure, the area in the constraint box in the lower left corner is the constraint box area of ​​the excavator, the area in the upper right corner is the constraint box area of ​​the transmission cable, and the length of the line segment between the two is the pixel distance between the two. Figure 3 In the figure, the actual size of the excavator is: Lm=7m, Wm=3m, Hm=3m, where the width and height pixel sizes of the excavator constraint box are 195px and 258px respectively. The calculated spatial conversion relationship is R=(7×3×3)1 / 3 / (195×258)1 / 2 =0.01774 m / px. Figure 3 The pixel distance between the excavator and the transmission cable is dmin=810 px, which is converted into the actual distance in the real scene: Dmin=810px × 0.01774 m / px = 14.37m.

[0119] Method 1 can calculate the actual distance more accurately without relying on the camera's intrinsic parameter matrix, thereby ensuring the accuracy of external damage hidden danger identification. Since it realizes calibration-free ranging, it has lower costs and is easier to use widely.

[0120] Of course, under the premise of knowing the camera intrinsic parameter matrix, this embodiment can also calculate the actual distance based on the camera intrinsic parameter matrix, that is, method 2:

[0121] Obtain a first target pixel point and a second target pixel point corresponding to the pixel distance;

[0122] According to a preset camera intrinsic parameter matrix, coordinate conversion is performed on the first image coordinates of the first target pixel point and the second image coordinates of the second target pixel point respectively to obtain the first spatial coordinates of the first target pixel point in the world coordinate system and the second spatial coordinates of the second target pixel point in the world coordinate system;

[0123] The distance between the first space coordinate and the second space coordinate is calculated to obtain the actual distance.

[0124] In this embodiment, if the preset camera intrinsic parameter matrix is ​​represented by M, the first image coordinate and the second image coordinate are respectively represented by di(ui, vi)dj(uj, vj), then the first space coordinate Di=M -1 di, the second spatial coordinate Dj=M -1 ·dj, where M -1 It is the inverse matrix of matrix M, and the actual distance Dmin=||Di-Dj||2, in meters.

[0125] In order to execute the corresponding steps in the above embodiment and each possible implementation method, the following provides a method for implementing a device 100 for identifying hidden dangers of external damage to a power transmission cable. Figure 4 , Figure 4 This is a block diagram of the device for identifying hidden dangers of external damage of a transmission cable provided in this embodiment. It should be noted that the basic principles and technical effects of the device for identifying hidden dangers of external damage of a transmission cable 100 provided by the present invention are the same as those of the corresponding above-mentioned embodiments, and are not mentioned in this embodiment for the sake of brief description.

[0126] The transmission cable external damage hidden danger identification device 100 includes an acquisition module 110 , a calculation module 120 and an identification module 130 .

[0127] An acquisition module 110 is used to acquire a monitoring image for monitoring the construction machinery and the transmission cable, wherein the monitoring image includes a first image area where the construction machinery is located and a second image area where the transmission cable is located;

[0128] A calculation module 120, configured to calculate an actual distance between the construction machine and the power transmission cable according to the first image area and the second image area if the first image area and the second image area are at the same relative depth in the monitoring image; the relative depth represents a front-to-back relationship between the construction machine in the first image area and the power transmission cable in the second image area in the monitoring image;

[0129] The identification module 130 is used to determine that the power transmission cable has a hidden danger of external damage if the actual distance is less than a preset threshold.

[0130] In an optional implementation manner, the acquisition module 110 is further configured to:

[0131] Obtain a depth map of the surveillance image;

[0132] Obtaining a relative depth value of the first image area according to relative depth values ​​of all pixels in the first image area in the depth map;

[0133] Determine, from the plurality of pixels in the depth map, a target pixel whose relative depth value is within a preset range determined by the relative depth value of the first image area, and use an area covered by pixels corresponding to the target pixel in the monitoring image as a reference area;

[0134] If the second image area is within the reference area or the second image area is identical to the reference area, it is determined that the first image area and the second image area are at the same relative depth in the monitoring image.

[0135] In an optional implementation, the step of calculating the average value of the relative depth values ​​of all pixels in the first image area in the depth map to obtain the relative depth value of the first image area includes:

[0136] The area in the depth map that overlaps with the first image area and has the same size as the first image area is taken as the intersection area;

[0137] The average value of the relative depth values ​​of all pixels in the intersection area is calculated to obtain the relative depth value of the first image area.

[0138] In an optional implementation manner, the calculation module 120 is specifically configured to:

[0139] calculating a pixel distance between the first image region and the second image region;

[0140] Perform spatial transformation on the pixel distance to obtain the actual distance.

[0141] In an optional implementation manner, when the calculation module 120 is specifically used to calculate the pixel distance between the first image area and the second image area, it is used to:

[0142] Acquire a first contour of the first image area and a second contour of the second image area respectively, the first contour includes a plurality of first pixel points, and the second contour includes a plurality of second pixel points;

[0143] Calculating the distance between each first pixel point and each second pixel point to obtain a plurality of point pair distances between the first pixel points and the second pixel points;

[0144] The minimum distance among all point pair distances is taken as the pixel distance.

[0145] In an optional implementation manner, the calculation module 120 is specifically used to perform spatial conversion on the pixel distance, and when the actual distance is obtained, is used to:

[0146] Obtaining a spatial conversion relationship according to the pixel size of the first image region and the actual size of the construction machinery;

[0147] The pixel distance is converted into the actual distance according to the spatial conversion relationship.

[0148] In an optional implementation manner, the calculation module 120 is specifically used to perform spatial conversion on the pixel distance to obtain the actual distance, and is also used to:

[0149] Obtain a first target pixel point and a second target pixel point corresponding to the pixel distance;

[0150] According to a preset camera intrinsic parameter matrix, coordinate conversion is performed on the first image coordinates of the first target pixel point and the second image coordinates of the second target pixel point respectively to obtain the first spatial coordinates of the first target pixel point in the world coordinate system and the second spatial coordinates of the second target pixel point in the world coordinate system;

[0151] The distance between the first space coordinate and the second space coordinate is calculated to obtain the actual distance.

[0152] The embodiment of the present invention also provides a block diagram of an electronic device 10, which implements the method for identifying hidden dangers of power transmission cable damage in the above embodiment. Figure 5 , Figure 5 This is a block diagram of an electronic device 10 provided in this embodiment. The electronic device 10 includes a processor 11, a memory 12 and a bus 13. The processor 11 and the memory 12 are connected via the bus 13.

[0153] The processor 11 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the transmission cable external damage hidden danger identification method of the above embodiment may be completed by the hardware integrated logic circuit or software instructions in the processor 11. The above processor 11 may be a general-purpose processor, including a CPU (Central Processing Unit), NP (Network Processor), etc.; it may also be a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Logic Gate Array) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0154] The memory 12 is used to store a program for implementing a method for identifying hidden dangers of external damage to a power transmission cable. The program may be a software function module stored in the memory 12 in the form of software or firmware or solidified in the OS (Operating System) of the electronic device 10 .

[0155] After receiving the execution instruction, the processor 11 executes the program to implement the method for identifying hidden dangers of external damage to a power transmission cable according to the aforementioned embodiment.

[0156] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for identifying hidden dangers of external damage to a power transmission cable as described in any one of the aforementioned implementations is implemented.

[0157] In summary, an embodiment of the present invention provides a method, device, electronic device and storage medium for identifying hidden dangers of external damage to a transmission cable, the method comprising: obtaining a monitoring image for monitoring construction machinery and transmission cables, the monitoring image comprising a first image area where the construction machinery is located and a second image area where the transmission cable is located; if the first image area and the second image area are at the same relative depth in the monitoring image, then calculating the actual distance between the construction machinery and the transmission cable based on the first image area and the second image area; the relative depth characterizes the front-to-back relationship between the construction machinery in the first image area and the transmission cable in the second image area in the monitoring image; if the actual distance is less than a preset threshold, it is determined that the transmission cable has a hidden danger of external damage. Compared with the prior art, this embodiment has at least the following advantages: (1) When the construction machinery and the transmission cable are at the same relative depth, the actual distance between the two is calculated. At this time, the calculated actual distance is more reference for whether there is a hidden danger of external damage to the transmission cable, and finally a more accurate identification result of the hidden danger of external damage to the transmission cable can be obtained; (2) The pixel distance between the two areas is determined by the pixel points on the contour of the first image area and the second image area, thereby avoiding the calculation of all pixel pairs between all pixel points in the first image area and all pixel points in the second image area, greatly reducing the amount of calculation; at the same time, it also avoids the influence of background noise on the accuracy of the distance when the distance between the two is determined by the center coordinate; (3) Without relying on the camera intrinsic parameter matrix, the actual distance can be calculated more accurately, thereby ensuring the accuracy of the identification of hidden dangers of external damage. Since the calibration-free distance measurement is realized, the cost is lower and it is easier to be widely used; (4) The semi-supervised monocular relative depth estimation large model is used to make the depth estimation more accurate and robust; (5) The relative depth is only used to match the nearest neighbor transmission cable of the construction machinery, and is not directly used for distance calculation, so it will not introduce depth estimation errors, thereby ensuring the accuracy of the results.

[0158] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for identifying hidden dangers of external damage to power transmission cables, characterized in that: The method comprises: Acquire a monitoring image for monitoring a construction machine and a power transmission cable, wherein the monitoring image includes a first image area where the construction machine is located and a second image area where the power transmission cable is located; If the first image area and the second image area are at the same relative depth in the monitoring image, the actual distance between the construction machine and the transmission cable is calculated based on the first image area and the second image area; the relative depth represents the front-to-back relationship between the construction machine in the first image area and the transmission cable in the second image area in the monitoring image; If the actual distance is less than a preset threshold, it is determined that the power transmission cable has a hidden danger of external damage; After the step of acquiring monitoring images for monitoring the construction machinery and the power transmission cables, the method further includes: Acquire a depth map of the monitoring image, wherein the depth map is a relative depth map for representing a relative depth relationship between the construction machinery in the first image area and the power transmission cables in the second image area in the monitoring image; Obtaining a relative depth value of the first image area according to relative depth values ​​of all pixels in the first image area in the depth map; Determine, from a plurality of pixels of the depth map, a target pixel whose relative depth value is within a preset range determined by the relative depth value of the first image area, and use an area covered by pixels corresponding to the target pixel in the monitoring image as a reference area, wherein the preset range determined by the relative depth value is [Dm-1%, Dm+1%], where Dm is the relative depth value; If the second image area is within the reference area or the second image area is the same as the reference area, it is determined that the first image area and the second image area are at the same relative depth in the monitoring image.

2. The method for identifying hidden dangers of external damage of power transmission cables according to claim 1, characterized in that: The step of calculating an average value of relative depth values ​​of all pixels in the first image area in the depth map to obtain the relative depth value of the first image area comprises: Taking an area in the depth map that overlaps with the first image area and has the same size as the first image area as an intersection area; An average value of the relative depth values ​​of all pixels in the intersection area is calculated to obtain a relative depth value of the first image area.

3. The method for identifying hidden dangers of external damage of power transmission cables according to claim 1, characterized in that: The step of calculating the actual distance between the construction machine and the power transmission cable according to the first image area and the second image area comprises: calculating a pixel distance between the first image area and the second image area; Performing spatial transformation on the pixel distance to obtain the actual distance.

4. The method for identifying hidden dangers of external damage of power transmission cables according to claim 3, characterized in that: The step of calculating the pixel distance between the first image area and the second image area comprises: Acquire a first contour of the first image area and a second contour of the second image area respectively, wherein the first contour includes a plurality of first pixel points, and the second contour includes a plurality of second pixel points; Calculating the distance between each of the first pixel points and each of the second pixel points to obtain a plurality of point pair distances between the first pixel points and the second pixel points; The minimum distance among all point pair distances is taken as the pixel distance.

5. The method for identifying hidden dangers of external damage of power transmission cables according to claim 3, characterized in that: The step of performing spatial conversion on the pixel distance to obtain the actual distance comprises: Obtaining a spatial conversion relationship according to a pixel size of the first image area and an actual size of the construction machinery; The pixel distance is converted into the actual distance according to the space conversion relationship.

6. The method for identifying hidden dangers of external damage of power transmission cables according to claim 3, characterized in that: The step of performing spatial conversion on the pixel distance to obtain the actual distance comprises: Acquire a first target pixel point and a second target pixel point corresponding to the pixel distance; According to a preset camera intrinsic parameter matrix, coordinate conversion is performed on the first image coordinates of the first target pixel point and the second image coordinates of the second target pixel point to obtain the first spatial coordinates of the first target pixel point in a world coordinate system and the second spatial coordinates of the second target pixel point in the world coordinate system; The distance between the first space coordinate and the second space coordinate is calculated to obtain the actual distance.

7. A device for identifying hidden dangers of external damage to power transmission cables, characterized in that: The device comprises: An acquisition module, used to acquire a monitoring image for monitoring a construction machine and a power transmission cable, wherein the monitoring image includes a first image area where the construction machine is located and a second image area where the power transmission cable is located; a calculation module, configured to calculate the actual distance between the construction machine and the power transmission cable according to the first image area and the second image area if the first image area and the second image area are at the same relative depth in the monitoring image; the relative depth represents the front-to-back relationship between the construction machine in the first image area and the power transmission cable in the second image area in the monitoring image; an identification module, configured to determine that the power transmission cable has a risk of external damage if the actual distance is less than a preset threshold; The acquisition module is also used to: acquire a depth map of the monitoring image, wherein the depth map is a relative depth map used to represent the relative depth relationship between the construction machinery in the first image area and the power transmission cables in the second image area in the monitoring image; obtain the relative depth value of the first image area according to the relative depth values ​​of all pixels in the first image area in the depth map; determine a target pixel point whose relative depth value is within a preset range determined by the relative depth value of the first image area from multiple pixels in the depth map, and use the area covered by the pixels corresponding to the target pixel point in the monitoring image as a reference area, the preset range determined by the relative depth value is [Dm-1%, Dm+1%], where Dm is the relative depth value; if the second image area is within the reference area or the second image area is the same as the reference area, then determine that the first image area and the second image area are at the same relative depth in the monitoring image.

8. An electronic device, characterized in that: It comprises a processor and a memory, the memory is used to store a program, and the processor is used to implement the method for identifying hidden dangers of external damage of a transmission cable as described in any one of claims 1-6 when executing the program.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for identifying hidden dangers of external damage of a power transmission cable as described in any one of claims 1-6 is implemented.

Images