External damage hidden danger judgment method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an external damage hidden danger judgment method and device, electronic equipment and a storage medium, and relates to the technical field of power cables. The method comprises the following steps: obtaining a monitoring image, and performing instance segmentation on the monitoring image to obtain construction machinery instance information and power transmission cable instance information; based on the construction machine instance information and the power transmission cable instance information, establishing a spatial distribution relationship between the construction machines and the power transmission cables, the spatial distribution relationship reflecting connection characteristics of each construction machine and the corresponding power transmission cable closest to the construction machine; and according to the spatial distribution relationship, determining the target construction machinery and the target power transmission cable with the external damage hidden danger, thereby realizing intelligent judgment on whether the construction machinery may collide with the power transmission cable or not on the premise of not depending on three-dimensional sensing equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power cables, in particular to an external damage hidden danger discrimination method and device, electronic equipment and storage medium. BACKGROUND

[0002] In a power transmission system, high-voltage transmission cables are very important infrastructure. If construction machinery (such as cranes, pile drivers, etc.) approaches or collides with these transmission cables, an "external damage" accident will occur, resulting in power outages, fires, and even serious consequences such as personal injury.

[0003] In order to automatically identify whether construction machinery poses a threat to transmission cables and provide timely warnings, the current approach is to use devices with depth sensors (such as laser radars, structured light cameras) to perceive the position of objects in three-dimensional space, thereby determining the distance. However, depth sensors are expensive and not suitable for upgrading old equipment. These devices also require a complex calibration process after installation, making maintenance difficult. In addition, three-dimensional point cloud data processing is complex and requires high computing resources, making real-time response difficult. Furthermore, many old devices only have ordinary monocular cameras and do not have depth perception capabilities, making them incompatible with depth sensors.

[0004] The problem to be solved by the present application is how to intelligently determine whether construction machinery may collide with transmission cables using only two-dimensional images from ordinary cameras without relying on three-dimensional perception devices. SUMMARY

[0005] Therefore, the purpose of the present application is to provide an external damage hidden danger discrimination method and device, electronic equipment and storage medium.

[0006] To achieve the above purpose, the technical solutions adopted by the embodiments of the present application are as follows: In a first aspect, the present application provides an external damage hidden danger discrimination method, which comprises: obtaining a monitoring image, performing instance segmentation on the monitoring image to obtain construction machinery instance information and transmission cable instance information; establishing a spatial distribution relationship between the construction machinery and the transmission cables based on the construction machinery instance information and the transmission cable instance information, the spatial distribution relationship reflecting the connection characteristics of each construction machinery and the corresponding closest transmission cable; determining target construction machinery and target transmission cables with external damage hidden dangers according to the spatial distribution relationship.

[0007] Optionally, the construction machinery instance information includes a mask for each construction machinery, and the transmission cable instance information includes a mask for each transmission cable. The step of establishing the spatial distribution relationship between the construction machinery and the power transmission cable based on the construction machinery instance information and the power transmission cable instance information comprises: extracting a contour point of each of the construction machinery from a mask of each of the construction machinery; extracting a contour point of each of the power transmission cable from a mask of each of the power transmission cable; performing matching analysis on the contour point of each of the construction machinery and the contour point of each of the power transmission cable to obtain first distribution information and second distribution information, wherein the first distribution information reflects a corresponding relationship between each of the construction machinery and the closest power transmission cable, and the second distribution information reflects a nearest neighbor line segment between each of the construction machinery and the corresponding closest power transmission cable, and the spatial distribution relationship comprises the first distribution information and the second distribution information.

[0008] Optionally, the step of performing matching analysis on the contour point of each of the construction machinery and the contour point of each of the power transmission cable to obtain the first distribution information and the second distribution information comprises: fitting the contour point of each of the power transmission cable to obtain a corresponding curve of each of the power transmission cable, and sampling a preset number of curve points on the corresponding curve of each of the power transmission cable; calculating a geometric center of the contour point of each of the construction machinery; determining the corresponding closest power transmission cable of each of the construction machinery according to the geometric center of the contour point of each of the construction machinery and the curve point of each of the power transmission cable to obtain the first distribution information; constructing the nearest neighbor line segment between each of the construction machinery and the corresponding closest power transmission cable according to the contour point of each of the construction machinery and the curve point of the corresponding closest power transmission cable to obtain the second distribution information.

[0009] Optionally, the spatial distribution relationship comprises the first distribution information and the second distribution information, the first distribution information reflects a corresponding relationship between each of the construction machinery and the closest power transmission cable, and the second distribution information reflects a nearest neighbor line segment between each of the construction machinery and the corresponding closest power transmission cable, the construction machinery instance information comprises a bounding box of each of the construction machinery, and the power transmission cable instance information comprises a bounding box of each of the power transmission cable. The step of determining the target construction machinery and the target power transmission cable with the external damage hidden danger according to the spatial distribution relationship comprises: for each of the construction machinery, judging whether the construction machinery and the corresponding closest power transmission cable are at the same depth level according to the bounding box of the construction machinery and the bounding box of the corresponding closest power transmission cable; determining, according to the bounding box of the construction machinery and the bounding box of the corresponding distance-closest power cable, whether the construction machinery is at the lower adjacent position of the corresponding distance-closest power cable; if the construction machinery is at the lower adjacent position of the corresponding distance-closest power cable, determining the construction machinery as the target construction machinery and the distance-closest power cable corresponding to the construction machinery as the target power cable.

[0010] Optionally, the step of determining, according to the bounding box of the construction machinery and the bounding box of the corresponding distance-closest power cable, whether the construction machinery and the corresponding distance-closest power cable are at the same depth level comprises: determining, according to the aspect ratio of the bounding box of the distance-closest power cable of the construction machinery, whether the distance-closest power cable of the construction machinery is visually vertical; if the distance-closest power cable of the construction machinery is visually vertical, determining that the construction machinery and the corresponding distance-closest power cable are at the same depth level; if the distance-closest power cable of the construction machinery is not visually vertical, obtaining a depth ratio according to the diagonal pixel ratio of the bounding box of the construction machinery and the diagonal pixel ratio of the bounding box of the corresponding distance-closest power cable; determining, according to the depth ratio, whether the construction machinery and the corresponding distance-closest power cable are at the same depth level.

[0011] Optionally, the step of determining, according to the bounding box of the construction machinery and the bounding box of the corresponding distance-closest power cable, whether the construction machinery is at the lower adjacent position of the corresponding distance-closest power cable comprises: if there is an overlapping area in the horizontal projection of the bounding box of the construction machinery and the bounding box of the corresponding distance-closest power cable on the image plane, determining that the construction machinery is not at the lower adjacent position of the corresponding distance-closest power cable; if there is no overlapping area in the horizontal projection of the bounding box of the construction machinery and the bounding box of the corresponding distance-closest power cable on the image plane, determining, according to the coordinates of the two end points of the nearest neighbor line segment between the construction machinery and the corresponding distance-closest power cable, whether the construction machinery is at the lower adjacent position of the corresponding distance-closest power cable.

[0012] Optionally, the construction machinery instance information comprises a mask and a bounding box of each construction machinery, and the power cable instance information comprises a mask and a bounding box of each power cable, and the method further comprises: According to intersection degrees of masks of the target construction machinery and target power transmission cable, overlapping degrees of bounding boxes of the target construction machinery and target power transmission cable, and the nearest distance between the target construction machinery and the target power transmission cable, a grade of an external damage hidden danger of the target construction machinery and the target power transmission cable is determined.

[0013] In a second aspect, the present application provides an external damage hidden danger discrimination device, the device comprising: An instance segmentation module is configured to acquire a monitoring image, perform instance segmentation on the monitoring image, and obtain construction machinery instance information and power transmission cable instance information. A distribution relationship establishing module is configured to establish a spatial distribution relationship between the construction machinery and the power transmission cable based on the construction machinery instance information and the power transmission cable instance information, the spatial distribution relationship reflecting a connection feature of each construction machinery and a corresponding closest power transmission cable. A hidden danger determining module is configured to determine a target construction machinery and a target power transmission cable with an external damage hidden danger according to the spatial distribution relationship.

[0014] In a third aspect, the present application provides an electronic device comprising a processor and a memory, the memory storing machine executable instructions executable by the processor, and the processor being capable of executing the machine executable instructions to implement the external damage hidden danger discrimination method of any one of the preceding embodiments.

[0015] In a fourth aspect, the present application provides a computer readable storage medium storing a computer program, the computer program being executable by a processor to implement the external damage hidden danger discrimination method of any one of the preceding embodiments.

[0016] The external damage hidden danger discrimination method, device, electronic device and storage medium provided by the embodiments of the present application acquire a monitoring image, perform instance segmentation on the monitoring image, and obtain construction machinery instance information and power transmission cable instance information; establish a spatial distribution relationship between the construction machinery and the power transmission cable based on the construction machinery instance information and the power transmission cable instance information, wherein the spatial distribution relationship reflects a connection feature of each construction machinery and a corresponding closest power transmission cable; and determine a target construction machinery and a target power transmission cable with an external damage hidden danger according to the spatial distribution relationship. Since the embodiments of the present application acquire a two-dimensional monitoring image and perform instance segmentation to obtain construction machinery and power transmission cable instance information, and establish a spatial distribution relationship between the construction machinery and the power transmission cable based on the information to determine a construction machinery and a power transmission cable with an external damage hidden danger, the embodiments of the present application can realize intelligent discrimination of whether a construction machinery is likely to collide with a power transmission cable without relying on a three-dimensional perception device.

[0017] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are referred to for a detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be considered as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0019] Figure 1 A structural schematic block diagram of an electronic device provided by an embodiment of the present application is shown; Figure 2 A flowchart of an external damage hidden danger discrimination method provided by an embodiment of the present application is shown Figure 1 ; Figure 3 A flowchart of an external damage hidden danger discrimination method provided by an embodiment of the present application is shown Figure 2 ; Figure 4 A before-and-after comparison example diagram of a power transmission cable provided by an embodiment of the present application is shown; Figure 5 A visual example diagram of a construction machine and a power transmission cable discriminated as a high-threat external damage hidden danger provided by an embodiment of the present application is shown; Figure 6 A visual example diagram of a construction machine and a power transmission cable discriminated as a medium-threat external damage hidden danger provided by an embodiment of the present application is shown; Figure 7 A visual example diagram of a construction machine and a power transmission cable discriminated as a low-threat external damage hidden danger provided by an embodiment of the present application is shown; Figure 8 A visual example diagram of a construction machine and a power transmission cable discriminated as a non-threat external damage hidden danger provided by an embodiment of the present application is shown; Figure 9 A hidden danger alarm proportion comparison diagram provided by an embodiment of the present application is shown; Figure 10 A functional module block diagram of an external damage hidden danger discrimination device provided by an embodiment of the present application is shown.

[0020] Figure legend: 100-electronic device; 110-memory; 120-processor; 130-communication module; 200-external damage hidden danger discrimination device; 201-instance segmentation module; 202-distribution relationship establishment module; 203-hidden danger determination module. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be apparently and completely described with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0023] It should be noted that the relational terms such as first and second and the like are used merely to distinguish one entity or action from another, without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0024] Please refer to Figure 1 is a block schematic diagram of an electronic device 100. The electronic device 100 includes a memory 110, a processor 120 and a communication module 130. The memory 110, the processor 120 and the communication module 130 are electrically connected with each other directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected with each other through one or more communication buses or signal lines.

[0025] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0026] The processor 120 is used to read / write data or programs stored in the memory 110 and to perform corresponding functions.

[0027] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0028] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0029] Please refer to Figure 2 The present invention provides a method for identifying potential external damage, which includes steps S101 to S103.

[0030] S101, acquire monitoring images, perform instance segmentation on the monitoring images, and obtain instance information of construction machinery and power transmission cables.

[0031] The monitoring images refer to image data in the monitored scene, which can be a single frame or consecutive frames in a video stream. This image data comes from surveillance camera equipment conventionally deployed around power transmission lines, such as older monocular cameras, and does not rely on depth sensors or other high-cost hardware.

[0032] After acquiring the image data, instance segmentation is performed to identify and segment construction machinery instances and power transmission cable instances in the image. A trained instance segmentation model is used to perform target detection and pixel-level segmentation on the construction machinery and power transmission cables in the image, thereby obtaining detailed information for each construction machinery instance and power transmission cable instance.

[0033] The construction machinery instance information at least includes the category of each construction machinery (such as a crane, a tower crane, a pile driver, etc.), a binary mask image, and a bounding box of the target.

[0034] The power cable instance information also includes the power cable category of each power cable, a binary mask image, and a bounding box. The mask is used to represent the specific pixel distribution of the instance in the image, the background pixel value is 0, the target pixel value is 1, and the size is consistent with the original input image.

[0035] In the embodiment of the application, the instance segmentation model used can be a lightweight model suitable for running on an end-side device, such as YOLOv8-Seg, ContourFormer, etc., which balances the inference speed and resource occupancy rate while ensuring the recognition accuracy, and is suitable for deployment in an old monitoring device environment.

[0036] In S102, a spatial distribution relationship between the construction machinery and the power cable is established based on the construction machinery instance information and the power cable instance information.

[0037] The spatial distribution relationship reflects the connection characteristics of each construction machinery and the corresponding power cable with the shortest distance.

[0038] In a possible implementation, step S102 can include sub-steps S102-1 to S102-2.

[0039] In S102-1, the contour points of each construction machinery are extracted from the mask of each construction machinery, and the contour points of each power cable are extracted from the mask of each power cable.

[0040] After the instance segmentation of the construction machinery and the power cable in the monitoring image is completed, the binary mask image of each construction machinery and each power cable is obtained. Next, the contour points of the construction machinery and the power cable are extracted based on the mask image for subsequent spatial position analysis and hazard discrimination.

[0041] The external contour of the mask image of each construction machinery is extracted using an image processing algorithm (such as a contour detection algorithm). The findContours function in OpenCV or other equivalent algorithms can be used to extract the external maximum contour of the target object in the mask image, and a two-dimensional ordered point set in the image coordinate system is used to represent it, denoted as the construction machinery contour point set.

[0042] Similarly, the same contour extraction operation is performed on the mask image of each power cable to obtain the external contour point set of the power cable. The contour point set accurately reflects the spatial distribution form of the power cable in the image.

[0043] In some possible implementation manners, in order to reduce the computational complexity and improve the processing efficiency, the extracted contour points can be subjected to down-sampling processing. For example, a uniform sampling strategy is adopted, and one contour point is reserved every N points (for example, one point is reserved every 5 points), so that the computational overhead of subsequent steps is reduced on the premise of ensuring the contour expression accuracy.

[0044] The contour point extraction process does not need to rely on depth information and can be implemented based on two-dimensional image information, is suitable for a monocular camera monitoring system, is particularly suitable for old equipment and a low-resource computing environment, and has good real-time performance and applicability.

[0045] S102-2, the contour points of each construction machinery and the contour points of each power cable are subjected to matching analysis, and first distribution information and second distribution information are obtained.

[0046] The first distribution information reflects the corresponding relationship between each construction machinery and the closest power cable, the second distribution information reflects the nearest neighbor line segment between each construction machinery and the corresponding closest power cable, and the spatial distribution relationship includes the first distribution information and the second distribution information.

[0047] Further, the implementation process of step S103-2 can be as follows: S103-2-1, the contour points of each power cable are subjected to fitting, the corresponding curve of each power cable is obtained, and a preset number of curve points on the corresponding curve of each power cable are sampled.

[0048] S103-2-2, the geometric center of the contour points of each construction machinery is calculated.

[0049] S103-2-3, the corresponding closest power cable of each construction machinery is determined according to the geometric center of the contour points of each construction machinery and the curve points of each power cable, and the first distribution information is obtained.

[0050] S103-2-4, the nearest neighbor line segment between each construction machinery and the corresponding closest power cable is constructed according to the contour points of each construction machinery and the curve points of the corresponding closest power cable, and the second distribution information is obtained.

[0051] After the contour points of the construction machinery and the power cable are extracted, in order to further analyze the spatial relationship between the two, first, the contour points of each power cable are subjected to curve fitting, so as to obtain a more robust and convenient geometric representation for subsequent distance calculation.

[0052] The set of contour points of each power cable can be fitted by using a polynomial curve fitting method, such as RANSAC robust fitting or least squares fitting, to obtain a mathematical expression of the power cable instance. The curve obtained after fitting is used to approximate the direction of the power cable in the image space. In order to reduce the subsequent calculation amount while ensuring the accuracy of the shape expression of the power cable, the fitted curve is uniformly sampled within the power cable bounding box, and the number of sampling points is preferably 10 (empirical value), to obtain a set of curve points of the power cable instance as the basis for subsequent distance calculation.

[0053] At the same time, the geometric center of the set of contour points of each construction machinery is calculated, that is, the average values of the x coordinates and y coordinates of all contour points are taken respectively to obtain the center position of the construction machinery in the image coordinate system.

[0054] Based on the above information, the distance between the geometric center of each construction machinery and the set of curve points of each power cable is calculated in turn. The point-to-line segment distance calculation formula is used to calculate the distance between the geometric center and each line segment of each power cable, and the minimum value is taken as the closest distance between the construction machinery and the power cable. By traversing all power cables, the power cable closest to the construction machinery is determined, forming a machinery-power cable matching pair, which is recorded as the first distribution information.

[0055] Further, to more accurately assess the potential contact risk between the construction machinery and the power cable, the minimum distance between the original contour points of the construction machinery and the set of curve points of the corresponding closest power cable is also calculated. Each contour point of the construction machinery is traversed to calculate the point-to-line segment distance between it and the curve points of the power cable, and the minimum set of point pairs is selected, each from the machinery contour and the power cable curve, to form the closest neighbor line segment in the matching pair, which is recorded as the second distribution information.

[0056] It can be understood that the first distribution information includes the matching relationship between the construction machinery and the closest power cable; and the second distribution information includes the endpoint coordinates of the closest neighbor line segment between the construction machinery contour and the power cable.

[0057] Compared with point-by-point matching and non-fitting methods, the method provided by the embodiment of the application has significantly improved calculation speed and matching speed, especially when there are many instances to be matched and the contour is complex. The experimental results are shown in the following table.

[0058]

[0059] S103, according to the spatial distribution relationship, determining the target construction machinery and the target power cable that exist external damage hidden dangers.

[0060] In a possible implementation, step S103 can include sub-steps S103-1-S103-3.

[0061] S103-1, for each construction machinery, according to the bounding box of the construction machinery and the bounding box of the corresponding nearest power cable, determine whether the construction machinery and the corresponding nearest power cable are in the same depth level.

[0062] Wherein, the implementation process of "determining whether the construction machinery and the corresponding nearest power cable are in the same depth level according to the bounding box of the construction machinery and the bounding box of the corresponding nearest power cable" can be: First, according to the aspect ratio of the bounding box of the nearest power cable of the construction machinery, determine whether the nearest power cable of the construction machinery is visually vertical.

[0063] At this time, there are two situations, one is "if the nearest power cable of the construction machinery is visually vertical", at this time, it is determined that the construction machinery and the corresponding nearest power cable are in the same depth level.

[0064] The other is "if the nearest power cable of the construction machinery is not visually vertical", at this time, according to the diagonal pixel ratio of the bounding box of the construction machinery and the diagonal pixel ratio of the bounding box of the corresponding nearest power cable, obtain the depth ratio; according to the depth ratio, determine whether the construction machinery and the corresponding nearest power cable are in the same depth level.

[0065] After the matching relationship between the construction machinery and the power cable is determined, in order to further determine whether there is a spatial proximity relationship between the construction machinery and its nearest neighbor power cable, that is, whether they are in the same depth level, the embodiment of the application estimates the depth level based on two-dimensional image information, without relying on depth sensors or complex calibration processes, and is suitable for monocular vision monitoring system.

[0066] First, according to the aspect ratio of the bounding box of the nearest neighbor power cable corresponding to the construction machinery, determine whether the power cable is in a visually vertical state in the image.

[0067] Wherein, the aspect ratio is the ratio of the width and height of the bounding box. If the aspect ratio is less than a preset threshold (for example, less than 2), it is considered that the power cable is in a vertical or approximately vertical state in the image, indicating that it may span multiple depth levels in three-dimensional space, and therefore the construction machinery and the power cable are in the same depth level.

[0068] If the power cable does not present a visually vertical state (i.e., the aspect ratio is greater than or equal to the preset threshold), further estimate the relative depth relationship between the two based on image pixel information.

[0069] The diagonal pixel ratios of the construction machinery bounding box and the corresponding power cable bounding box are respectively denoted as SR_m and SR_p, SR_m = L2Norm(L_m, W_m, H_m) / L2Norm(Bh_m, Bw_m), and SR_p = L2Norm(L_p, W_p, H_p) / L2Norm(Bh_p, Bw_p).

[0070] L2Norm represents an L2 norm, L_m, W_m, H_m, and L_p, W_p, H_p are prior standard dimensions (length, width, and height) of the construction machinery and the power cable, and Bh_m, Bw_m, and Bh_p, Bw_p are heights and widths (in pixels) of the bounding boxes of the construction machinery and the power cable.

[0071] Based on the diagonal pixel ratios, a depth log ratio DLR between the construction machinery and the power cable is calculated. If the DLR is less than a preset threshold (for example, 0.65), it is determined that the construction machinery and the corresponding power cable are at the same depth level, otherwise, it is considered that the two are at different depth levels.

[0072] Since the embodiment of the present application is based on prior size information and image bounding box information when judging the depth level, the dependence on the depth sensor is avoided, and it is suitable for old equipment deployment environment, and has good robustness and applicability.

[0073] S103-2, in the case of determining that the construction machinery and the corresponding closest power cable are at the same depth level, whether the construction machinery is at the lower adjacent position of the corresponding closest power cable is determined according to the bounding box of the construction machinery and the bounding box of the corresponding closest power cable.

[0074] The implementation process of "determining whether the construction machinery is at the lower adjacent position of the corresponding closest power cable according to the bounding box of the construction machinery and the bounding box of the corresponding closest power cable" is as follows: First, it is determined whether there is an overlapping area in the horizontal projection of the bounding box of the construction machinery and the bounding box of the corresponding closest power cable on the image plane.

[0075] In the case that there is an overlapping area in the horizontal projection of the bounding box of the construction machinery and the bounding box of the corresponding closest power cable on the image plane, it is determined that the construction machinery is not at the lower adjacent position of the corresponding closest power cable.

[0076] In the case that the surrounding frame of the construction machinery and the horizontal projection of the surrounding frame of the corresponding nearest power cable on the image plane do not have an overlapping area, it is determined whether the construction machinery is in the lower adjacent position of the corresponding nearest power cable according to the coordinates of the two end points of the nearest line segment between the construction machinery and the corresponding nearest power cable.

[0077] In the process of determining whether the construction machinery and the corresponding nearest power cable have a potential collision risk, in addition to determining whether they are in the same depth level, it is further determined whether the construction machinery is in the lower adjacent position of the power cable. The determination is based on the spatial geometric relationship in the image plane and does not rely on depth information, and is suitable for monocular vision monitoring systems.

[0078] Firstly, it is determined whether the surrounding frame of the construction machinery and the surrounding frame of the corresponding power cable have an overlapping area on the horizontal projection on the image plane.

[0079] The implementation is to obtain the projection range [Bm_xmin, Bm_xmax] of the surrounding frame of the construction machinery on the x-axis, and the projection range [Bp_xmin, Bp_xmax] of the surrounding frame of the power cable on the x-axis. If the two projection ranges have an intersection, it is determined that they overlap in the horizontal direction, that is, the construction machinery is directly below or below the power cable, and does not have the possibility of approaching the power cable from above, so it is determined that the construction machinery is not in the lower adjacent position of the power cable.

[0080] In the case that the surrounding frame of the construction machinery and the surrounding frame of the power cable do not have an overlapping area on the horizontal projection on the image plane, it is further determined whether the construction machinery is in the lower adjacent position of the power cable according to the nearest line segment between them.

[0081] The implementation is to extract the coordinates of the two end points of the nearest line segment, one end point belongs to the construction machinery contour point Lm, and the other end point belongs to the power cable curve point Lp. The coordinates Lm_y and Lp_y of the two end points in the y-axis direction of the image are obtained respectively.

[0082] If Lm_y < Lp_y, that is, the construction machinery contour point is lower than the power cable curve point in the vertical direction of the image, it is determined that the construction machinery is in the lower adjacent position of the power cable; otherwise, it is determined that it is not in the lower adjacent position.

[0083] It can be understood that by combining the projection of the surrounding frame and the position of the end point of the nearest line segment based on the geometric relationship in the image plane, the spatial relative position between the construction machinery and the power cable is effectively identified, which provides a key basis for subsequent hazard level discrimination, and at the same time avoids the dependence on depth sensors, and is suitable for old equipment and resource-limited environments.

[0084] S103-3, if the construction machinery is in the corresponding distance closest power cable below adjacent place, the construction machinery is determined as the target construction machinery, and the construction machinery corresponding distance closest power cable is taken as the target power cable.

[0085] Understandably, if any construction machinery and its corresponding distance closest power cable are in the same depth level and the construction machinery is also in the corresponding distance closest power cable below adjacent place, it means that the construction machinery and its corresponding distance closest power cable have contact risk, and there is an external damage hidden danger.

[0086] Please refer to Figure 3 The external damage hidden danger discrimination method further includes step S104.

[0087] S104, according to the intersection degree of the mask of the target construction machinery and the mask of the target power cable, the overlap degree of the bounding box of the target construction machinery and the bounding box of the target power cable, and the nearest distance between the target construction machinery and the target power cable, the external damage hidden danger grade of the target construction machinery and the target power cable is determined.

[0088] After the spatial relationship analysis between the construction machinery and the power cable is completed, the next step is to combine the image spatial feature indexes to grade whether there is an external damage hidden danger between the construction machinery and the power cable.

[0089] According to the mask intersection degree, the bounding box overlap degree and the nearest distance, the external damage hidden danger grade between the target construction machinery and the target power cable is comprehensively judged in the embodiment of the application: The mask intersection degree is the pixel intersection area of the target construction machinery mask M_m and the target power cable mask M_p, which is used to judge whether there is visual overlap between the two in the image, and indicates whether contact has occurred or is at the edge of contact at the current moment.

[0090] The bounding box overlap degree is the intersection and union ratio of the target construction machinery bounding box Bm and the target power cable bounding box Bp on the image plane, which is used to measure the spatial proximity of the two in the image.

[0091] The nearest distance is the minimum distance between the target construction machinery contour point and the target power cable curve point, which is used to quantify the proximity between the two, and is an important basis for judging potential threats. That is, the nearest distance is the length of the nearest neighbor segment between each construction machinery and the corresponding distance closest power cable.

[0092] Based on the above indexes, the embodiment of the application adopts a set of fuzzy rule discrimination system, and by setting multiple discrimination conditions and threshold values, the hidden danger grade is divided into four grades: high threat, medium threat, low threat and no threat. The discrimination logic is as follows: 1. High threat grade discrimination If the target construction machinery and the target power transmission cable meet any of the following conditions, they are judged to be of high threat level: (1) The mask intersection area is greater than zero, indicating that the construction machinery and the power transmission cable have pixel-level overlap in the image and may have made contact; (2) The nearest neighbor distance is less than the set threshold (for example, the empirical value is 8 pixels), and the construction machinery and the power transmission cable are at the same depth level.

[0093] A high threat level indicates that there is a risk of direct contact between the target construction machinery and the target power transmission cable, requiring priority for manual verification or triggering an emergency warning.

[0094] 2. Medium Threat Level Assessment If the target construction machinery and the target power transmission cable do not meet the high threat criteria, but meet any of the following conditions, they are classified as medium threat level: (1) the encirclement of the construction machinery is appropriately expanded (e.g., Figure 4 As shown, after the expansion ratio is 1.8 times, the intersection area between it and the power transmission cable mask is greater than zero; (2) the nearest neighbor distance is less than the slight wagging threshold based on the length of the power transmission cable (e.g., the pixel distance corresponding to 5% of the length of the power transmission cable).

[0095] A medium threat level indicates that the target construction machinery and the target power transmission cable may come into contact after slight movement, posing a potential risk and requiring medium-priority manual verification.

[0096] 3. Low threat level assessment If the target construction machinery and the target power transmission cable do not meet the high or medium threat conditions, but meet any of the following conditions, they are judged to be low threat level: (1) The bounding box of the construction machinery is further expanded (e.g., the expansion ratio is 2.0 times), and the power transmission cable is simulated to move a certain distance along the nearest neighbor direction (e.g., 10% of the galloping threshold), and the masks of the two have an intersection; (2) The nearest neighbor distance is less than the large galloping threshold (e.g., the pixel distance corresponding to 10% of the length of the power transmission cable).

[0097] A low threat level indicates that the target construction machinery and the target power transmission cable may come into contact after significant movement, which is a potential threat and can be considered as an object of attention for subsequent inspections.

[0098] 4. Non-threat level assessment If the target construction machinery and the target power transmission cable do not meet any of the above conditions, or if the type of construction machinery does not have the potential to cause external damage to the power transmission cable (such as agricultural machinery, road rollers, and other low-height equipment), then it is determined to be of a no-threat level.

[0099] This level indicates that there is no foreseeable risk of contact between the construction machinery and the power transmission cables, and no alarm is required or only logging is necessary.

[0100] The embodiment of the present application can realize intelligent grade division of external damage hidden dangers without relying on depth information by the multi-feature fusion discrimination method based on the mask intersection degree, the bounding box overlap degree and the nearest neighbor distance, has good practicability and popularization value, and is suitable for old equipment and resource limited scenes.

[0101] After the above steps are completed, the embodiment of the present application will count the threat levels of all matching pairs in the current monitoring scene, report the highest threat level as the threat level of the current scene, report the threat levels of the specific matching pairs, and visualize the instance segmentation contour, bounding box and nearest neighbor line segment in the monitoring image to assist manual checking.

[0102] As shown in Figure 5 , the instance segmentation contour, bounding box and nearest neighbor line segment of the construction machinery and power transmission cable judged as high threat are visualized as red in the monitoring image.

[0103] As shown in Figure 6 , the instance segmentation contour, bounding box and nearest neighbor line segment of the construction machinery and power transmission cable judged as medium threat are visualized as yellow in the monitoring image.

[0104] As shown in Figure 7 , the instance segmentation contour, bounding box and nearest neighbor line segment of the construction machinery and power transmission cable judged as low threat are visualized as blue in the monitoring image.

[0105] As shown in Figure 8 , the instance segmentation contour, bounding box and nearest neighbor line segment of the construction machinery and power transmission cable judged as no threat are visualized as green in the monitoring image.

[0106] In the actual application scene, as shown in Figure 9 , using the traditional indiscriminate reporting mode, the alarm proportion is 79%, and the normal proportion is 21%. Using the method provided by the embodiment of the present application, the high threat alarm is 7%, the medium threat alarm is 5%, the low threat alarm is 3%, and the no threat accounts for 85%.

[0107] The results show that the manual review rate of the embodiment of the present application is only 15%, which is reduced by 64% compared with the traditional scheme of 79%, significantly improving the manual review efficiency and reducing the workload. In addition, the embodiment of the present application can also formulate priority strategies according to the hidden danger level, further improve the efficiency, and ensure the safety of the power transmission scene.

[0108] In order to perform the corresponding steps in the above embodiment and each possible mode, an implementation mode of an external damage hidden danger discrimination device 200 is given below. Further, please refer to Figure 10 , Figure 10This is a functional block diagram of an external damage hazard identification device 200 provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the external damage hazard identification device 200 provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The external damage hazard identification device 200 includes: The instance segmentation module 201 is used to acquire monitoring images, segment the monitoring images into instances, and obtain construction machinery instance information and power transmission cable instance information.

[0109] The distribution relationship establishment module 202 is used to establish the spatial distribution relationship between construction machinery and power transmission cables based on the instance information of construction machinery and the instance information of power transmission cables. The spatial distribution relationship reflects the connection characteristics of each construction machinery and the corresponding nearest power transmission cable.

[0110] The hazard identification module 203 is used to identify target construction machinery and target power transmission cables that have potential external damage hazards based on their spatial distribution.

[0111] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown is either stored in or embedded in the operating system (OS) of the electronic device 100, and can be used by... Figure 1 The processor 120 in the memory executes the commands. Meanwhile, the data and program code required to execute the above modules can be stored in memory.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0113] In addition, each functional module in various embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0114] If the functions are realized in the form of software functional modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, an electronic device 100, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0115] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying potential external damage, characterized in that, The method includes: Acquire monitoring images, perform instance segmentation on the monitoring images, and obtain construction machinery instance information and power transmission cable instance information; Based on the construction machinery instance information and the power transmission cable instance information, a spatial distribution relationship between construction machinery and power transmission cables is established. The spatial distribution relationship reflects the connection characteristics of each construction machinery and the corresponding nearest power transmission cable. Based on the spatial distribution relationship, target construction machinery and target power transmission cables with potential external damage are identified.

2. The method for identifying potential external damage as described in claim 1, characterized in that, The construction machinery instance information includes the mask for each construction machinery, and the power transmission cable instance information includes the mask for each power transmission cable. The step of establishing the spatial distribution relationship between construction machinery and power transmission cables based on the construction machinery instance information and the power transmission cable instance information includes: Extract the contour points of each construction machine from the mask of each construction machine; Extract the outline points of each of the power transmission cables from the mask of each power transmission cable; Matching analysis is performed on the contour points of each construction machine and the contour points of each power transmission cable to obtain first distribution information and second distribution information. The first distribution information reflects the correspondence between each construction machine and the nearest power transmission cable, and the second distribution information reflects the nearest neighbor line segment between each construction machine and the corresponding nearest power transmission cable. The spatial distribution relationship includes the first distribution information and the second distribution information.

3. The method for identifying potential external damage as described in claim 2, characterized in that, The step of matching and analyzing the contour points of each construction machine and the contour points of each power transmission cable to obtain the first distribution information and the second distribution information includes: The contour points of each of the power transmission cables are fitted to obtain the curve corresponding to each power transmission cable, and a preset number of curve points are sampled on the curve corresponding to each power transmission cable. Calculate the geometric center of the outline points of each of the construction machines; Based on the geometric center of the outline point of each construction machine and the curve point corresponding to each power transmission cable, determine the nearest power transmission cable corresponding to each construction machine to obtain the first distribution information; Based on the contour points of each construction machine and the curve points corresponding to the nearest power transmission cable, the nearest neighbor line segments between each construction machine and the corresponding nearest power transmission cable are constructed to obtain the second distribution information.

4. The method for identifying potential external damage as described in claim 1, characterized in that, The spatial distribution relationship includes first distribution information and second distribution information. The first distribution information reflects the correspondence between each construction machine and the nearest power transmission cable. The second distribution information reflects the nearest neighbor line segment between each construction machine and the corresponding nearest power transmission cable. The construction machine instance information includes the bounding box of each construction machine. The power transmission cable instance information includes the bounding box of each power transmission cable. The step of determining the target construction machinery and target power transmission cables with potential external damage based on the spatial distribution relationship includes: For each construction machine, based on the bounding frame of the construction machine and the bounding frame of the corresponding nearest power transmission cable, it is determined whether the construction machine and the corresponding nearest power transmission cable are at the same depth level; If it is determined that the construction machinery and the corresponding nearest power transmission cable are at the same depth level, based on the bounding frame of the construction machinery and the bounding frame of the corresponding nearest power transmission cable, it is determined whether the construction machinery is located below or near the corresponding nearest power transmission cable. If the construction machinery is located below and adjacent to the nearest power transmission cable, then the construction machinery is identified as the target construction machinery, and the nearest power transmission cable corresponding to the construction machinery is identified as the target power transmission cable.

5. The method for identifying potential external damage as described in claim 4, characterized in that, The step of determining whether the construction machinery and the corresponding nearest power transmission cable are at the same depth level based on the bounding frame of the construction machinery and the bounding frame of the corresponding nearest power transmission cable includes: Based on the aspect ratio of the frame surrounding the nearest power transmission cable to the construction machinery, determine whether the nearest power transmission cable to the construction machinery is visually vertical. If the nearest power transmission cable to the construction machinery is visually vertical, then the construction machinery and the corresponding nearest power transmission cable are determined to be at the same depth level. If the nearest power transmission cable to the construction machinery is not visually vertical, the depth ratio is obtained based on the diagonal pixel ratio of the bounding box of the construction machinery and the diagonal pixel ratio of the bounding box of the nearest power transmission cable. Based on the depth ratio, determine whether the construction machinery and the nearest power transmission cable are at the same depth level.

6. The method for identifying potential external damage as described in claim 4, characterized in that, The step of determining whether the construction machinery is located below or near the corresponding nearest power transmission cable based on the bounding frame of the construction machinery and the bounding frame of the corresponding nearest power transmission cable includes: If the horizontal projections of the bounding box of the construction machinery and the bounding box of the corresponding nearest power transmission cable on the image plane overlap, it is determined that the construction machinery is not located below or near the corresponding nearest power transmission cable. If there is no overlap in the horizontal projection of the bounding box of the construction machinery and the bounding box of the corresponding nearest power transmission cable on the image plane, it is determined whether the construction machinery is located below and adjacent to the corresponding nearest power transmission cable based on the coordinates of the two endpoints of the nearest neighbor line segment between the construction machinery and the corresponding nearest power transmission cable.

7. The method for identifying potential external damage as described in claim 1, characterized in that, The construction machinery instance information includes a mask and bounding box for each construction machinery, and the power transmission cable instance information includes a mask and bounding box for each power transmission cable. The method further includes: The external damage hazard level of the target construction machinery and the target power transmission cable is determined based on the intersection degree of the mask of the target construction machinery and the mask of the target power transmission cable, the overlap degree of the bounding frame of the target construction machinery and the bounding frame of the target power transmission cable, and the nearest distance between the target construction machinery and the target power transmission cable.

8. A device for identifying potential external damage, characterized in that, The device includes: The instance segmentation module is used to acquire monitoring images, perform instance segmentation on the monitoring images, and obtain construction machinery instance information and power transmission cable instance information. The distribution relationship establishment module is used to establish the spatial distribution relationship between the construction machinery and the power transmission cable based on the construction machinery instance information and the power transmission cable instance information. The spatial distribution relationship reflects the connection characteristics of each construction machinery and the corresponding nearest power transmission cable. The hazard identification module is used to identify target construction machinery and target power transmission cables that have potential external damage hazards based on the spatial distribution relationship.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the external damage hazard identification method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for identifying external damage hazards as described in any one of claims 1-7.

Images