Cable anti-breakage monitoring method, system, equipment, storage medium and product

Abstract

The present invention discloses a cable anti-external damage monitoring method, system, equipment, storage medium and product, the monitoring method includes target detection on the cable peripheral image, obtaining each target detection frame and its category and confidence; calculating the horizontal angle and pitch angle of each visual target, and converting to the world coordinate system; performing target detection on the target radar echo signal, obtaining the distance, horizontal angle, pitch angle and spatial position of each radar target, and converting to the world coordinate system; matching the visual target with the radar target, and obtaining the spatial position of each matching target in the world coordinate system; determining the cable point set according to the catenary coordinate system and the catenary model, and converting to the world coordinate system; judging whether the matching target has a risk of damage to the cable according to the spatial position of each matching target in the world coordinate system and the cable point set. The present invention greatly improves the monitoring accuracy of external damage targets and improves the monitoring accuracy of cable external damage.

Description

Technical Field

[0001] The present invention belongs to the technical field of power transmission line monitoring, and in particular relates to a cable anti-external damage monitoring method, system, equipment, storage medium and product based on lightning and vision fusion. Background Art

[0002] The safety of transmission and distribution cables is closely related to the stable operation of key infrastructure such as electricity and communications. Mobile construction is an important cause of cable damage. As cables are suspended in the air, they are vulnerable to threats from external objects such as tower cranes and drones. These external objects may collide with the cables due to wind, operational errors or other factors, causing damage or even breakage of the cables. Line damage is the main cause of unplanned power outages, accounting for more than half of the overall failures.

[0003] At present, the cable damage monitoring methods mainly include manual inspection and video monitoring. However, manual inspection has the problems of low efficiency and poor real-time performance, and it is unable to detect and prevent damage events in time. Although traditional video monitoring can provide a certain degree of real-time performance, it is limited by factors such as viewing angle and weather, and its monitoring effect is also limited. In addition, the existing monitoring methods are mostly based on a single sensor, which makes it difficult to achieve comprehensive perception and accurate early warning of complex damage events. Among them, vibration sensors, inclination sensors, and acceleration sensors can only identify damage after it occurs and cannot be monitored in advance; LiDAR can measure distance and angle, but the cost is high and it will also be affected by bad weather. Millimeter wave radar has the advantages of low cost and all-day and all-weather, but the angular resolution is low and it cannot output rich point cloud and image information. Summary of the invention

[0004] The purpose of the present invention is to provide a cable anti-external damage monitoring method, system, equipment, storage medium and product to solve at least one of the problems that traditional video monitoring technology is limited by factors such as viewing angle and weather, resulting in poor monitoring effect, and single sensor monitoring technology is difficult to fully perceive and accurately warn, and some sensors cannot monitor in advance.

[0005] The present invention solves the above technical problems through the following technical solutions: a cable anti-external damage monitoring method, comprising:

[0006] Perform target detection on the acquired cable surrounding images to obtain each target detection frame and its category and confidence;

[0007] Calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system according to the size of the image around the cable, the internal and external parameters of the visual sensor, and each target detection frame; convert the horizontal angle and pitch angle of each visual target in the visual coordinate system to the world coordinate system to obtain the horizontal angle and pitch angle of each visual target in the world coordinate system;

[0008] Perform target detection on the received target radar echo signal to obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system; convert the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to the world coordinate system to obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the world coordinate system;

[0009] According to the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system, the visual target and the radar target are matched to obtain the spatial position of each matched target in the world coordinate system;

[0010] Constructing a catenary coordinate system and a catenary model of the cable in the catenary coordinate system, and determining a cable point set according to the catenary coordinate system and the catenary model; converting each point in the cable point set into a point in a world coordinate system to obtain a cable point set in the world coordinate system;

[0011] According to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system, it is judged whether the matching target poses a risk of damaging the cable.

[0012] Furthermore, a single-stage multi-box detector-residual neural network model is used to perform target detection on the acquired cable surrounding images.

[0013] Further, the horizontal angle and the pitch angle of each visual target in the visual coordinate system are calculated according to the size of the cable surrounding image, the internal and external parameters of the visual sensor, and each target detection frame, specifically including:

[0014] According to the size of the cable surrounding image, the offset of the center of each target detection frame relative to the center of the cable surrounding image is calculated. The specific formula is:

[0015] △x=cx box -W img / 2; △y=cy box -H img / 2;

[0016] Among them, △x and △y represent the offset of the center of each target detection frame relative to the center of the cable surrounding image in the pixel coordinate system, and cx box 、cy box Represents the center of the target detection box in the pixel coordinate system, (W img , H img ) represents the pixel size of the image around the cable;

[0017] According to the internal and external parameters of the visual sensor and the offset of the center of each target detection frame relative to the center of the cable surrounding image, the horizontal angle and pitch angle of each visual target in the visual coordinate system are calculated. The specific formula is:

[0018] ; ;

[0019] Among them, f x 、f y represents the camera focal length of the vision sensor, Represents the horizontal angle of the visual target in the visual coordinate system, It represents the pitch angle of the visual target in the visual coordinate system, and arctan represents the inverse tangent function.

[0020] Furthermore, before calculating the horizontal angle and the pitch angle of each visual target in the visual coordinate system, the monitoring method further includes performing redundant processing on each target detection frame using a non-maximum suppression algorithm, specifically including:

[0021] Sort multiple target detection boxes belonging to the same category according to their confidence;

[0022] Select the target detection frame corresponding to the highest confidence as the reference frame of the category, and calculate the intersection-over-union ratio between the reference frame and other target detection frames in the category;

[0023] Determine whether each intersection-over-union ratio is greater than the overlap threshold. If so, delete the other target detection frames corresponding to the intersection-over-union ratio; otherwise, retain the other target detection frames corresponding to the intersection-over-union ratio;

[0024] For each of the multiple target detection frames belonging to the same category, the confidence sorting, intersection-over-union calculation and judgment steps are repeated to obtain the target detection frames after redundant processing and their categories and confidences.

[0025] Further, a single 4D imaging radar or a radar group is used to perform target detection on the received target radar echo signal; wherein the radar group includes a first radar and a second radar, the antenna direction of the first radar is a horizontal direction, the antenna direction of the second radar is a pitch direction, the radar group is located below the cable, the horizontal direction refers to a direction parallel to the horizontal line of the cable, and the pitch direction refers to a direction perpendicular to the horizontal direction and the ground; using the radar group to perform target detection on the received target radar echo signal specifically includes:

[0026] Utilizing the first radar to perform target detection on the received first target radar echo signal, and obtaining the distance and horizontal angle of the first radar target in the radar coordinate system; wherein the first target radar echo signal refers to the electromagnetic wave signal radiated externally by the first radar, the electromagnetic wave signal is reflected by the target and received by the first radar, and the first radar target refers to the target detected by the first radar;

[0027] The second radar is used to perform target detection on the received second target radar echo signal to obtain the distance and elevation angle of the second radar target in the radar coordinate system; wherein the second target radar echo signal refers to the electromagnetic wave signal radiated by the second radar, the electromagnetic wave signal is reflected by the target and received by the second radar, and the second radar target refers to the target detected by the second radar;

[0028] Determining whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system;

[0029] When the second radar target matches the first radar target, the average value of the distances between the second radar target and the first radar target in the radar coordinate system is used as the distance of the radar target, the elevation angle of the second radar target in the radar coordinate system is used as the elevation angle of the radar target, and the horizontal angle of the first radar target in the radar coordinate system is used as the horizontal angle of the radar target; the spatial position of the radar target is calculated according to the distance, horizontal angle and elevation angle of the radar target;

[0030] When the second radar target does not match the first radar target, deleting the second radar target;

[0031] The first radar target that is not matched is deleted, and the distance, horizontal angle, elevation angle and spatial position of all radar targets in the radar coordinate system are obtained.

[0032] Further, judging whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system specifically includes:

[0033] For each second radar target, calculating a distance difference between the second radar target and each first radar target according to the distance of the second radar target in the radar coordinate system and the distance of each first radar target in the radar coordinate system;

[0034] Select the first radar target corresponding to the minimum distance difference as the target to be matched;

[0035] If the second radar target and the target to be matched meet the conditions: (|r ndj -r sti ∣ / r sti )<T r, then the second radar target matches the target to be matched, and a radar target is obtained; otherwise, the second radar target does not match the target to be matched, and the second radar target is deleted; wherein, r ndj represents the distance of the jth second radar target in the radar coordinate system, r sti represents the distance of the target to be matched in the radar coordinate system, and the target to be matched is the i-th first radar target, T r Represents the ratio threshold.

[0036] Furthermore, before converting the distance, horizontal angle, elevation angle and spatial position of each radar target in the radar coordinate system to the world coordinate system, the monitoring method further includes clustering each radar target using a density-based clustering algorithm, which specifically includes:

[0037] For each radar target, taking the radar target as a starting point, calculating the number of radar targets in a neighborhood of the starting point;

[0038] Determine whether the number of radar targets in the neighborhood of the starting point is greater than or equal to a point number threshold, if so, the starting point is a core point; otherwise, the starting point is a non-core point;

[0039] For each core point, all radar targets within the neighborhood of the core point are classified into the same category;

[0040] For each non-core point, determine whether the non-core point is within the neighborhood of a core point; if so, classify the non-core point into the category of the core point; otherwise, the non-core point is a noise point and is deleted;

[0041] The core point in the same category is taken as the final radar target, and the distance, horizontal angle, pitch angle and spatial position of the core point in the radar coordinate system are taken as the distance, horizontal angle, pitch angle and spatial position of the final radar target in the radar coordinate system, and other radar targets in the category are deleted.

[0042] Furthermore, according to the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system, the visual target and the radar target are matched, specifically including:

[0043] A cost matrix is ​​constructed according to the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system; wherein the expression of the cost matrix is:

[0044] ;

[0045] ;

[0046] ;

[0047] Where D represents the cost matrix, D ij represents the cost between the i-th visual target and the j-th radar target, dis α () represents the weight-based cost metric function, C i represents the i-th visual target, i = 1, 2, ..., n, n represents the number of visual targets, R j represents the jth radar target, j = 1, 2, ..., m, m represents the number of radar targets, ω α represents the angle weight, ω d represents the distance weight, e ij represents the angle deviation between the i-th visual target and the j-th radar target, r' r represents the distance of the jth radar target in the world coordinate system, θ' azi represents the horizontal angle of the jth radar target in the world coordinate system, θ' ele represents the pitch angle of the jth radar target in the world coordinate system, represents the horizontal angle of the i-th visual target in the world coordinate system, Represents the pitch angle of the i-th visual target in the world coordinate system;

[0048] Based on the cost matrix, a matching matrix is ​​calculated using the Hungarian algorithm;

[0049] It is determined whether the cost value corresponding to the 0 element in the matching matrix is ​​less than the mismatch threshold. If so, the visual target corresponding to the cost value matches the radar target, and a matching target and the spatial position of the matching target in the world coordinate system are obtained; if not, the visual target corresponding to the cost value mismatches the radar target.

[0050] Furthermore, the Hungarian algorithm is used to calculate the matching matrix, which specifically includes:

[0051] Step A1: Determine whether the cost matrix is ​​a square matrix; if so, proceed to step A2; if not, fill the cost matrix to obtain a square matrix; wherein the value of the filling element in the square matrix is ​​greater than the mismatch threshold;

[0052] Step A2: Subtract the minimum element in each row from the elements in the square matrix;

[0053] Step A3: mark the first mark on the 0 elements in the square matrix that have no first mark in both the row and column;

[0054] Step A4: Mark all columns containing the first mark with the second mark;

[0055] Step A5: Determine whether all columns are marked with the second mark; if so, take the square matrix as the matching matrix and output it; if not, go to step A6;

[0056] Step A6: Determine whether there is a 0 element in the matrix that is not marked with the second mark; if so, mark the 0 element that is not marked with the second mark with the third mark, and proceed to step A7; if not, proceed to step A11;

[0057] Step A7: Determine whether the row where the 0 element marked with the third mark is located in the square matrix contains the 0 element marked with the first mark; if so, mark the row where the 0 element marked with the third mark is located with the second mark, cancel the second mark in the column where the 0 element marked with the first mark is located in the row where the 0 element marked with the third mark is located, and go to step A6; if not, use the 0 element marked with the third mark as the first edge of the augmented path, and go to step A8;

[0058] Step A8: If the column where the 0 element marked with the third mark exists in the square matrix contains the 0 element marked with the first mark, the 0 element marked with the first mark is used as the next edge of the augmented path; if the row where the 0 element marked with the first mark exists contains the 0 element marked with the third mark, the 0 element marked with the third mark is used as the next next edge of the augmented path, and so on, until there is no new 0 element marked with the first mark, then go to step A9;

[0059] Step A9: cancel the mark of the 0 element marked with the first mark in the augmented path, and re-mark the 0 element marked with the third mark in the augmented path with the first mark;

[0060] Step A10: cancel the second marks of all rows and columns, cancel the marks of all 0 elements marked with the third mark, and go to step A4;

[0061] Step A11: Find the smallest element in the matrix that is not marked with the second mark, add the smallest element to each element in the row marked with the second mark, subtract the smallest element from each element in the column not marked with the second mark, and go to step A6.

[0062] Furthermore, the catenary coordinate system takes the midpoint of the cable horizontal line as the origin, the cable horizontal line as the X-axis, the direction perpendicular to the ground downward as the Z-axis, and the direction perpendicular to the XOZ plane as the Y-axis; wherein the cable horizontal line refers to the straight line between the two end points of the cable and the two end points of the cable are at the same height;

[0063] The catenary model in the catenary coordinate system is:

[0064] z=acosh(x / a),y=0;

[0065] Where x, y, and z represent points on the cable in the catenary coordinate system, a represents the z-coordinate value of the lowest point on the cable, and cosh represents a hyperbolic function.

[0066] Further, determining the cable point set according to the catenary coordinate system and the catenary model includes:

[0067] Determine the value range of the first coordinate value of the point on the cable according to the distance between the two end points of the cable;

[0068] Selecting a plurality of first coordinate values ​​from the value range;

[0069] Calculate a second coordinate value corresponding to each first coordinate value according to the catenary model of the cable in the catenary coordinate system and each first coordinate value;

[0070] The coordinate position of a point on the cable in the catenary coordinate system is composed of a first coordinate value, a second coordinate value corresponding to the first coordinate value, and a third coordinate value; wherein the third coordinate value of each point on the cable is 0;

[0071] The cable point set is composed of the coordinate positions of all points on the cable in the catenary coordinate system.

[0072] Furthermore, according to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system, it is judged whether the matching target has a risk of damaging the cable, including:

[0073] For each matching target, calculating the minimum distance between the matching target and each point in the cable point set;

[0074] It is determined whether the minimum distance value is less than or equal to the external damage threshold value. If so, the matching target has a damage risk to the cable; otherwise, the matching target has no damage risk to the cable.

[0075] Further, a KD tree nearest neighbor algorithm is used to calculate the minimum distance between the matching target and each point in the cable point set, specifically including:

[0076] Constructing a KD tree based on the points in the cable point set;

[0077] Based on the KD tree, a nearest neighbor search algorithm is used to find a point in the cable point set with the smallest distance to the matching target;

[0078] The distance between the point and the matching target is calculated, and the distance is the minimum distance between the matching target and each point in the cable point set.

[0079] Based on the same concept, the present invention provides a cable anti-external damage monitoring system, the system comprising a visual sensor, a radar monitor and a data processor connected to the visual sensor and the radar monitor; the visual sensor and the radar monitor are both located below the cable and are arranged on a column at one end of the cable;

[0080] The visual sensor is used to collect images of the surrounding area of ​​the cable and send the images of the surrounding area of ​​the cable to the data processor;

[0081] The radar monitor is used to radiate electromagnetic wave signals outward, receive target radar echo signals, perform target detection on the target radar echo signals, obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system; and send the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to the data processor;

[0082] The data processor is used to perform target detection on the acquired cable surrounding image to obtain each target detection frame and its category and confidence; calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system according to the size of the cable surrounding image, the internal and external parameters of the visual sensor and each target detection frame; convert the horizontal angle and pitch angle of each visual target in the visual coordinate system to the world coordinate system to obtain the horizontal angle and pitch angle of each visual target in the world coordinate system; convert the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to the world coordinate system to obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the world coordinate system; and calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system according to the size of the cable surrounding image, the internal and external parameters of the visual sensor and the target detection frame. The horizontal angle and elevation angle in the world coordinate system and the distance, horizontal angle, elevation angle and spatial position of each radar target in the world coordinate system are used to match the visual target with the radar target to obtain the spatial position of each matching target in the world coordinate system; the catenary coordinate system of the cable and the catenary model of the cable in the catenary coordinate system are constructed, and the cable point set is determined according to the catenary coordinate system and the catenary model; each point in the cable point set is converted into a point in the world coordinate system to obtain the cable point set in the world coordinate system; and according to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system, whether the matching target poses a risk of damage to the cable is determined.

[0083] Further, the data processor is used to calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system according to the size of the cable surrounding image, the internal and external parameters of the visual sensor, and each target detection frame, specifically including:

[0084] According to the size of the cable surrounding image, the offset of the center of each target detection frame relative to the center of the cable surrounding image is calculated. The specific formula is:

[0085] △x=cxbox -W img / 2; △y=cy box -H ing / 2;

[0086] Among them, △x and △y represent the offset of the center of each target detection frame relative to the center of the cable surrounding image in the pixel coordinate system, and cx box 、cy box Represents the center of the target detection box in the pixel coordinate system, (W img , H img ) represents the pixel size of the image around the cable;

[0087] According to the internal and external parameters of the visual sensor and the offset of the center of each target detection frame relative to the center of the cable surrounding image, the horizontal angle and pitch angle of each visual target in the visual coordinate system are calculated. The specific formula is:

[0088] ; ;

[0089] Among them, f x 、f y represents the camera focal length of the vision sensor, Represents the horizontal angle of the visual target in the visual coordinate system, It represents the pitch angle of the visual target in the visual coordinate system, and arctan represents the inverse tangent function.

[0090] Furthermore, the data processor is also used to perform redundancy processing on each target detection frame using a non-maximum suppression algorithm, specifically including:

[0091] Sort multiple target detection boxes belonging to the same category according to their confidence;

[0092] Select the target detection frame corresponding to the highest confidence as the reference frame of the category, and calculate the intersection-over-union ratio between the reference frame and other target detection frames in the category;

[0093] Determine whether each intersection-over-union ratio is greater than the overlap threshold. If so, delete the other target detection frames corresponding to the intersection-over-union ratio; otherwise, retain the other target detection frames corresponding to the intersection-over-union ratio;

[0094] For each of the multiple target detection frames belonging to the same category, the confidence sorting, intersection-over-union calculation and judgment steps are repeated to obtain the target detection frames after redundant processing and their categories and confidences.

[0095] Further, the radar monitor includes a 4D imaging radar, which is used to radiate electromagnetic wave signals outward, receive target radar echo signals, perform target detection on the target radar echo signals, and obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system; and send the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to the data processor;

[0096] Alternatively, the radar monitor includes a radar group and a processing module; the radar group includes a first radar and a second radar, the antenna direction of the first radar is a horizontal direction, the antenna direction of the second radar is a pitch direction, the horizontal direction refers to a direction parallel to a horizontal line of the cable, and the pitch direction refers to a direction perpendicular to the horizontal direction and the ground;

[0097] The first radar is used to radiate a first electromagnetic wave signal, receive a first target radar echo signal, perform target detection on the first target radar echo signal, and obtain a distance and a horizontal angle of the first radar target in a radar coordinate system; wherein the first target radar echo signal refers to an echo signal of the first electromagnetic wave signal reflected by the target and received by the first radar, and the first radar target refers to a target detected by the first radar;

[0098] The second radar is used to radiate a second electromagnetic wave signal, receive a second target radar echo signal, perform target detection on the second target radar echo signal, and obtain a distance and an elevation angle of the second radar target in a radar coordinate system; wherein the second target radar echo signal refers to an echo signal of the second electromagnetic wave signal reflected by the target and received by the second radar, and the second radar target refers to a target detected by the second radar;

[0099] The processing module is used to determine whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system;

[0100] When the second radar target matches the first radar target, the average value of the distances between the second radar target and the first radar target in the radar coordinate system is used as the distance of the radar target, the elevation angle of the second radar target in the radar coordinate system is used as the elevation angle of the radar target, and the horizontal angle of the first radar target in the radar coordinate system is used as the horizontal angle of the radar target; the spatial position of the radar target is calculated according to the distance, horizontal angle and elevation angle of the radar target;

[0101] When the second radar target does not match the first radar target, deleting the second radar target;

[0102] The first radar target that is not matched is deleted, and the distance, horizontal angle, elevation angle and spatial position of all radar targets in the radar coordinate system are obtained.

[0103] Further, the processing module is used to determine whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system, specifically including:

[0104] For each second radar target, calculating a distance difference between the second radar target and each first radar target according to the distance of the second radar target in the radar coordinate system and the distance of each first radar target in the radar coordinate system;

[0105] Select the first radar target corresponding to the minimum distance difference as the target to be matched;

[0106] If the second radar target and the target to be matched meet the conditions: (|r ndj -r sti ∣ / r sti )<T r , then the second radar target matches the target to be matched, and a radar target is obtained; otherwise, the second radar target does not match the target to be matched, and the second radar target is deleted; wherein, r ndj represents the distance of the jth second radar target in the radar coordinate system, r sti represents the distance of the target to be matched in the radar coordinate system, and the target to be matched is the i-th first radar target, T r Represents the ratio threshold.

[0107] Furthermore, the data processor is also used to cluster the radar targets using a density-based clustering algorithm, specifically including:

[0108] For each radar target, taking the radar target as a starting point, calculating the number of radar targets in a neighborhood of the starting point;

[0109] Determine whether the number of radar targets in the neighborhood of the starting point is greater than or equal to a point number threshold, if so, the starting point is a core point; otherwise, the starting point is a non-core point;

[0110] For each core point, all radar targets within the neighborhood of the core point are classified into the same category;

[0111] For each non-core point, determine whether the non-core point is within the neighborhood of a core point; if so, classify the non-core point into the category of the core point; otherwise, the non-core point is a noise point and is deleted;

[0112] The core point in the same category is taken as the final radar target, and the distance, horizontal angle, pitch angle and spatial position of the core point in the radar coordinate system are taken as the distance, horizontal angle, pitch angle and spatial position of the final radar target in the radar coordinate system, and other radar targets in the category are deleted.

[0113] Further, the data processor is used to match the visual target with the radar target according to the horizontal angle and the pitch angle of each visual target in the world coordinate system and the distance, horizontal angle and pitch angle of each radar target in the world coordinate system, specifically including:

[0114] A cost matrix is ​​constructed according to the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system; wherein the expression of the cost matrix is:

[0115] ;

[0116] ;

[0117] ;

[0118] Where D represents the cost matrix, D ij represents the cost between the i-th visual target and the j-th radar target, dis α () represents the weight-based cost metric function, C i represents the i-th visual target, i = 1, 2, ..., n, n represents the number of visual targets, R j represents the jth radar target, j = 1, 2, ..., m, m represents the number of radar targets, ω α represents the angle weight, ω d represents the distance weight, e ij represents the angle deviation between the i-th visual target and the j-th radar target, r' r represents the distance of the jth radar target in the world coordinate system, θ' azi represents the horizontal angle of the jth radar target in the world coordinate system, θ' ele represents the pitch angle of the jth radar target in the world coordinate system, represents the horizontal angle of the i-th visual target in the world coordinate system, Represents the pitch angle of the i-th visual target in the world coordinate system;

[0119] Based on the cost matrix, a matching matrix is ​​calculated using the Hungarian algorithm;

[0120] It is determined whether the cost value corresponding to the 0 element in the matching matrix is ​​less than the mismatch threshold. If so, the visual target corresponding to the cost value matches the radar target, and a matching target and the spatial position of the matching target in the world coordinate system are obtained; if not, the visual target corresponding to the cost value mismatches the radar target.

[0121] Furthermore, the data processor is used to calculate the matching matrix using the Hungarian algorithm, specifically including:

[0122] Step A1: Determine whether the cost matrix is ​​a square matrix; if so, proceed to step A2; if not, fill the cost matrix to obtain a square matrix; wherein the value of the filling element in the square matrix is ​​greater than the mismatch threshold;

[0123] Step A2: Subtract the minimum element in each row from the elements in the square matrix;

[0124] Step A3: mark the first mark on the 0 elements in the square matrix that have no first mark in both the row and column;

[0125] Step A4: Mark all columns containing the first mark with the second mark;

[0126] Step A5: Determine whether all columns are marked with the second mark; if so, take the square matrix as the matching matrix and output it; if not, go to step A6;

[0127] Step A6: Determine whether there is a 0 element in the matrix that is not marked with the second mark; if so, mark the 0 element that is not marked with the second mark with the third mark, and proceed to step A7; if not, proceed to step A11;

[0128] Step A7: Determine whether the row where the 0 element marked with the third mark is located in the square matrix contains the 0 element marked with the first mark; if so, mark the row where the 0 element marked with the third mark is located with the second mark, cancel the second mark in the column where the 0 element marked with the first mark is located in the row where the 0 element marked with the third mark is located, and go to step A6; if not, use the 0 element marked with the third mark as the first edge of the augmented path, and go to step A8;

[0129] Step A8: If the column where the 0 element marked with the third mark exists in the square matrix contains the 0 element marked with the first mark, the 0 element marked with the first mark is used as the next edge of the augmented path; if the row where the 0 element marked with the first mark exists contains the 0 element marked with the third mark, the 0 element marked with the third mark is used as the next next edge of the augmented path, and so on, until there is no new 0 element marked with the first mark, then go to step A9;

[0130] Step A9: cancel the mark of the 0 element marked with the first mark in the augmented path, and re-mark the 0 element marked with the third mark in the augmented path with the first mark;

[0131] Step A10: cancel the second marks of all rows and columns, cancel the marks of all 0 elements marked with the third mark, and go to step A4;

[0132] Step A11: Find the smallest element in the matrix that is not marked with the second mark, add the smallest element to each element in the row marked with the second mark, subtract the smallest element from each element in the column not marked with the second mark, and go to step A6.

[0133] Further, the data processor is used to determine the cable point set according to the catenary coordinate system and the catenary model, specifically including:

[0134] Determine the value range of the first coordinate value of the point on the cable according to the distance between the two end points of the cable;

[0135] Selecting a plurality of first coordinate values ​​from the value range;

[0136] Calculate a second coordinate value corresponding to each first coordinate value according to the catenary model of the cable in the catenary coordinate system and each first coordinate value;

[0137] The coordinate position of a point on the cable in the catenary coordinate system is composed of a first coordinate value, a second coordinate value corresponding to the first coordinate value, and a third coordinate value; wherein the third coordinate value of each point on the cable is 0;

[0138] The cable point set is composed of the coordinate positions of all points on the cable in the catenary coordinate system.

[0139] Further, the data processor is used to determine whether the matching target has a risk of damaging the cable according to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system, specifically including:

[0140] For each matching target, calculating the minimum distance between the matching target and each point in the cable point set;

[0141] It is determined whether the minimum distance value is less than or equal to the external damage threshold value. If so, the matching target has a damage risk to the cable; otherwise, the matching target has no damage risk to the cable.

[0142] Furthermore, the data processor is used to calculate the minimum distance between the matching target and each point in the cable point set using a KD tree nearest neighbor algorithm, specifically including:

[0143] Constructing a KD tree based on the points in the cable point set;

[0144] Based on the KD tree, a nearest neighbor search algorithm is used to find a point in the cable point set with the smallest distance to the matching target;

[0145] The distance between the point and the matching target is calculated, and the distance is the minimum distance between the matching target and each point in the cable point set.

[0146] Based on the same concept, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instruction stored in the memory, wherein the processor executes the computer program / instruction to implement the cable anti-external damage monitoring method as described above.

[0147] Based on the same concept, the present invention also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the cable anti-external damage monitoring method as described above is implemented.

[0148] Based on the same concept, the present invention also provides a computer program product, including a computer program / instruction, which implements the cable anti-external damage monitoring method as described above when executed by a processor.

[0149] Beneficial Effects

[0150] Compared with the prior art, the advantages of the present invention are:

[0151] The present invention adopts the radar-visual fusion technology to monitor the external damage targets around the cable, and combines the advantages of visual sensors and millimeter-wave radars to achieve all-day and all-weather visual perception, greatly improving the monitoring accuracy of external damage targets, thereby improving the accuracy of cable external damage monitoring, avoiding the problem of poor monitoring accuracy caused by video monitoring alone, avoiding the problem of difficulty in comprehensive perception of single sensor monitoring, and avoiding the problem that some sensors cannot monitor in advance;

[0152] The present invention introduces a catenary model to model the cable, and combines the catenary coordinate system to determine the cable point set, which greatly reduces the calculation overhead while improving the cable positioning accuracy, and further improves the cable external damage monitoring accuracy and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0153] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0154] Figure 1 This is a flow chart of a cable anti-external damage monitoring method according to an embodiment of the present invention;

[0155] Figure 2 This is a diagram of the SSD-ResNet model architecture in an embodiment of the present invention;

[0156] Figure 3 : is a flow chart of the Hungarian algorithm for calculating the matching matrix in an embodiment of the present invention;

[0157] Figure 4 is a schematic diagram of marking a first mark on a 0 element in a square matrix that has no first mark in both its row and column in an embodiment of the present invention;

[0158] Figure 5 Schematic diagram of the catenary coordinate system in an embodiment of the present invention. DETAILED DESCRIPTION

[0159] The following is a clear and complete description of the technical solutions in the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0160] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0161] Example 1

[0162] Monitoring technology based on a single sensor is difficult to achieve comprehensive perception and accurate warning of complex external damage events, and some sensors can only identify external damage events after they occur, but cannot provide advance warnings. Therefore, the present invention uses radar-visual fusion technology to monitor cable external damage events. Millimeter-wave radar can make up for the lack of perception ability of visual monitoring technology in bad weather and directly measure accurate distance and location information, while visual monitoring technology can provide rich image information and realize object recognition and classification. The fusion of the two can give full play to their respective advantages and improve monitoring accuracy and robustness.

[0163] like Figure 1 As shown, a cable anti-external damage monitoring method provided by an embodiment of the present invention includes the following steps:

[0164] Step 1: Perform target detection on the acquired cable surrounding image to obtain each target detection box and its category and confidence.

[0165] The cable is fixed and pulled by the columns at both ends. A visual sensor is installed on one of the columns to collect images around the cable. Then, the target detection model is used to detect targets in the images around the cable to obtain each target detection frame and its category and confidence. The target detection model can obtain all target detection frames in the image and the category and confidence of each target detection frame.

[0166] In a specific embodiment of the present invention, a single-stage multi-box detector-residual neural network model (SingleShot MultiBox Detector-Residual Neural Network, SSD-ResNet) is used to perform target detection on the cable surrounding image. Figure 2 The SSD-ResNet model architecture diagram is shown, which uses ResNet-50, multi-layer convolutional network, and multi-dimensional fusion detector to obtain the detection box, category, and confidence of the external target. Figure 2 The w×w×c in the above formula represents the length×width×number of channels of the current feature layer. The feature layer size and number of channels are controlled by different convolution steps and padding. The size of the feature layer is reduced and the target features are extracted through multi-layer convolution operations. The SSD-ResNet model introduces multi-scale feature maps to capture small target information with rich details on low-level feature maps and detect global information on high-level feature maps, thereby achieving target detection with different levels of fusion. For each position of the feature map of the i-th layer, a different number of k i The default detection boxes generated by different layers have different areas and aspect ratios, so as to match more real target boundary information as much as possible. At the same time, q+1 category predictions and 4 detection box regression parameter predictions are performed. The category prediction includes the original q categories and the background class of category 0, while the detection box includes the center coordinates, width and height of the detection box.

[0167] ResNet-50 is used as the classification model to replace the VGG16 model. By introducing residual links, the gradient vanishing problem is effectively alleviated, thereby greatly enhancing the feature extraction capability and computational efficiency. A multi-task loss function is used to optimize the model, and the loss function L is divided into the classification loss L conf and the localization loss L loc Two parts, namely:

[0168] (1)

[0169] Where N represents the number of positive sample detection boxes detected; α L Represents the proportional coefficient. The classification loss calculates the cross entropy loss between the classification confidence and the true category label for each predicted detection box, while the localization loss uses the L2 loss to measure the difference between the predicted detection box and the true box. Since negative samples do not have corresponding true boxes, the localization loss only considers positive samples, and obtains the classification result through error back propagation and forward inference.

[0170] In order to avoid the influence of redundant target detection frames, before calculating the horizontal angle and pitch angle of each visual target in the visual coordinate system, the non-maximum suppression algorithm is used to perform redundant processing on each target detection frame, including:

[0171] Step 1.1: Sort multiple target detection boxes belonging to the same category according to their confidence;

[0172] Step 1.2: Select the target detection box corresponding to the highest confidence as the reference box of this category, and calculate the intersection and union ratio of the reference box and other target detection boxes in this category;

[0173] Step 1.3: Determine whether each intersection-over-union ratio is greater than the overlap threshold. If so, delete the other target detection frames corresponding to the intersection-over-union ratio; otherwise, retain the other target detection frames corresponding to the intersection-over-union ratio;

[0174] Step 1.4: For each of the multiple target detection boxes belonging to the same category, repeat steps 1.1 to 1.3 to obtain the target detection boxes and their categories and confidences after redundant processing.

[0175] Step 2: Calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system according to the size of the image around the cable, the internal and external parameters of the visual sensor, and each target detection frame.

[0176] By acquiring the internal and external parameters of the visual sensor, the position of the target detection frame in the image (i.e., the position based on the pixel coordinate system) is converted to the visual coordinate system. In this embodiment, the visual sensor is a high-definition camera, so the visual coordinate system is the camera coordinate system. In a specific embodiment of the present invention, the horizontal angle and pitch angle of each visual target in the visual coordinate system are calculated, specifically including:

[0177] Step 2.1: According to the size of the cable surrounding image, calculate the offset of the center of each target detection frame relative to the center of the cable surrounding image. The specific formula is:

[0178] △x=cx box -W img / twenty two)

[0179] △y=cy box -H img / twenty three)

[0180] Among them, △x and △y represent the offset of the center of each target detection frame relative to the center of the cable surrounding image in the pixel coordinate system, and cx box 、cy box Represents the center of the target detection box in the pixel coordinate system, (W img , H img ) represents the pixel size of the image around the cable.

[0181] Step 2.2: Based on the internal and external parameters of the visual sensor and the offset of the center of each target detection frame relative to the center of the cable surrounding image, calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system. The specific formula is:

[0182] (4)

[0183] (5)

[0184] Among them, f x 、f y represents the camera focal length of the vision sensor, Represents the horizontal angle of the visual target in the visual coordinate system, It represents the pitch angle of the visual target in the visual coordinate system, and arctan represents the inverse tangent function.

[0185] Step 3: Convert the horizontal angle and pitch angle of each visual target in the visual coordinate system to the world coordinate system to obtain the horizontal angle and pitch angle of each visual target in the world coordinate system.

[0186] Coordinate transformation includes two steps: translation and rotation. Let P old is a point in the old coordinate system, which can be expressed as P in the homogeneous coordinate system. old =[x,y,z,1] T , the superscript T represents transposition, and using the homogeneous transformation matrix, we can get the coordinates of the point in the new coordinate system:

[0187] P new =M·P old (6)

[0188] , , (7)

[0189] Among them, P new represents the point in the new coordinate system; M represents the coordinate transformation matrix; T represents the translation matrix, T x ,T y ,T z Respectively represent the translation on the X, Y, and Z coordinate axes; R represents the combined rotation matrix, the rotation matrix R of the X, Y, and Z coordinate axes x (γ x ), R y (γ y ) and R z (γ z ) are:

[0190] (8)

[0191] (9)

[0192] (10)

[0193] Among them, γ x ,γ y and γ z Respectively represent the rotation angles from the old coordinate system to the new coordinate system on the X, Y, and Z coordinate axes, and their value ranges are all 0 to 360°. In this embodiment, the visual coordinate system coincides with the world coordinate system, so it is only necessary to map the horizontal angle and pitch angle of each visual target in the visual coordinate system to the world coordinate system.

[0194] Step 4: Perform target detection on the received target radar echo signal to obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system.

[0195] The present invention can use a single 4D imaging radar to detect the target radar echo signal, or a radar group to detect the target radar echo signal. In order to reduce costs and obtain a higher level of angular resolution, the present embodiment uses a radar group to detect the target radar echo signal. The radar group is located directly below the visual sensor, and the mounting plate of the radar group is parallel to the mounting position of the visual sensor. The radar group includes a first radar and a second radar with the same parameters. The antenna direction of the first radar is the horizontal direction, and the antenna direction of the second radar is the pitch direction. The antenna positions of the first radar and the second radar are as close as possible; wherein the horizontal direction refers to the direction parallel to the horizontal line of the cable, and the pitch direction refers to the direction perpendicular to the horizontal direction and the ground. The first radar and the second radar are both BPM-MIMO (Binary Phase Modulation- Multiple Input Multiple Output) array radars. The first radar and the second radar include N T transmit antennas and N R By expanding the antenna technology, N receiving antennas are formed. T ×N R The virtual antenna has a high angular resolution in the antenna direction. In this embodiment, the radar coordinate system is a right-handed system constructed with the center position of the radar group as the coordinate origin, the front of the radar group as the Y axis, the direction perpendicular to the Y axis and horizontal to the X axis, and the direction perpendicular to the ground as the Z axis.

[0196] In a specific embodiment of the present invention, a radar group is used to perform target detection on a target radar echo signal, specifically including:

[0197] Step 4.1: Detect the received first target radar echo signal using the first radar to obtain the distance and horizontal angle of the first radar target in the radar coordinate system; wherein the first target radar echo signal refers to the electromagnetic wave signal radiated by the first radar, which is reflected by the target and received by the first radar, and the first radar target refers to the target detected by the first radar;

[0198] Step 4.2: Detect the received second target radar echo signal using the second radar to obtain the distance and elevation angle of the second radar target in the radar coordinate system; wherein the second target radar echo signal refers to the electromagnetic wave signal radiated by the second radar, which is reflected by the target and received by the second radar, and the second radar target refers to the target detected by the second radar;

[0199] Step 4.3: judging whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system;

[0200] When the second radar target matches the first radar target, the average value of the distance between the second radar target and the first radar target in the radar coordinate system is used as the distance of the radar target, the elevation angle of the second radar target in the radar coordinate system is used as the elevation angle of the radar target, and the horizontal angle of the first radar target in the radar coordinate system is used as the horizontal angle of the radar target; the spatial position of the radar target is calculated according to the distance, horizontal angle and elevation angle of the radar target, and the specific calculation formula is:

[0201] x ri =r ri sinθ azii cosθ elei (11)

[0202] y ri =r ri cosθ azii cosθ elei (12)

[0203] z ri =r ri sinθ elei (13)

[0204] Among them, (x ri ,y ri ,z ri ) represents the spatial position of the i-th radar target in the radar coordinate system, θ azii represents the horizontal angle of the i-th radar target in the radar coordinate system, θ elei represents the elevation angle of the ith radar target in the radar coordinate system, r rirepresents the distance of the i-th radar target in the radar coordinate system;

[0205] When the second radar target does not match the first radar target, deleting the second radar target;

[0206] Step 4.4: Delete the first radar target that is not matched, and obtain the distance, horizontal angle, elevation angle and spatial position of all radar targets in the radar coordinate system.

[0207] In step 4.3, judging whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system specifically includes:

[0208] Step 4.31: For each second radar target, calculate the distance difference between the second radar target and each first radar target according to the distance of the second radar target in the radar coordinate system and the distance of each first radar target in the radar coordinate system;

[0209] Step 4.32: Select the first radar target corresponding to the minimum distance difference as the target to be matched;

[0210] Step 4.33: If the second radar target and the target to be matched meet the conditions: (|r ndj -r sti ∣ / r sti )<T r , the second radar target matches the target to be matched, and a radar target is obtained; otherwise, the second radar target does not match the target to be matched, and the second radar target is deleted;

[0211] Among them, r ndj represents the distance of the jth second radar target in the radar coordinate system, r sti represents the distance of the target to be matched in the radar coordinate system, and the target to be matched is the i-th first radar target, T r Represents the ratio threshold.

[0212] In the radar group, according to the characteristics of different beams, the observable interval is expanded, and the observed radar targets are fused. Specifically, the radar group of the present invention adopts a radar system in which wide-beam and narrow-beam antennas coexist. The wide-beam antenna has a shorter ranging range, a larger field of view, and a better resolution effect, while the narrow-beam radar can detect a farther distance. By fusing the targets measured by the wide-beam and narrow-beam antennas, a larger measurement range can be guaranteed. When the target to be measured moves in a certain direction at a constant rate, due to the difference in propagation path, phase and frequency changes will occur. The millimeter-wave radar measures the target speed through Doppler frequency shift and calculates the target speed. In addition, the Doppler compensation algorithm is used for speed expansion, and phase correction is performed during parameter estimation to ensure the correctness of the estimated angle. The virtual antenna array formed by the radar group based on the MIMO array obtains the horizontal-pitch angle information of the radar target in the radar coordinate system, and multi-frame accumulation is performed to obtain richer point cloud information.

[0213] Due to the application of MIMO antenna expansion technology and dual-beam system, for the target with volume and contour information, if its size is larger than the radar resolution unit, it will be reflected as multiple scattering points in the radar, that is, the same target is detected by the radar group to obtain multiple radar targets. In order to merge multiple radar targets belonging to the same target into one radar target, before converting the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to the world coordinate system, the density-based spatial clustering algorithm (Density-Based Spatial Clustering ofApplications with Noise, DBSCAN) is used to cluster each radar target, including:

[0214] For each radar target, take the radar target as the starting point and calculate the number of radar targets in the neighborhood of the starting point;

[0215] Determine whether the number of radar targets in the neighborhood of the starting point is greater than or equal to the point number threshold. If so, the starting point is a core point; otherwise, the starting point is a non-core point;

[0216] For each core point, all radar targets in the neighborhood of the core point are classified into the same category;

[0217] For each non-core point, determine whether the non-core point is in the neighborhood of a core point; if so, classify the non-core point into the category of the core point; otherwise, the non-core point is a noise point and is deleted;

[0218] The core point in the same category is taken as the final radar target, and the distance, horizontal angle, pitch angle and spatial position of the core point in the radar coordinate system are taken as the distance, horizontal angle, pitch angle and spatial position of the final radar target in the radar coordinate system, and other radar targets in the category are deleted.

[0219] Step 5: Convert the distance, horizontal angle, elevation angle and spatial position of each radar target in the radar coordinate system to the world coordinate system to obtain the distance, horizontal angle, elevation angle and spatial position of each radar target in the world coordinate system.

[0220] Taking the radar coordinate system as the old coordinate system and the world coordinate system as the new coordinate system, the spatial position of each radar target in the radar coordinate system is first converted to the world coordinate system according to formula (6) to formula (10), and then the distance, horizontal angle and pitch angle of the corresponding radar target in the world coordinate system are calculated according to the spatial position of each radar target in the world coordinate system. The specific calculation formula is:

[0221] (14)

[0222] (15)

[0223] (16)

[0224] Among them, (x' r ,y' r ,z' r ) represents the spatial position of the radar target in the world coordinate system, r' r Represents the distance of the radar target in the world coordinate system, θ' azi Represents the horizontal angle of the radar target in the world coordinate system, θ' ele Indicates the elevation angle of the radar target in the world coordinate system.

[0225] Step 6: According to the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system, the visual target is matched with the radar target to obtain the spatial position of each matched target in the world coordinate system.

[0226] n visual targets are detected by visual monitoring technology, m radar targets are detected by radar detection technology, and the targets detected by the visual sensor and the radar group are matched so that each target has a unique association. The matching problem of visual targets and radar targets can be converted into a matching problem of bipartite graphs. In a specific embodiment of the present invention, matching visual targets and radar targets specifically includes:

[0227] Step 6.1: Construct a cost matrix based on the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system; wherein the expression of the cost matrix is:

[0228] (14)

[0229] (15)

[0230] (16)

[0231] Where D represents the cost matrix, D ij represents the cost between the i-th visual target and the j-th radar target, dis α () represents the weight-based cost metric function, C i represents the i-th visual target, i = 1, 2, ..., n, n represents the number of visual targets, R j represents the jth radar target, j = 1, 2, ..., m, m represents the number of radar targets, ω α represents the angle weight, ω d represents the distance weight, e ij represents the angle deviation between the i-th visual target and the j-th radar target, r' r represents the distance of the jth radar target in the world coordinate system, θ' azi represents the horizontal angle of the jth radar target in the world coordinate system, θ' ele represents the pitch angle of the jth radar target in the world coordinate system, represents the horizontal angle of the i-th visual target in the world coordinate system, Represents the pitch angle of the i-th visual target in the world coordinate system.

[0232] Since the visual sensor has an occlusion problem, it tends to select targets that are closer to the visual sensor at similar angles. α ,ω d To change this tendency, generally speaking α >0,ω d >0.

[0233] Step 6.2: Based on the cost matrix, use the Hungarian algorithm to calculate the matching matrix.

[0234] Step 6.3: Determine whether the cost value corresponding to the 0 element in the matching matrix is ​​less than the mismatch threshold. If so, the visual target corresponding to the cost value matches the radar target, and a matching target and the spatial position of the matching target in the world coordinate system are obtained; if not, the visual target corresponding to the cost value mismatches the radar target.

[0235] In this embodiment, the mismatch threshold is defined as:

[0236] T m =n α ·ω α +n d ·ω d (17)

[0237] Among them, T m represents the mismatch threshold, n α 、n d Represents angle noise and distance noise respectively. Due to the artificial and systematic errors in the installation and calibration of the radar group and visual sensor, the measurement process is also affected by environmental noise. Even the same target will not completely overlap. Therefore, when dis α (C i ,R j )<T m , it is considered a successful match; otherwise, a mismatch.

[0238] In step 6.2, if Figure 3 As shown, the Hungarian algorithm is used to calculate the matching matrix, including:

[0239] Step A1: Determine whether the cost matrix is ​​a square matrix; if so, proceed to step A2; if not, fill the cost matrix to obtain a square matrix; wherein the value of the filling element in the square matrix is ​​greater than the mismatch threshold.

[0240] Since the number of visual targets n detected by the visual sensor is not necessarily equal to the number of radar targets m detected by the radar group, it is necessary to first expand the cost matrix into a square matrix by adding virtual rows or virtual columns. When n < m, add n-m virtual rows; when n > m, add m-n virtual columns. In order to ensure that virtual rows or virtual columns are not added to the match, let the elements in the virtual rows or virtual columns be T m +1.

[0241] Step A2: Subtract the minimum element in each row from the elements in the square matrix.

[0242] Step A3: Add the first mark to the 0 elements in the matrix that have no first mark in both the row and the column.

[0243] For example, Figure 4 As shown, the square matrix processed by step A2 is processed by step A3, and the first zero element in the fourth column is marked with the first mark (i.e., an asterisk mark); since the first zero element in the fourth column is marked with the first mark, other zero elements cannot be marked with the first mark. Among them, the fourth column is a virtual column.

[0244] Step A4: Mark all columns containing the first mark with the second mark.

[0245] by Figure 4 For example, after processing in step A4, the first 0 element in the 4th column has both the first mark and the second mark, and the other elements in the 4th column have only the second mark.

[0246] Step A5: Determine whether all columns are marked with the second mark; if so, use the square matrix as the matching matrix and output it; if not, go to step A6.

[0247] Step A6: Determine whether there is a 0 element in the matrix that is not marked with the second mark; if so, mark the 0 element that is not marked with the second mark with the third mark and proceed to step A7; if not, proceed to step A11.

[0248] Step A7: Determine whether the row where the 0 element marked with the third mark in the square matrix is ​​located contains the 0 element marked with the first mark; if so, mark the row where the 0 element marked with the third mark is located with the second mark, cancel the second mark in the column where the 0 element marked with the first mark in the row where the 0 element marked with the third mark is located, and go to step A6; if not, use the 0 element marked with the third mark as the first edge of the augmented path, and go to step A8.

[0249] Step A8: Construct an augmented path: If the column where the 0 element marked with the third mark is located in the square matrix contains a 0 element marked with the first mark, then the 0 element marked with the first mark is used as the next edge of the augmented path; if the row where the 0 element marked with the first mark is located contains a 0 element marked with the third mark, then the 0 element marked with the third mark is used as the next-next edge of the augmented path, and so on, until there is no new 0 element marked with the first mark, then go to step A9.

[0250] Step A9: cancel the mark of the 0 element marked with the first mark in the augmented path, and re-mark the 0 element marked with the third mark in the augmented path with the first mark.

[0251] Step A10: Cancel the second marks of all rows and columns, cancel the marks of all 0 elements marked with the third mark, and go to step A4.

[0252] Step A11: Find the smallest element in the matrix that is not marked with the second mark, add the smallest element to each element in the row marked with the second mark, subtract the smallest element from each element in the column not marked with the second mark, and go to step A6.

[0253] After processing from step A1 to step A11, a matching matrix is ​​obtained. Exemplarily, if the element of the 1st column and the 4th row of the obtained matching matrix is ​​0, corresponding to the cost matrix, the 1st visual target and the 4th radar target may match, and then it is determined whether the cost value of the 1st visual target and the 4th radar target (i.e., the element of the 1st column and the 4th row in the cost matrix) is less than the mismatch threshold. If so, the 1st visual target and the 4th radar target are matched, and a matching target is obtained, and the spatial position of the 4th radar target is used as the spatial position of the matching target.

[0254] The present invention matches visual targets with radar targets based on horizontal-pitch angle information and utilizes the Hungarian algorithm of optimized cost matrix, accurately provides accurate distance information for each target detected in the target detection frame, jointly outputs the target position, and realizes real-time feedback of the target position.

[0255] Step 7: Construct a catenary coordinate system and a catenary model of the cable in the catenary coordinate system, and determine the cable point set based on the catenary coordinate system and the catenary model.

[0256] Pre-break monitoring relies on the setting of warning areas, a process also known as "virtual fences". Traditional warning areas are set based on the physical space or radius around the wires, generally set to spheres or irregular polygons to adapt to the terrain or buildings around the cables. Different monitoring areas and monitoring levels can be set for different application scenarios to match different working mechanisms and alarm mechanisms. Since the cable is a catenary, it is difficult to set the warning area, and it is difficult to reduce the probability of false alarms while ensuring that there is no missed detection. In addition, considering that the catenary equation of the cable is established and fitted through coordinate system conversion, a surface is set to surround the catenary, so that for any point on the catenary, a point can always be found on the surface, and the distance of the point is equal to the warning distance. However, in actual deployment, the analytical equation from the point to the catenary is difficult to obtain directly due to coordinate transformation. Based on this, the cable point set is determined according to the catenary coordinate system and the catenary model.

[0257] like Figure 5 As shown in the figure, the catenary coordinate system takes the midpoint of the cable horizontal line as the origin, the cable horizontal line as the X-axis, the direction perpendicular to the ground downward as the Z-axis, and the direction perpendicular to the XOZ plane as the Y-axis; the cable horizontal line refers to the straight line between the two end points of the cable and the two end points of the cable are at the same height. The catenary model in the catenary coordinate system is:

[0258] z=acosh(x / a),y=0 (18)

[0259] Where x, y, z represent points on the cable in the catenary coordinate system, a represents the z-coordinate value of the lowest point on the cable, and cosh represents the hyperbolic function. Since the cables are on the same plane, the Y-coordinate of each point in the cable point set is 0. The distance a from the lowest point of the cable to the origin can be measured, or converted to formula (18) based on the distance between the two columns:

[0260] (19)

[0261] Among them, L c Indicates the cable length, D c represents the distance between the two pillars, and then a is solved by Newton's iteration method.

[0262] In a specific embodiment of the present invention, determining the cable point set according to the catenary coordinate system and the catenary model includes:

[0263] Step 7.1: Determine the range of the first coordinate value of the point on the cable according to the distance between the two end points of the cable.

[0264] For example, for Figure 5 In the catenary coordinate system, the first coordinate value is the X coordinate value, the second coordinate value is the Z coordinate value, and the third coordinate value is the Y coordinate value.

[0265] Step 7.2: Select multiple first coordinate values ​​from the value range.

[0266] Step 7.3: Calculate a second coordinate value corresponding to each first coordinate value based on the catenary model of the cable in the catenary coordinate system and each first coordinate value.

[0267] Substitute multiple first coordinate values ​​into the catenary model (Formula (18)) to obtain the corresponding second coordinate values.

[0268] Step 7.4: The coordinate position of the point on the cable in the catenary coordinate system is composed of the first coordinate value, the second coordinate value corresponding to the first coordinate value, and the third coordinate value; wherein the third coordinate value of each point on the cable is 0.

[0269] Step 7.5: The cable point set is formed by the coordinate positions of all points on the cable in the catenary coordinate system.

[0270] Step 8: Convert each point in the cable point set into a point in the world coordinate system to obtain the cable point set in the world coordinate system.

[0271] The catenary coordinate system is used as the old coordinate system and the world coordinate system is used as the new coordinate system. According to formulas (6) to (10), the cable point set in the catenary coordinate system can be converted to the world coordinate system. The present invention introduces a catenary model to model the cable and performs coordinate conversion on the discrete cable point set, thereby improving the cable positioning accuracy and greatly reducing the calculation overhead.

[0272] Step 9: Based on the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system, determine whether the matching target poses a risk of damaging the cable.

[0273] In a specific embodiment of the present invention, judging whether the matching target has a risk of damaging the cable according to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system includes:

[0274] For each matching target, calculate the minimum distance between the matching target and each point in the cable point set;

[0275] It is determined whether the minimum distance value is less than or equal to the external damage threshold. If so, the matching target has a risk of damaging the cable and an early warning is issued; otherwise, the matching target has no risk of damaging the cable.

[0276] In order to reduce the computational complexity, the KD tree (k-dimensional tree) nearest neighbor algorithm is used to calculate the minimum distance between the matching target and each point in the cable point set, including:

[0277] Construct a KD tree based on the points in the cable point set;

[0278] Based on the KD tree, the nearest neighbor search algorithm is used to find the point in the cable point set with the smallest distance to the matching target;

[0279] Calculate the distance between the point and the matching target, which is the minimum distance between the matching target and each point in the cable point set.

[0280] The KD tree nearest neighbor algorithm greatly reduces the computational complexity, and the construction of the KD tree can be processed offline in advance. Since the sensor position is fixed, the structure of the KD tree will not change, which greatly improves the operating efficiency.

[0281] Example 2

[0282] An embodiment of the present invention provides a cable anti-external damage monitoring system including a visual sensor, a radar monitor and a data processor connected to the visual sensor and the radar monitor; the visual sensor and the radar monitor are both located below the cable and arranged on a column at one end of the cable.

[0283] The visual sensor is used to collect images around the cable and send them to the data processor. The data processor is used to perform target detection on the acquired images around the cable to obtain each target detection frame and its category and confidence; calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system according to the size of the image around the cable, the internal and external parameters of the visual sensor and each target detection frame; convert the horizontal angle and pitch angle of each visual target in the visual coordinate system to the world coordinate system to obtain the horizontal angle and pitch angle of each visual target in the world coordinate system.

[0284] The radar monitor is used to radiate electromagnetic wave signals outward, receive target radar echo signals, perform target detection on target radar echo signals, and obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system; the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system are sent to the data processor. The data processor is used to convert the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to the world coordinate system, and obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the world coordinate system.

[0285] The data processor is used to match the visual target with the radar target according to the horizontal angle and pitch angle of each visual target in the world coordinate system and the distance, horizontal angle, pitch angle and spatial position of each radar target in the world coordinate system, so as to obtain the spatial position of each matching target in the world coordinate system; to construct the catenary coordinate system of the cable and the catenary model of the cable in the catenary coordinate system, and to determine the cable point set according to the catenary coordinate system and the catenary model; to convert each point in the cable point set into a point in the world coordinate system, so as to obtain the cable point set in the world coordinate system; and to judge whether the matching target poses a risk of damage to the cable according to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system.

[0286] In a specific embodiment of the present invention, the data processor is used to calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system according to the size of the cable surrounding image, the internal and external parameters of the visual sensor, and each target detection frame, specifically including:

[0287] According to the size of the image around the cable, the offset of the center of each target detection frame relative to the center of the image around the cable is calculated, see formula (2) and formula (3); according to the internal and external parameters of the visual sensor and the offset of the center of each target detection frame relative to the center of the image around the cable, the horizontal angle and pitch angle of each visual target in the visual coordinate system are calculated, see formula (4) and formula (5).

[0288] In a specific embodiment of the present invention, the data processor is further used to perform redundancy processing on each target detection frame using a non-maximum suppression algorithm, specifically including:

[0289] Multiple target detection frames belonging to the same category are sorted according to their confidences; the target detection frame corresponding to the highest confidence is selected as the reference frame of the category, and the intersection-over-union ratio between the reference frame and other target detection frames in the category is calculated; it is determined whether each intersection-over-union ratio is greater than the overlap threshold, and if so, the other target detection frames corresponding to the intersection-over-union ratio are deleted; otherwise, the other target detection frames corresponding to the intersection-over-union ratio are retained; for each of the multiple target detection frames belonging to the same category, the confidence sorting, intersection-over-union ratio calculation and judgment steps are repeated to obtain each target detection frame after redundant processing and its category and confidence.

[0290] In a specific embodiment of the present invention, the radar monitor includes a 4D imaging radar, which is used to radiate electromagnetic wave signals outward, receive target radar echo signals, perform target detection on the target radar echo signals, and obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system; and send the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to a data processor.

[0291] In another specific embodiment of the present invention, the radar monitor includes a radar group and a processing module; the radar group includes a first radar and a second radar, the antenna direction of the first radar is a horizontal direction, and the antenna direction of the second radar is a pitch direction, the horizontal direction refers to a direction parallel to the horizontal line of the cable, and the pitch direction refers to a direction perpendicular to the horizontal direction and the ground.

[0292] The first radar is used to radiate a first electromagnetic wave signal externally, receive a first target radar echo signal, perform target detection on the first target radar echo signal, and obtain a distance and a horizontal angle of the first radar target in a radar coordinate system; wherein the first target radar echo signal refers to an echo signal of the first electromagnetic wave signal reflected by the target and received by the first radar, and the first radar target refers to a target detected by the first radar.

[0293] The second radar is used to radiate a second electromagnetic wave signal externally, receive a second target radar echo signal, perform target detection on the second target radar echo signal, and obtain the distance and elevation angle of the second radar target in the radar coordinate system; wherein the second target radar echo signal refers to the echo signal of the second electromagnetic wave signal reflected by the target and received by the second radar, and the second radar target refers to the target detected by the second radar.

[0294] The processing module is used to determine whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system;

[0295] When the second radar target matches the first radar target, the average value of the distances between the second radar target and the first radar target in the radar coordinate system is used as the distance of the radar target, the elevation angle of the second radar target in the radar coordinate system is used as the elevation angle of the radar target, and the horizontal angle of the first radar target in the radar coordinate system is used as the horizontal angle of the radar target; the spatial position of the radar target is calculated according to the distance, horizontal angle and elevation angle of the radar target, see formula (11) to formula (13);

[0296] When the second radar target does not match the first radar target, deleting the second radar target;

[0297] The first radar target that is not matched is deleted, and the distance, horizontal angle, elevation angle and spatial position of all radar targets in the radar coordinate system are obtained.

[0298] In a specific embodiment of the present invention, the processing module is used to determine whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system, specifically including:

[0299] For each second radar target, calculating a distance difference between the second radar target and each first radar target according to the distance of the second radar target in the radar coordinate system and the distance of each first radar target in the radar coordinate system;

[0300] Select the first radar target corresponding to the minimum distance difference as the target to be matched;

[0301] If the second radar target and the target to be matched meet the conditions: (|r ndj -r sti ∣ / r sti )<T r , then the second radar target matches the target to be matched, and a radar target is obtained; otherwise, the second radar target does not match the target to be matched, and the second radar target is deleted; wherein, r ndj represents the distance of the jth second radar target in the radar coordinate system, r sti represents the distance of the target to be matched in the radar coordinate system, and the target to be matched is the i-th first radar target, T r Represents the ratio threshold.

[0302] In a specific embodiment of the present invention, the data processor is further used to cluster each radar target using a density-based clustering algorithm, specifically including:

[0303] For each radar target, take the radar target as the starting point and calculate the number of radar targets in the neighborhood of the starting point;

[0304] Determine whether the number of radar targets in the neighborhood of the starting point is greater than or equal to the point number threshold. If so, the starting point is a core point; otherwise, the starting point is a non-core point;

[0305] For each core point, all radar targets in the neighborhood of the core point are classified into the same category;

[0306] For each non-core point, determine whether the non-core point is in the neighborhood of a core point; if so, classify the non-core point into the category of the core point; otherwise, the non-core point is a noise point and is deleted;

[0307] The core point in the same category is taken as the final radar target, and the distance, horizontal angle, pitch angle and spatial position of the core point in the radar coordinate system are taken as the distance, horizontal angle, pitch angle and spatial position of the final radar target in the radar coordinate system, and other radar targets in the category are deleted.

[0308] In a specific embodiment of the present invention, the data processor is used to match the visual target with the radar target according to the horizontal angle and the pitch angle of each visual target in the world coordinate system and the distance, horizontal angle and pitch angle of each radar target in the world coordinate system, specifically including:

[0309] A cost matrix is ​​constructed according to the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system; wherein the cost matrix is ​​shown in formula (14) to formula (16);

[0310] Based on the cost matrix, the Hungarian algorithm is used to calculate the matching matrix;

[0311] Determine whether the cost value corresponding to the 0 element in the matching matrix is ​​less than the mismatch threshold. If so, the visual target corresponding to the cost value matches the radar target, and a matching target and the spatial position of the matching target in the world coordinate system are obtained; if not, the visual target corresponding to the cost value mismatches the radar target.

[0312] In a specific embodiment of the present invention, Figure 3 As shown, the data processor is used to calculate the matching matrix using the Hungarian algorithm, specifically including:

[0313] Step A1: Determine whether the cost matrix is ​​a square matrix; if so, proceed to step A2; if not, fill the cost matrix to obtain a square matrix; wherein the value of the filling element in the square matrix is ​​greater than the mismatch threshold;

[0314] Step A2: Subtract the minimum element in each row from the elements in the square matrix;

[0315] Step A3: mark the first mark on the 0 elements in the square matrix that have no first mark in both the row and column;

[0316] Step A4: Mark all columns containing the first mark with the second mark;

[0317] Step A5: Determine whether all columns are marked with the second mark; if so, take the square matrix as the matching matrix and output it; if not, go to step A6;

[0318] Step A6: Determine whether there is a 0 element in the matrix that is not marked with the second mark; if so, mark the 0 element that is not marked with the second mark with the third mark, and proceed to step A7; if not, proceed to step A11;

[0319] Step A7: Determine whether the row where the 0 element marked with the third mark is located in the square matrix contains the 0 element marked with the first mark; if so, mark the row where the 0 element marked with the third mark is located with the second mark, cancel the second mark in the column where the 0 element marked with the first mark is located in the row where the 0 element marked with the third mark is located, and go to step A6; if not, use the 0 element marked with the third mark as the first edge of the augmented path, and go to step A8;

[0320] Step A8: If the column where the 0 element marked with the third mark exists in the square matrix contains the 0 element marked with the first mark, the 0 element marked with the first mark is used as the next edge of the augmented path; if the row where the 0 element marked with the first mark exists contains the 0 element marked with the third mark, the 0 element marked with the third mark is used as the next next edge of the augmented path, and so on, until there is no new 0 element marked with the first mark, then go to step A9;

[0321] Step A9: cancel the mark of the 0 element marked with the first mark in the augmented path, and re-mark the 0 element marked with the third mark in the augmented path with the first mark;

[0322] Step A10: cancel the second marks of all rows and columns, cancel the marks of all 0 elements marked with the third mark, and go to step A4;

[0323] Step A11: Find the smallest element in the matrix that is not marked with the second mark, add the smallest element to each element in the row marked with the second mark, subtract the smallest element from each element in the column not marked with the second mark, and go to step A6.

[0324] In a specific embodiment of the present invention, the data processor is used to determine the cable point set according to the catenary coordinate system and the catenary model, specifically including:

[0325] Determine the range of the first coordinate value of a point on the cable according to the distance between the two end points of the cable; select multiple first coordinate values ​​from the range; calculate the second coordinate value corresponding to each first coordinate value according to the catenary model of the cable in the catenary coordinate system and each first coordinate value; the coordinate position of the point on the cable in the catenary coordinate system is composed of the first coordinate value, the second coordinate value corresponding to the first coordinate value, and the third coordinate value; wherein the third coordinate value of each point on the cable is 0; and the coordinate positions of all points on the cable in the catenary coordinate system constitute a cable point set.

[0326] In a specific embodiment of the present invention, the data processor is used to determine whether the matching target has a risk of damaging the cable according to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system, specifically including:

[0327] For each matching target, calculate the minimum distance between the matching target and each point in the cable point set;

[0328] It is determined whether the minimum distance value is less than or equal to the external damage threshold. If so, the matching target has a damage risk to the cable; otherwise, the matching target has no damage risk to the cable.

[0329] In a specific embodiment of the present invention, the data processor is used to calculate the minimum distance between the matching target and each point in the cable point set using the KD tree nearest neighbor algorithm, specifically including:

[0330] A KD tree is constructed based on the points in the cable point set. Based on the KD tree, a nearest neighbor search algorithm is used to find the point in the cable point set with the smallest distance to the matching target. The distance between the point and the matching target is calculated, and the distance is the minimum distance between the matching target and each point in the cable point set.

[0331] Example 3

[0332] An embodiment of the present invention further provides an electronic device, which includes: a memory, a processor, and a computer program / instructions stored in the memory, and the processor executes the computer program / instructions to implement the cable anti-external damage monitoring method in the embodiment of the present application.

[0333] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) or the programs and / or data loaded from the storage part into the random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM, and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0334] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0335] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the cable anti-external damage monitoring method in the embodiment of the present application is implemented.

[0336] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0337] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the cable anti-external damage monitoring method in the embodiment of the present application.

[0338] What is disclosed above is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A cable anti-breakage monitoring method, characterized in that: The monitoring method comprises: Perform target detection on the acquired cable surrounding images to obtain each target detection frame and its category and confidence; Calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system according to the size of the image around the cable, the internal and external parameters of the visual sensor, and each target detection frame; convert the horizontal angle and pitch angle of each visual target in the visual coordinate system to the world coordinate system to obtain the horizontal angle and pitch angle of each visual target in the world coordinate system; Perform target detection on the received target radar echo signal to obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system; convert the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to the world coordinate system to obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the world coordinate system; According to the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system, the visual target and the radar target are matched to obtain the spatial position of each matched target in the world coordinate system; Constructing a catenary coordinate system and a catenary model of the cable in the catenary coordinate system, and determining a cable point set according to the catenary coordinate system and the catenary model; converting each point in the cable point set into a point in a world coordinate system to obtain a cable point set in the world coordinate system; According to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system, it is determined whether the matching target poses a risk of damaging the cable; Wherein, determining the cable point set according to the catenary coordinate system and the catenary model includes: Determine the value range of the first coordinate value of the point on the cable according to the distance between the two end points of the cable; Selecting a plurality of first coordinate values ​​from the value range; Calculate a second coordinate value corresponding to each first coordinate value according to the catenary model of the cable in the catenary coordinate system and each first coordinate value; The coordinate position of a point on the cable in the catenary coordinate system is composed of a first coordinate value, a second coordinate value corresponding to the first coordinate value, and a third coordinate value; wherein the third coordinate value of each point on the cable is 0; The cable point set is composed of the coordinate positions of all points on the cable in the catenary coordinate system.

2. The cable anti-breakage monitoring method according to claim 1, characterized in that: The single-stage multi-box detector-residual neural network model is used to perform target detection on the acquired cable surrounding images.

3. The cable anti-external damage monitoring method according to claim 1 is characterized in that: The horizontal angle and the pitch angle of each visual target in the visual coordinate system are calculated according to the size of the image around the cable, the internal and external parameters of the visual sensor, and each target detection frame, specifically including: According to the size of the cable surrounding image, the offset of the center of each target detection frame relative to the center of the cable surrounding image is calculated. The specific formula is: △x=cx box -W img / 2;△y=cy box -H img / 2; Among them, △x and △y represent the offset of the center of each target detection frame relative to the center of the cable surrounding image in the pixel coordinate system, and cx box 、cy box Represents the center of the target detection box in the pixel coordinate system, (W img , H img ) represents the pixel size of the image around the cable; According to the internal and external parameters of the visual sensor and the offset of the center of each target detection frame relative to the center of the cable surrounding image, the horizontal angle and pitch angle of each visual target in the visual coordinate system are calculated. The specific formula is: ； ； Among them, f x 、f y represents the camera focal length of the vision sensor, Represents the horizontal angle of the visual target in the visual coordinate system, It represents the pitch angle of the visual target in the visual coordinate system, and arctan represents the inverse tangent function.

4. The cable anti-breakage monitoring method according to claim 1, characterized in that: Before calculating the horizontal angle and the pitch angle of each visual target in the visual coordinate system, the monitoring method further includes performing redundant processing on each target detection frame using a non-maximum suppression algorithm, specifically including: Sort multiple target detection boxes belonging to the same category according to their confidence; Select the target detection frame corresponding to the highest confidence as the reference frame of the category, and calculate the intersection-over-union ratio between the reference frame and other target detection frames in the category; Determine whether each intersection-over-union ratio is greater than the overlap threshold. If so, delete the other target detection frames corresponding to the intersection-over-union ratio; otherwise, retain the other target detection frames corresponding to the intersection-over-union ratio; For each of the multiple target detection frames belonging to the same category, the confidence sorting, intersection-over-union calculation and judgment steps are repeated to obtain the target detection frames after redundant processing and their categories and confidences.

5. The cable anti-breakage monitoring method according to claim 1, characterized in that: A single 4D imaging radar or a radar group is used to perform target detection on the received target radar echo signal; wherein the radar group includes a first radar and a second radar, the antenna direction of the first radar is a horizontal direction, the antenna direction of the second radar is a pitch direction, the radar group is located below the cable, the horizontal direction refers to a direction parallel to the horizontal line of the cable, and the pitch direction refers to a direction perpendicular to the horizontal direction and the ground; the radar group is used to perform target detection on the received target radar echo signal, specifically including: Utilizing the first radar to perform target detection on the received first target radar echo signal, and obtaining the distance and horizontal angle of the first radar target in the radar coordinate system; wherein the first target radar echo signal refers to the electromagnetic wave signal radiated externally by the first radar, the electromagnetic wave signal is reflected by the target and received by the first radar, and the first radar target refers to the target detected by the first radar; The second radar is used to perform target detection on the received second target radar echo signal to obtain the distance and elevation angle of the second radar target in the radar coordinate system; wherein the second target radar echo signal refers to the electromagnetic wave signal radiated by the second radar, the electromagnetic wave signal is reflected by the target and received by the second radar, and the second radar target refers to the target detected by the second radar; Determining whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system; When the second radar target matches the first radar target, the average value of the distances between the second radar target and the first radar target in the radar coordinate system is used as the distance of the radar target, the elevation angle of the second radar target in the radar coordinate system is used as the elevation angle of the radar target, and the horizontal angle of the first radar target in the radar coordinate system is used as the horizontal angle of the radar target; the spatial position of the radar target is calculated according to the distance, horizontal angle and elevation angle of the radar target; When the second radar target does not match the first radar target, deleting the second radar target; The first radar target that is not matched is deleted, and the distance, horizontal angle, elevation angle and spatial position of all radar targets in the radar coordinate system are obtained.

6. The cable anti-external damage monitoring method according to claim 5 is characterized in that: Judging whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system specifically includes: For each second radar target, calculating a distance difference between the second radar target and each first radar target according to the distance of the second radar target in the radar coordinate system and the distance of each first radar target in the radar coordinate system; Select the first radar target corresponding to the minimum distance difference as the target to be matched; If the second radar target and the target to be matched meet the conditions: (|r ndj -r sti ∣ / r sti )<T r , then the second radar target matches the target to be matched, and a radar target is obtained; otherwise, the second radar target does not match the target to be matched, and the second radar target is deleted; wherein, r ndj represents the distance of the jth second radar target in the radar coordinate system, r sti represents the distance of the target to be matched in the radar coordinate system, and the target to be matched is the i-th first radar target, T r Represents the ratio threshold.

7. The cable anti-breakage monitoring method according to claim 1, characterized in that: Before converting the distance, horizontal angle, elevation angle and spatial position of each radar target in the radar coordinate system to the world coordinate system, the monitoring method further includes clustering each radar target using a density-based clustering algorithm, which specifically includes: For each radar target, taking the radar target as a starting point, calculating the number of radar targets in a neighborhood of the starting point; Determine whether the number of radar targets in the neighborhood of the starting point is greater than or equal to a point number threshold, if so, the starting point is a core point; otherwise, the starting point is a non-core point; For each core point, all radar targets within the neighborhood of the core point are classified into the same category; For each non-core point, determine whether the non-core point is within the neighborhood of a core point; if so, classify the non-core point into the category of the core point; otherwise, the non-core point is a noise point and is deleted; The core point in the same category is taken as the final radar target, and the distance, horizontal angle, pitch angle and spatial position of the core point in the radar coordinate system are taken as the distance, horizontal angle, pitch angle and spatial position of the final radar target in the radar coordinate system, and other radar targets in the category are deleted.

8. The cable anti-external damage monitoring method according to claim 1, characterized in that: According to the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system, the visual target and the radar target are matched, including: A cost matrix is ​​constructed according to the horizontal angle and elevation angle of each visual target in the world coordinate system and the distance, horizontal angle and elevation angle of each radar target in the world coordinate system; wherein the expression of the cost matrix is: ； ； ； Where D represents the cost matrix, D ij represents the cost between the i-th visual target and the j-th radar target, dis α () represents the weight-based cost metric function, C i represents the i-th visual target, i = 1, 2, ..., n, n represents the number of visual targets, R j represents the jth radar target, j = 1, 2, ..., m, m represents the number of radar targets, represents the angle weight, ω d represents the distance weight, e ij represents the angle deviation between the i-th visual target and the j-th radar target, r' r represents the distance of the jth radar target in the world coordinate system, θ' azi represents the horizontal angle of the jth radar target in the world coordinate system, θ' ele represents the pitch angle of the jth radar target in the world coordinate system, represents the horizontal angle of the i-th visual target in the world coordinate system, Represents the pitch angle of the i-th visual target in the world coordinate system; Based on the cost matrix, a matching matrix is ​​calculated using the Hungarian algorithm; It is determined whether the cost value corresponding to the 0 element in the matching matrix is ​​less than the mismatch threshold. If so, the visual target corresponding to the cost value matches the radar target, and a matching target and the spatial position of the matching target in the world coordinate system are obtained; if not, the visual target corresponding to the cost value mismatches the radar target.

9. The cable anti-external damage monitoring method according to claim 8, characterized in that: The Hungarian algorithm is used to calculate the matching matrix, which includes: Step A1: Determine whether the cost matrix is ​​a square matrix; if so, proceed to step A2; if not, fill the cost matrix to obtain a square matrix; wherein the value of the filling element in the square matrix is ​​greater than the mismatch threshold; Step A2: Subtract the minimum element in each row from the elements in the square matrix; Step A3: mark the first mark on the 0 elements in the square matrix that have no first mark in both the row and column; Step A4: Mark all columns containing the first mark with the second mark; Step A5: Determine whether all columns are marked with the second mark; if so, take the square matrix as the matching matrix and output it; if not, go to step A6; Step A6: Determine whether there is a 0 element in the matrix that is not marked with the second mark; if so, mark the 0 element that is not marked with the second mark with the third mark, and proceed to step A7; if not, proceed to step A11; Step A7: Determine whether the row where the 0 element marked with the third mark is located in the square matrix contains the 0 element marked with the first mark; if so, mark the row where the 0 element marked with the third mark is located with the second mark, cancel the second mark in the column where the 0 element marked with the first mark is located in the row where the 0 element marked with the third mark is located, and go to step A6; if not, use the 0 element marked with the third mark as the first edge of the augmented path, and go to step A8; Step A8: If the column where the 0 element marked with the third mark exists in the square matrix contains the 0 element marked with the first mark, the 0 element marked with the first mark is used as the next edge of the augmented path; if the row where the 0 element marked with the first mark exists contains the 0 element marked with the third mark, the 0 element marked with the third mark is used as the next next edge of the augmented path, and so on, until there is no new 0 element marked with the first mark, then go to step A9; Step A9: cancel the mark of the 0 element marked with the first mark in the augmented path, and re-mark the 0 element marked with the third mark in the augmented path with the first mark; Step A10: cancel the second marks of all rows and columns, cancel the marks of all 0 elements marked with the third mark, and go to step A4; Step A11: Find the smallest element in the matrix that is not marked with the second mark, add the smallest element to each element in the row marked with the second mark, subtract the smallest element from each element in the column not marked with the second mark, and go to step A6.

10. The cable anti-external damage monitoring method according to claim 1, characterized in that: The catenary coordinate system takes the midpoint of the cable horizontal line as the origin, the cable horizontal line as the X-axis, the direction perpendicular to the ground downward as the Z-axis, and the direction perpendicular to the XOZ plane as the Y-axis; wherein the cable horizontal line refers to the straight line between the two end points of the cable and the two end points of the cable are at the same height; The catenary model in the catenary coordinate system is: z=acosh(x / a),y=0; Where x, y, and z represent points on the cable in the catenary coordinate system, a represents the z-coordinate value of the lowest point on the cable, and cosh represents a hyperbolic function.

11. The cable anti-external damage monitoring method according to claim 1, characterized in that: According to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system, it is judged whether the matching target has a risk of damaging the cable, including: For each matching target, calculating the minimum distance between the matching target and each point in the cable point set; It is determined whether the minimum distance value is less than or equal to the external damage threshold value. If so, the matching target has a damage risk to the cable; otherwise, the matching target has no damage risk to the cable.

12. The cable anti-external damage monitoring method according to claim 11, characterized in that: The KD tree nearest neighbor algorithm is used to calculate the minimum distance between the matching target and each point in the cable point set, specifically including: Constructing a KD tree based on the points in the cable point set; Based on the KD tree, a nearest neighbor search algorithm is used to find a point in the cable point set with the smallest distance to the matching target; The distance between the point and the matching target is calculated, and the distance is the minimum distance between the matching target and each point in the cable point set.

13. A cable anti-breakage monitoring system, characterized in that: The system includes a visual sensor, a radar monitor and a data processor connected to the visual sensor and the radar monitor; the visual sensor and the radar monitor are both located below the cable and are arranged on a column at one end of the cable; The visual sensor is used to collect images of the surrounding area of ​​the cable and send the images of the surrounding area of ​​the cable to the data processor; The radar monitor is used to radiate electromagnetic wave signals outward, receive target radar echo signals, perform target detection on the target radar echo signals, obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system; and send the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to the data processor; The data processor is used to perform target detection on the acquired cable surrounding image to obtain each target detection frame and its category and confidence; calculate the horizontal angle and pitch angle of each visual target in the visual coordinate system according to the size of the cable surrounding image, the internal and external parameters of the visual sensor and each target detection frame; convert the horizontal angle and pitch angle of each visual target in the visual coordinate system to the world coordinate system to obtain the horizontal angle and pitch angle of each visual target in the world coordinate system; convert the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to the world coordinate system to obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the world coordinate system; match the visual target with the radar target according to the horizontal angle and pitch angle of each visual target in the world coordinate system and the distance, horizontal angle, pitch angle and spatial position of each radar target in the world coordinate system to obtain the spatial position of each matched target in the world coordinate system; Used to construct a catenary coordinate system of the cable and a catenary model of the cable in the catenary coordinate system, and determine a cable point set according to the catenary coordinate system and the catenary model; Convert each point in the cable point set into a point in the world coordinate system to obtain a cable point set in the world coordinate system; and for judging whether the matching target has a risk of damaging the cable according to the spatial position of each matching target in the world coordinate system and the cable point set in the world coordinate system; The data processor is used to determine the cable point set according to the catenary coordinate system and the catenary model, including: Determine the value range of the first coordinate value of the point on the cable according to the distance between the two end points of the cable; Selecting a plurality of first coordinate values ​​from the value range; Calculate a second coordinate value corresponding to each first coordinate value according to the catenary model of the cable in the catenary coordinate system and each first coordinate value; The coordinate position of a point on the cable in the catenary coordinate system is composed of a first coordinate value, a second coordinate value corresponding to the first coordinate value, and a third coordinate value; wherein the third coordinate value of each point on the cable is 0; The cable point set is composed of the coordinate positions of all points on the cable in the catenary coordinate system.

14. The cable anti-external damage monitoring system according to claim 13, characterized in that: The radar monitor includes a 4D imaging radar, which is used to radiate electromagnetic wave signals outward, receive target radar echo signals, perform target detection on the target radar echo signals, obtain the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system; and send the distance, horizontal angle, pitch angle and spatial position of each radar target in the radar coordinate system to a data processor; Alternatively, the radar monitor includes a radar group and a processing module; the radar group includes a first radar and a second radar, the antenna direction of the first radar is a horizontal direction, the antenna direction of the second radar is a pitch direction, the horizontal direction refers to a direction parallel to a horizontal line of the cable, and the pitch direction refers to a direction perpendicular to the horizontal direction and the ground; The first radar is used to radiate a first electromagnetic wave signal, receive a first target radar echo signal, perform target detection on the first target radar echo signal, and obtain a distance and a horizontal angle of the first radar target in a radar coordinate system; wherein the first target radar echo signal refers to an echo signal of the first electromagnetic wave signal reflected by the target and received by the first radar, and the first radar target refers to a target detected by the first radar; The second radar is used to radiate a second electromagnetic wave signal, receive a second target radar echo signal, perform target detection on the second target radar echo signal, and obtain a distance and an elevation angle of the second radar target in a radar coordinate system; wherein the second target radar echo signal refers to an echo signal of the second electromagnetic wave signal reflected by the target and received by the second radar, and the second radar target refers to a target detected by the second radar; The processing module is used to determine whether the first radar target matches the second radar target according to the distance between the first radar target and the second radar target in the radar coordinate system; When the second radar target matches the first radar target, the average value of the distances between the second radar target and the first radar target in the radar coordinate system is used as the distance of the radar target, the elevation angle of the second radar target in the radar coordinate system is used as the elevation angle of the radar target, and the horizontal angle of the first radar target in the radar coordinate system is used as the horizontal angle of the radar target; the spatial position of the radar target is calculated according to the distance, horizontal angle and elevation angle of the radar target; When the second radar target does not match the first radar target, deleting the second radar target; The first radar target that is not matched is deleted, and the distance, horizontal angle, elevation angle and spatial position of all radar targets in the radar coordinate system are obtained.

15. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the cable anti-external damage monitoring method as described in any one of claims 1 to 12.

16. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the cable anti-external damage monitoring method as described in any one of claims 1 to 12 is implemented.

17. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the cable anti-external damage monitoring method as described in any one of claims 1 to 12 is implemented.

Images