A method and system for identifying and spatially locating power cables

Abstract

The present invention relates to the field of cable identification technology, and discloses a method and system for power cable identification and spatial positioning, including the following steps: S1: Detect target cables, complete cable identification in the environment, and obtain cable point cloud data; S2: Process point cloud data through a visual processing program, and perform fitting reconstruction and pose calculation in combination with cable identification data; S3: The visual processing module sends the final processing result to the terminal. The present invention uses three types of sensors to cooperate with each other, complement each other's advantages, and uses an optimization algorithm installed in the device chip, so that the device can achieve accurate positioning of cables even in unfavorable operating environments such as strong light, backlight, and complex spatial environment.

Description

Technical Field

[0001] The present invention relates to the technical field of cable identification, and in particular to a method and system for identifying and spatially locating a power cable. Background Art

[0002] There are two main ways to identify and locate cables during live operations such as disconnecting leads: manual identification and positioning and single sensor identification and positioning. Manual identification and positioning is done by workers locating the cable posture based on remotely transmitted video images and then performing subsequent operations; the latter uses sensors to scan the on-site environment and inputs the scanned images into algorithms for processing to achieve positioning.

[0003] The shortcomings of the above technical solutions are:

[0004] 1. Low efficiency;

[0005] 2. Poor recognition accuracy;

[0006] 3. The recognition effect is poor in strong light and backlight environments, and the resulting image data has large noise;

[0007] The few multi-sensor fusion recognition technologies currently available focus on universality, but have poor professionalism in the field of live cable identification and cannot meet all operational needs. Summary of the invention

[0008] The purpose of the present invention is to provide a method and system for identifying and spatially locating power cables to solve the problems raised in the background technology.

[0009] To achieve the above object, the present invention provides the following technical solution: a method for identifying and spatially locating a power cable, comprising the following steps:

[0010] S1: Detect the target cable, identify the cables in the environment, and obtain the cable point cloud data;

[0011] S2: Process the point cloud data through a visual processing program, and combine it with the cable identification data to perform fitting reconstruction and calculate the pose;

[0012] S3: The visual processing module sends the final processing results to the terminal.

[0013] As a preferred embodiment of the present invention, the step S1 includes the following steps:

[0014] S1.1: Use binocular cameras to identify cables and stripping heads in the environment;

[0015] S1.2: LiDAR detects targets by emitting laser beams and collects reflected beams to form point clouds and acquire data:

[0016] S1.3: The depth camera uses its own infrared projector to project a light spot, which helps improve the measurement accuracy of the binocular camera and forms point cloud data.

[0017] As a preferred embodiment of the present invention, in step S1.1, the binocular camera completes the recognition of cables and stripping heads in the environment through an image segmentation module and a stereo vision positioning module, wherein the image segmentation module includes a cable recognition network submodule and a feature extraction network submodule, and the specific implementation steps of step S1.1 are as follows:

[0018] S1.11: First, the cable image is collected by a calibrated binocular camera, and then the obtained left and right images of the cable are respectively input into the cable recognition network submodule for recognition;

[0019] S1.12: The cable identification network submodule first extracts features from the input image through a convolutional neural network to obtain a feature layer, which is then fed into a trained region proposal network to obtain a proposal box. The proposal box is then used to extract features on the feature layer, and proposal boxes of different sizes are resized through ROI Align. Finally, these resized proposal boxes are classified, regressed, and masked to obtain left and right mask images of the cable, which are then input into the feature extraction network submodule for feature point extraction.

[0020] S1.13: Feature extraction network submodule, first analyzes the direction of the transmission line contour. When the aspect ratio is much greater than 1, the direction of the transmission line is close to the vertical direction. When the aspect ratio is much less than 1, the direction of the transmission line is closer to the parallel direction. If the overall contour of the transmission line is close to the vertical direction, the contour is scanned row by row to obtain the center point set of the contour. The scanning process is as follows:

[0021] Assume that the outer contour point set of the transmission line is P1(u1,v1),P2(u2,v2),...P k (u k ,v k ), where the u axis points to the horizontal direction of the image, and the v axis points to the vertical direction of the image. Scan downwards row by row from the top of the circumscribed rectangle. Assume that the circumscribed rectangle of the contour has a total of m rows from the top to the bottom, and their ordinates are y1, y2..., y m , take yi as the ordinate of the candidate center point in turn, find the value equal to the ordinate from the contour point set, get several points that meet the conditions, perform deduplication processing on these points, set a distance threshold, if the Euclidean distance between two points is less than the threshold, it means that the distance between the two points is very close, then delete one of them. After the processing is completed, generally two edge points can be obtained for one yi, and the average of the two edge points can be obtained to get a center point, whose pixel coordinates are (xi ,y i ), after scanning all the row coordinates of the circumscribed rectangle, we can get a set of center points, recorded as Q1(x1,y1),Q2(x2,y2),...,Q n (x n ,y n ), and then the relevant feature points of the left and right target cables are obtained.

[0022] S1.14: Stereo vision positioning module, directly sparsely matches the feature points in two two-dimensional circuit images;

[0023] S1.15: Generate disparity map;

[0024] S1.16: Calculate the depth based on the disparity, calculate the three-dimensional coordinates, and complete the positioning;

[0025] As a preferred embodiment of the present invention, in step S2, the visual processing module includes a point cloud data processing module, a point cloud filtering module, a point cloud segmentation module, a straight line fitting module, a coordinate acquisition module and a posture calculation module. The specific implementation steps of step S2 are as follows:

[0026] S2.1 point cloud data processing module, reads the point cloud data received from three sensors;

[0027] S2.2 point cloud filtering module performs filtering operation on the point cloud. Point cloud filtering helps to remove noise points and outliers, making the point cloud smoother. The point cloud filtering method used is one or more of straight-through filtering, outlier filtering, voxel filtering, and radius filtering.

[0028] S2.3 point cloud segmentation module performs segmentation processing on point cloud data, and the point cloud segmentation processing adopts the Euclidean clustering segmentation method;

[0029] S2.4 straight line fitting module performs straight line fitting on the segmented point cloud data. The straight line fitting adopts RanSaC algorithm to reduce the influence of noise.

[0030] S2.5 coordinate acquisition module acquires coordinate data;

[0031] S2.6 The posture calculation module calculates the posture.

[0032] As a preferred embodiment of the present invention, in step S2, the laser radar and the depth camera can directly generate point cloud data, but the three-dimensional point coordinates obtained by the binocular camera are sent to the visual processing module, which publishes them as point cloud data and unifies them into the point cloud processing flow program of the visual processing, wherein the point cloud data of the three-dimensional point coordinates does not need to be processed such as denoising and can be directly applied.

[0033] A power cable identification and spatial positioning system, applied in a power cable identification and spatial positioning method, comprising a binocular camera, a laser radar, a depth camera, an AI chip and a terminal;

[0034] The laser radar uses non-contact laser ranging technology, and its working principle is similar to that of a general radar system, which detects targets by emitting laser beams and forms point clouds and acquires data by collecting reflected beams.

[0035] The binocular camera is used to identify and measure the target with high precision, which mainly utilizes the direct difference in the lateral coordinates of the target point imaged in the left and right views, and then uses the triangulation principle to obtain depth data to form a three-dimensional point cloud;

[0036] The depth camera adopts active binocular vision technology, and transmits light spots through an infrared projector to provide infrared patterns in low-texture environments to improve binocular measurement accuracy;

[0037] The AI ​​chip internally stores an image segmentation module, a stereoscopic vision positioning module and a vision processing module.

[0038] As a preferred implementation manner of the present invention, the depth camera is a RealSense depth camera.

[0039] Compared with the prior art, the present invention provides a method for identifying and spatially locating power cables, which has the following beneficial effects:

[0040] 1. LiDAR has good environmental adaptability and high reliability; binocular camera has high precision; Realsense depth camera can better handle adverse environments such as strong light and backlight. The three types of sensors cooperate with each other and complement each other. The optimization algorithm installed in the device chip enables the device to achieve accurate positioning of cables even in adverse operating environments such as strong light, backlight, and complex spatial environment.

[0041] 2. High precision, the error can be controlled below 10mm;

[0042] 3. Low power consumption, small size and light weight;

[0043] 4. The positioning process is not affected by environmental factors such as temperature, air pressure, dust, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0045] Figure 1 It is a schematic block diagram of a power cable identification and spatial positioning system of the present invention;

[0046] Figure 2 This is a schematic diagram of the image segmentation module of a method for identifying and spatially locating power cables according to the present invention;

[0047] Figure 3 The figure is a schematic diagram of the visual processing module of a method for identifying and spatially locating power cables according to the present invention.

[0048] In the figure: 1. Binocular camera; 2. LiDAR; 3. Depth camera; 4. AI chip; 5. Terminal; 6. Image segmentation module; 7. Stereo vision positioning module; 8. Visual processing module. DETAILED DESCRIPTION

[0049] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0050] See also Figure 1 , Figure 2 and Figure 3 The present invention provides a technical solution: a method for identifying and spatially locating a power cable, comprising the following steps:

[0051] S1: Detect the target cable, identify the cables in the environment, and obtain the cable point cloud data;

[0052] S2: Process the point cloud data through a visual processing program, and combine it with the cable identification data to perform fitting reconstruction and calculate the pose;

[0053] S3: The visual processing module 8 sends the final processing result to the terminal.

[0054] In this embodiment, the step S1 includes the following steps:

[0055] S1.1: Identify cables and stripping heads in the environment through binocular camera 1;

[0056] S1.2: Laser radar 2 detects the target by emitting laser beams, and collects the reflected beams to form a point cloud and obtain data:

[0057] S1.3: Depth camera 3 uses its own infrared projector to project a light spot to help improve the measurement accuracy of the binocular camera and form point cloud data.

[0058] In this embodiment, in step S1.1, the binocular camera 1 completes the recognition of cables and stripping heads in the environment through the image segmentation module 6 and the stereoscopic vision positioning module 7, wherein the image segmentation module 6 includes a cable recognition network submodule and a feature extraction network submodule. The specific implementation steps of step S1.1 are as follows:

[0059] S1.11: First, the cable image is collected by the calibrated binocular camera 1, and then the obtained left and right images of the cable are respectively input into the cable recognition network submodule for recognition;

[0060] S1.12: Cable identification network submodule, using Mask R-CNN as the main network structure, firstly extracts features from the input image through a convolutional neural network to obtain a feature layer, and then feeds the feature layer into a trained region proposal network to obtain a proposal box, and then extracts on the feature layer through the proposal box. The proposal box obtained here is a rough extraction, and then the final proposal box is obtained through further parameter adjustment, and the proposal boxes of different sizes are resized through ROI Align. Finally, these resized proposal boxes are classified, regressed, and masked to obtain left and right mask images of the cable, and the obtained left and right mask images are input into the feature extraction network submodule for feature point extraction;

[0061] S1.13: The feature extraction network submodule adopts a centerline extraction algorithm based on edge contour and curve fitting. The algorithm extracts the center point set for the contour edge, and then performs secondary smoothing through curve fitting. First, the direction of the transmission line contour is analyzed. When the aspect ratio is much greater than 1, the direction of the transmission line is close to the vertical direction. When the aspect ratio is much less than 1, the direction of the transmission line is closer to the parallel direction. If the overall contour of the transmission line is close to the vertical direction, the contour is scanned row by row to obtain the center point set of the contour. The scanning process is as follows:

[0062] Assume that the outer contour point set of the transmission line is P1(u1,v1),P2(u2,v2),...P k (u k ,v k ), where the u axis points to the horizontal direction of the image, and the v axis points to the vertical direction of the image. Scan downwards row by row from the top of the circumscribed rectangle. Assume that the circumscribed rectangle of the contour has a total of m rows from the top to the bottom, and their ordinates are y1, y2..., y m, take yi as the ordinate of the candidate center point in turn, find the value equal to the ordinate from the contour point set, get several points that meet the conditions, perform deduplication processing on these points, set a distance threshold, and set the threshold as follows: divide a sphere in this Euclidean space, which contains all the above points, and then calculate the Euclidean distance between every two points in the sphere, and then take the arithmetic mean of the obtained distance to get the average distance of all points, and set 1 / 4 of the average distance as the threshold here; if the Euclidean distance between two points is less than the threshold, it means that the distance between the two points is very close, then delete one of them. After the processing is completed, generally two edge points can be obtained for one yi, and a center point can be obtained by averaging the two edge points, whose pixel coordinates are (x i ,y i ), after scanning all the row coordinates of the circumscribed rectangle, we can get a set of center points, recorded as Q1(x1,y1),Q2(x2,y2),...,Q n (x n ,y n ), then the relevant feature points of the left and right target cables are obtained. If the overall contour of the transmission line is close to the horizontal direction, the contour is scanned from left to right. The subsequent algorithm is similar to the vertical scanning, except that the horizontal and vertical coordinates are swapped.

[0063] S1.14: Stereoscopic vision positioning module 7, directly sparsely matching feature points in two two-dimensional circuit images;

[0064] S1.15: Generate disparity map;

[0065] S1.16: Calculate the depth based on the disparity, calculate the three-dimensional coordinates, and complete the positioning;

[0066] In the above process, the stereoscopic vision positioning module 7 and the calculation of the three-dimensional coordinates adopt the cable matching method based on the epipolar line constraint, and the center line point set is used for binocular matching and three-dimensional coordinate calculation through the epipolar line.

[0067] In this embodiment, in step S2, the visual processing module 8 includes a point cloud data processing module, a point cloud filtering module, a point cloud segmentation module, a straight line fitting module, a coordinate acquisition module and a posture calculation module. The specific implementation steps of step S2 are as follows:

[0068] S2.1 point cloud data processing module, reads the point cloud data received from three sensors;

[0069] S2.2 point cloud filtering module performs filtering operation on the point cloud. Point cloud filtering helps to remove noise points and outliers, making the point cloud smoother. The point cloud filtering method used is one or more of straight-through filtering, outlier filtering, voxel filtering, and radius filtering.

[0070] S2.3 point cloud segmentation module performs segmentation processing on point cloud data, and the point cloud segmentation processing adopts the Euclidean clustering segmentation method;

[0071] S2.4 straight line fitting module performs straight line fitting on the segmented point cloud data. The straight line fitting adopts RanSaC algorithm to reduce the influence of noise.

[0072] S2.5 coordinate acquisition module acquires coordinate data;

[0073] S2.6 The posture calculation module calculates the posture.

[0074] In this embodiment, in step S2, the laser radar 2 and the depth camera 3 can directly generate point cloud data, but the three-dimensional point coordinates obtained by the binocular camera 1 are sent to the visual processing module 8 through the socket interface, and the visual processing module 8 publishes it as point cloud data and unifies it into the point cloud processing flow program of the visual processing, wherein the point cloud data of the three-dimensional point coordinates does not need to be processed such as denoising and can be directly applied, that is, directly acquired by the coordinate acquisition module.

[0075] The present invention also proposes a power cable identification and spatial positioning system, which is applied in a power cable identification and spatial positioning method, and includes a binocular camera 1, a laser radar 2, a depth camera 3, an AI chip 4 and a terminal 5;

[0076] The laser radar 2 uses non-contact laser ranging technology, and its working principle is similar to that of a general radar system, which detects targets by emitting laser beams and forms point clouds and acquires data by collecting reflected beams;

[0077] The binocular camera 1 is used to perform high-precision recognition and measurement of the target, which mainly utilizes the direct difference in the lateral coordinates of the target point imaged in the left and right views, and then uses the triangulation principle to obtain depth data to form a three-dimensional point cloud;

[0078] The depth camera 3 adopts active binocular vision technology, which can provide infrared patterns in low-texture environments by transmitting light spots through infrared projectors, thereby improving binocular measurement accuracy;

[0079] The AI ​​chip 4 internally stores an image segmentation module 6, a stereoscopic vision positioning module 7 and a vision processing module 8. The AI ​​chip 4 adopts the Cambrian MLU200 architecture and the Linux operating system, with a computing power of 8TOPS to 16TOPS. The total of the system memory and AI memory reaches 8GB, and the storage capacity reaches 16GB EMMC, which can better support the operation of the algorithm.

[0080] In this embodiment, the depth camera 3 is a RealSense depth camera.

[0081] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

[0082] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for identifying and spatially locating a power cable, characterized in that: The following steps are involved: S1: Detect the target cable, identify the cables in the environment, and obtain the cable point cloud data; S2: Process the point cloud data through a visual processing program, and combine it with the cable identification data to perform fitting reconstruction and calculate the pose; S3: The visual processing module sends the final processing result to the terminal; The step S1 includes the following steps: S1.1: Use binocular cameras to identify cables and stripping heads in the environment; S1.2: The laser radar emits a laser beam to detect the target, and collects the reflected beam to form a point cloud and obtain data: S1.3: The depth camera uses its own infrared projector to project light spots to help improve the measurement accuracy of the binocular camera and form point cloud data; In step S1.1, the binocular camera completes the recognition of cables and stripping heads in the environment through an image segmentation module and a stereoscopic vision positioning module, wherein the image segmentation module includes a cable recognition network submodule and a feature extraction network submodule. The specific implementation steps of step S1.1 are as follows: S1.11: First, the cable image is collected by a calibrated binocular camera, and then the obtained left and right images of the cable are respectively input into the cable recognition network submodule for recognition; S1.12: The cable identification network submodule first extracts features from the input image through a convolutional neural network to obtain a feature layer, which is then fed into a trained region proposal network to obtain a proposal box. The proposal box is then used to extract features on the feature layer, and proposal boxes of different sizes are resized through ROI Align. Finally, these resized proposal boxes are classified, regressed, and masked to obtain left and right mask images of the cable, which are then input into the feature extraction network submodule for feature point extraction. S1.13: Feature extraction network submodule, first analyzes the direction of the transmission line contour. When the aspect ratio is much greater than 1, the direction of the transmission line is close to the vertical direction. When the aspect ratio is much less than 1, the direction of the transmission line is closer to the parallel direction. If the overall contour of the transmission line is close to the vertical direction, the contour is scanned row by row to obtain the center point set of the contour. The scanning process is as follows: Assume that the outer contour point set of the transmission line is P1(u1,v1),P2(u2,v2),...P k (u k ,v k ), where the u axis points to the horizontal direction of the image, and the v axis points to the vertical direction of the image. Scan downwards row by row from the top of the circumscribed rectangle. Assume that there are m rows of circumscribed rectangles from the top to the bottom, and their ordinates are y 1 ,y 2 ...,y m , take yi as the ordinate of the candidate center point in turn, find the value equal to the ordinate from the contour point set, get several points that meet the conditions, perform duplicate removal on these points, set a distance threshold, if the Euclidean distance between two points is less than the threshold, it means that the distance between the two points is very close, then delete one of them. After the processing is completed, in general, for a y i Two edge points can be obtained, and a center point can be obtained by averaging the two edge points. Its pixel coordinates are (x i ,y i ), after scanning all the row coordinates of the circumscribed rectangle, we can get a set of center points, recorded as Q1(x1,y1),Q2(x2,y2),...,Q n (x n ,y n ), then the relevant feature points of the left and right target cables are obtained; S1.14: Stereo vision positioning module, directly sparsely matches the feature points in two two-dimensional circuit images; S1.15: Generate disparity map; S1.16: Calculate the depth based on the disparity, calculate the three-dimensional coordinates, and complete the positioning.

2. A method for identifying and spatially locating a power cable according to claim 1, characterized in that: In step S2, the visual processing module includes a point cloud data processing module, a point cloud filtering module, a point cloud segmentation module, a straight line fitting module, a coordinate acquisition module and a posture calculation module. The specific implementation steps of step S2 are as follows: S2.1 point cloud data processing module, reads the point cloud data received from three sensors; S2.2 point cloud filtering module performs filtering operation on the point cloud. Point cloud filtering helps to remove noise points and outliers, making the point cloud smoother. The point cloud filtering method used is one or more of straight-through filtering, outlier filtering, voxel filtering, and radius filtering. S2.3 point cloud segmentation module performs segmentation processing on point cloud data, and the point cloud segmentation processing adopts the Euclidean clustering segmentation method; S2.4 straight line fitting module performs straight line fitting on the segmented point cloud data. The straight line fitting adopts RanSaC algorithm to reduce the influence of noise. S2.5 coordinate acquisition module acquires coordinate data; S2.6 The posture calculation module calculates the posture.

3. A method for identifying and spatially locating a power cable according to claim 2, characterized in that: In step S2, the laser radar and depth camera can directly generate point cloud data, but the three-dimensional point coordinates obtained by the binocular camera are sent to the visual processing module, which publishes them as point cloud data and unifies them into the point cloud processing flow program of the visual processing, wherein the point cloud data of the three-dimensional point coordinates does not need to be processed such as denoising and can be directly applied.

4. A power cable identification and spatial positioning system, applied in a power cable identification and spatial positioning method according to any one of claims 1 to 3, characterized in that: Including binocular cameras, lidar, depth cameras, AI chips and terminals; The laser radar uses non-contact laser ranging technology, and its working principle is similar to that of a general radar system, which detects targets by emitting laser beams and forms point clouds and acquires data by collecting reflected beams. The binocular camera is used to identify and measure the target with high precision, which mainly utilizes the direct difference in the lateral coordinates of the target point imaged in the left and right views, and then uses the triangulation principle to obtain depth data to form a three-dimensional point cloud; The depth camera adopts active binocular vision technology, and transmits light spots through an infrared projector to provide infrared patterns in low-texture environments to improve binocular measurement accuracy; The AI ​​chip internally stores an image segmentation module, a stereoscopic vision positioning module and a vision processing module.

Images