Cable tunnel multi-source inspection sensing method and system, medium and product

Abstract

The invention discloses a cable tunnel multi-source inspection sensing method and system, a medium and a product. The cable tunnel multi-source inspection sensing method comprises the following steps: acquiring a cable tunnel image acquired by a camera of an inspection robot and a point cloud acquired by a laser radar; performing image analysis on the cable tunnel image to obtain a 2D positioning rectangular frame of the cable equipment; the 2D positioning rectangular frame of the cable equipment is converted into a laser radar coordinate system to be associated with the point cloud data; and the 2D positioning rectangular frame of the cable equipment is converted to the point cloud in the area in the laser radar coordinate system, and a 3D target detection model which completes training in advance is utilized to carry out target detection of cable equipment defects. The invention aims to realize real-time monitoring and accurate sensing of a cable tunnel environment by fusing various sensors and an image recognition technology, reduce the labor intensity of inspection personnel and improve the inspection efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of cable tunnel inspection, and in particular to a cable tunnel multi-source inspection sensing method, system, medium and product. Background Art

[0002] As an important part of the safe operation and management of power grids, cable tunnel inspection has become a very important task. In recent years, with the continuous development of technologies such as artificial intelligence, big data, and computer vision, it has become possible to inspect cable tunnels through intelligent robots. Intelligent inspection robots for cable tunnels can improve the level of modern management of cable networks, effectively reduce the workload of inspection personnel, reduce safety hazards and accident hazards, and avoid cable accidents, thereby greatly improving the modernization and intelligence of cable management and achieving refined monitoring. As an important facility for power transmission, the safety and stability of cable tunnels are crucial to power supply. However, the traditional cable tunnel inspection method has problems such as long inspection cycle, low inspection efficiency, and heavy workload of inspection personnel, which makes it difficult to meet the safety, efficiency, and intelligence requirements of modern power production. Summary of the invention

[0003] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a multi-source inspection and perception method, system, medium and product for a cable tunnel are provided. The present invention aims to achieve real-time monitoring and accurate perception of the cable tunnel environment, reduce the labor intensity of inspection personnel and improve inspection efficiency by integrating multiple sensors and image recognition technologies.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: A cable tunnel multi-source inspection and sensing method comprises the following steps: S1, obtain the cable tunnel image collected by the inspection robot's camera and the point cloud collected by the laser radar; S2, performing image analysis on the cable tunnel image to obtain a 2D positioning rectangular frame of the cable equipment; S3, converting the 2D positioning rectangle of the cable equipment into the laser radar coordinate system and associating it with the point cloud data; S4, converting the 2D positioning rectangle of the cable equipment into a point cloud in the area in the laser radar coordinate system, and performing target detection of cable equipment defects using a pre-trained 3D target detection model.

[0005] Optionally, step S2 includes: S2.1, the ORB algorithm combined with the BRIEF operator is used to extract and describe feature points of the cable tunnel image; S2.2, fitting the extracted feature points using the least square fitting method to extract the line segment features of the cable; S2.3, locate the 2D positioning rectangle of the cable according to the line segment features of the cable.

[0006] Optionally, step S3 includes: S3.1, transforming the 2D positioning rectangle into the camera coordinate system of the camera through the pre-calibrated camera intrinsic parameters and extrinsic parameters; S3.2, using the external reference relationship between the lidar and the camera, the 2D positioning rectangle in the camera coordinate system is converted into the lidar coordinate system, thereby achieving association with the point cloud data.

[0007] Optionally, step S4 includes: S4.1, converting the 2D positioning rectangle of the cable equipment into a point cloud in the region in the laser radar coordinate system, and using the RANSAC algorithm to perform outlier detection and remove outliers; S4.2, using a pre-trained 3D target detection model to perform target detection of cable equipment defects on the point cloud after outliers are removed, wherein the 3D target detection model is a PointNet network, a PointNet++ network, an F-PointNet network, a VoxelNet network, a PointPillars network, a SECOND network, or a PV-RCNN network.

[0008] Optionally, in step S4.2, the 2D positioning rectangular box of the cable equipment is converted into a point cloud in the area in the laser radar coordinate system, and the target detection of the cable equipment defects using the pre-trained 3D target detection model includes training the 3D target detection model, and the training of the 3D target detection model includes maximizing the likelihood function of the fault probability density function to obtain the optimal model parameters of the 3D target detection model, and the function expression of the likelihood function of the fault probability density function is: , in, is the likelihood function of the failure probability density function, They represent the failure probability, the set failure rate statistical period and the constant parameter respectively. is the total number of data in the dataset, is the total number of fault data in the data set, For in time and status The probability density of failure is For in time and status The probability density of no failure under the condition, and the calculation function expression of the failure probability is: , in, is the failure probability, is the number of cable equipment failures, It is the set failure rate statistical period.

[0009] Optionally, step S4 also includes: submitting target detection results obtained by performing target detection of cable equipment defects to an intelligent monitoring platform, and monitoring, real-time display and alarming the target detection results through the intelligent monitoring platform.

[0010] Optionally, the monitoring, real-time display and alarm of the target detection results by the intelligent monitoring platform include calculating the failure rate of each cable device according to the following formula, displaying the failure probability and alarming when exceeding a preset threshold: , in, is the failure probability, is the number of cable equipment failures, It is the set failure rate statistical period.

[0011] In addition, the present invention also provides a cable tunnel multi-source inspection and perception system, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the cable tunnel multi-source inspection and perception method.

[0012] In addition, the present invention also provides a computer-readable storage medium, in which a computer program or instruction is stored. The computer program or instruction is programmed or configured to execute the cable tunnel multi-source inspection and perception method through a processor.

[0013] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the cable tunnel multi-source inspection and perception method through a processor.

[0014] Compared with the prior art, the present invention mainly has the following advantages: the cable tunnel multi-source inspection perception method of the present invention realizes real-time monitoring and accurate perception of the cable tunnel environment by integrating multiple sensors and image recognition technologies, significantly improving the inspection efficiency; the inspection robot autonomously navigates and inspects, which reduces the labor intensity of the inspection personnel and improves the inspection safety; through real-time monitoring and alarm functions, the safety hazards in the cable tunnel are discovered in time, and the level of power production safety is improved; the intelligent monitoring platform realizes real-time display and analysis of data, providing intelligent support for power production management. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of the basic flow of the method of the embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0017] like Figure 1 As shown, this embodiment provides a cable tunnel multi-source inspection and perception method, including the following steps: S1, obtain the cable tunnel image collected by the inspection robot's camera and the point cloud collected by the laser radar; S2, performing image analysis on the cable tunnel image to obtain a 2D positioning rectangular frame of the cable equipment (including cables and equipment); S3, converting the 2D positioning rectangle of the cable equipment into the laser radar coordinate system and associating it with the point cloud data; S4, converting the 2D positioning rectangle of the cable equipment into a point cloud in the area in the laser radar coordinate system, and performing target detection of cable equipment defects using a pre-trained 3D target detection model.

[0018] The inspection robot in this embodiment is an inspection machine with autonomous navigation capabilities, equipped with high-definition cameras and multiple detection sensors to achieve all-round monitoring of the internal environment of the cable tunnel and the status of the equipment. The inspection robot can collect and store the status data of the cable line and its channel in real time through a non-contact data sending component. The inspection robot can not only receive the preset inspection plan, but also assist in the execution of inspection tasks, including recording defects and potential hidden dangers of equipment and facilities. In addition, it can also perform regular inspection tasks, as well as differentiated inspections for different areas, especially focusing on monitoring and reviewing those areas identified as high-risk risk points. Step S1 obtains the cable tunnel image captured by the camera of the inspection robot and the point cloud captured by the laser radar, and the subject of steps S2 to S4 can be either the inspection robot itself or a computer device that establishes a communication connection with the inspection robot.

[0019] Step S2 is used to analyze the image information collected by the intelligent inspection module and extract high-level semantic features to achieve accurate identification and positioning of targets in the cable tunnel. In this embodiment, step S2 includes: S2.1, the ORB algorithm combined with the BRIEF operator is used to extract and describe feature points of the cable tunnel image; S2.2, fitting the extracted feature points using the least square fitting method to extract the line segment features of the cable; S2.3, locate the 2D positioning rectangle of the cable according to the line segment features of the cable.

[0020] In step S2.1, the ORB algorithm combined with the BRIEF operator to extract and describe feature points is an existing method, including the feature points extracted by the ORB algorithm are depicted using the BRIEF operator. The depiction process is to randomly select an image block, which must be composed of some points around the key point, and then binarize the image block, and then perform binary expression conversion on the image to form the final feature descriptor. Then, the method used for the line segment feature in step S2.2 is the least squares fitting method, and the 2D positioning rectangle of the cable is located according to the line segment feature of the cable in step S2.3.

[0021] In addition, as an optional implementation, before performing image analysis on the cable tunnel image in step S2 of this embodiment to obtain the 2D positioning rectangular frame of the cable equipment (including cables and equipment), it also includes using adaptive threshold adjustment technology for the cable tunnel image to automatically optimize the threshold setting according to the ambient lighting conditions and image content to achieve image binarization processing to ensure stable feature detection under different lighting and complex backgrounds. In addition, images of different scales can be generated for the cable tunnel image, and multi-scale feature fusion can be performed to enhance the accuracy of target recognition. In addition, according to the need to focus on real-time performance, a more efficient algorithm can be selected to ensure high-accuracy real-time processing capabilities to meet the strict speed requirements of the inspection robot. Finally, by introducing advanced image processing technologies such as noise suppression and edge enhancement, the robustness of the system can be significantly improved, enabling it to extract clear feature points in low-quality images.

[0022] It should be pointed out that a point cloud is composed of a large number of points, each of which contains spatial position (X, Y, Z) information, and sometimes also includes attributes such as color and reflectivity. In the field of computer vision and 3D reconstruction, point cloud is a commonly used representation. Point cloud fusion is to merge these scattered point cloud data into a more complete and accurate point cloud data set for subsequent 3D modeling, environmental perception and other tasks. In this embodiment, step S3 is used for point cloud fusion, which specifically includes: S3.1, transforming the 2D positioning rectangle into the camera coordinate system of the camera through the pre-calibrated camera intrinsic parameters and extrinsic parameters; S3.2, using the external reference relationship between the lidar and the camera to transform the 2D positioning rectangle in the camera coordinate system into the lidar coordinate system, thereby achieving association with the point cloud data, thereby generating fused point cloud information containing local color coding to ensure the precise correspondence between the point cloud data and the 2D positioning information, so as to achieve all-round perception of the cable tunnel environment.

[0023] In this embodiment, step S4 includes: S4.1, convert the 2D positioning rectangle of the cable equipment into the point cloud in the area in the laser radar coordinate system, and use the RANSAC algorithm to detect outliers and remove outliers; in point cloud fusion, the input data is usually multiple point cloud data sets. These point cloud data sets may come from different sensors, different scanning angles or different time points. Each point cloud data set contains a series of three-dimensional points, which have spatial coordinate (X, Y, Z) information, and sometimes also include attributes such as color and reflectivity. These point cloud data sets may have certain overlapping areas, and may also contain noise and redundant points. By using the RANSAC algorithm to detect outliers and remove outliers in step S4.1, the accuracy of target detection of cable equipment defects using the pre-trained 3D target detection model can be improved; it should be noted that the RANSAC algorithm is an existing outlier detection algorithm, so its implementation details will not be described in detail here; S4.2, the point cloud after removing the outliers is used to detect the defects of the cable equipment using a pre-trained 3D target detection model, wherein the 3D target detection model is a PointNet network, a PointNet++ network, an F-PointNet network, a VoxelNet network, a PointPillars network, a SECOND network or a PV-RCNN network. For example, as an optional implementation, the 3D target detection model in this embodiment is a PointNet network. Since the above 3D target detection models are all existing known network models, their implementation details are not described in detail here.

[0024] In order to ensure that the inspection data can be analyzed in time and intelligent feedback can be generated. In step S4.2 of this embodiment, the 2D positioning rectangle of the cable equipment is converted to the point cloud in the area in the laser radar coordinate system. The target detection of cable equipment defects using the pre-trained 3D target detection model includes training the 3D target detection model, and the training of the 3D target detection model includes maximizing the likelihood function of the fault probability density function to obtain the optimal model parameters of the 3D target detection model, and the function expression of the likelihood function of the fault probability density function is: , in, is the likelihood function of the failure probability density function, They represent the failure probability, the set failure rate statistical period and the constant parameter respectively. is the total number of data in the dataset, is the total number of fault data in the data set, For in time and status The probability density of failure is For in time and status The probability density of no failure under the condition, and the calculation function expression of the failure probability is: , in, is the failure probability, is the number of cable equipment failures, The above-mentioned fault detection algorithm based on probability calculation is used for fault detection and monitoring in the inspection robot system. This method can significantly improve the accuracy of fault detection, reduce false alarms and missed alarms, and thus improve the reliability and maintenance efficiency of the entire system through an accurate mathematical model.

[0025] In this embodiment, after step S4, it also includes: submitting the target detection results obtained by the target detection of cable equipment defects to the intelligent monitoring platform, and monitoring, real-time display and alarming the target detection results through the intelligent monitoring platform. The intelligent monitoring platform is used to receive and display the data collected by the inspection robot, and realize the functions of real-time monitoring, alarm and report generation; in this embodiment, the intelligent monitoring platform includes a background diagnosis module, which combines the relevant components in the fields of cable insulation aging, comprehensive status diagnosis, remaining life assessment and related intelligent identification and analysis technology at home and abroad, and provides comprehensive full-process cable status detection and evaluation data of comprehensive status perception, offline diagnosis and material performance analysis, and compares, evaluates and records the data of each inspection.

[0026] In this embodiment, when monitoring, real-time display and alarming are performed on the target detection results through the intelligent monitoring platform, the failure rate of each cable device is calculated according to the following formula, the failure probability is displayed, and an alarm is issued when the preset threshold is exceeded: , in, is the failure probability, is the number of cable equipment failures, is the set failure rate statistical period. The core of this method is that when the failure probability density function increases significantly at a certain point in time, it can indicate that the device or system may be about to fail. At this time, the error detection alarm system of the method of this embodiment will automatically send out an early warning signal to remind relevant personnel to conduct inspections and repairs in time, thereby avoiding the occurrence of failures or reducing their impact.

[0027] To sum up, the multi-source inspection and perception method for cable tunnels in this embodiment completes the perception and detection process through data collection, image analysis, point cloud fusion, target detection, data display, and background recording. By integrating multiple sensors and image recognition technologies, real-time monitoring and accurate perception of the cable tunnel environment are achieved, which significantly improves the inspection efficiency. The inspection robot autonomously navigates and inspects, which reduces the labor intensity of inspection personnel and improves the inspection safety. Through real-time monitoring and alarm functions, safety hazards in cable tunnels can be discovered in time, and the level of power production safety can be improved. The intelligent monitoring platform realizes real-time display and analysis of data, providing intelligent support for power production management.

[0028] In addition, this embodiment also provides a cable tunnel multi-source inspection and perception system, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the cable tunnel multi-source inspection and perception method.

[0029] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program or instruction is stored. The computer program or instruction is programmed or configured to execute the cable tunnel multi-source inspection and perception method through a processor.

[0030] In addition, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the cable tunnel multi-source inspection and perception method through a processor.

[0031] Those skilled in the art should understand that the technical solutions provided by the embodiments of the present application may be in the form of methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0032] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A cable tunnel multi-source inspection and sensing method, characterized in that: The steps include: S1, obtain the cable tunnel image captured by the inspection robot's camera and the point cloud collected by the laser radar; S2, performing image analysis on the cable tunnel image to obtain a 2D positioning rectangular frame of the cable equipment; S3, converting the 2D positioning rectangle of the cable equipment into the laser radar coordinate system and associating it with the point cloud data; S4, converting the 2D positioning rectangle of the cable equipment into a point cloud in the area in the laser radar coordinate system, and performing target detection of cable equipment defects using a pre-trained 3D target detection model.

2. The cable tunnel multi-source inspection and sensing method according to claim 1 is characterized in that: Step S2 includes: S2.1, the ORB algorithm combined with the BRIEF operator is used to extract and describe feature points of the cable tunnel image; S2.2, fitting the extracted feature points using the least square fitting method to extract the line segment features of the cable; S2.3, locate the 2D positioning rectangle of the cable according to the line segment features of the cable.

3. The cable tunnel multi-source inspection and sensing method according to claim 1 is characterized in that: Step S3 includes: S3.1, transforming the 2D positioning rectangle into the camera coordinate system of the camera through the pre-calibrated camera intrinsic parameters and extrinsic parameters; S3.2, using the external reference relationship between the lidar and the camera, the 2D positioning rectangle in the camera coordinate system is converted into the lidar coordinate system, thereby achieving association with the point cloud data.

4. The cable tunnel multi-source inspection and sensing method according to claim 1 is characterized in that: Step S4 includes: S4.1, converting the 2D positioning rectangle of the cable equipment into a point cloud in the region in the laser radar coordinate system, and using the RANSAC algorithm to perform outlier detection and remove outliers; S4.2, using a pre-trained 3D target detection model to perform target detection of cable equipment defects on the point cloud after outliers are removed, wherein the 3D target detection model is a PointNet network, a PointNet++ network, an F-PointNet network, a VoxelNet network, a PointPillars network, a SECOND network, or a PV-RCNN network.

5. The cable tunnel multi-source inspection and sensing method according to claim 4 is characterized in that: In step S4.2, the 2D positioning rectangular box of the cable equipment is converted into a point cloud in the area in the laser radar coordinate system. Before the target detection of the cable equipment defects is performed using the pre-trained 3D target detection model, the 3D target detection model is trained, and the training of the 3D target detection model includes maximizing the likelihood function of the fault probability density function to obtain the optimal model parameters of the 3D target detection model, and the function expression of the likelihood function of the fault probability density function is: ， in, is the likelihood function of the failure probability density function, They represent the failure probability, the set failure rate statistical period and the constant parameter respectively. is the total number of data in the dataset, is the total number of fault data in the data set, For in time and status The probability density of failure is For in time and status The probability density of no failure under the condition, and the calculation function expression of the failure probability is: ， in, is the failure probability, is the number of cable equipment failures, It is the set failure rate statistical period.

6. The cable tunnel multi-source inspection and sensing method according to any one of claims 1 to 5, characterized in that: After step S4, it also includes: submitting the target detection results obtained by the target detection of cable equipment defects to the intelligent monitoring platform, and monitoring, real-time display and alarming the target detection results through the intelligent monitoring platform.

7. The cable tunnel multi-source inspection and sensing method according to claim 5 is characterized in that: The monitoring, real-time display and alarm of the target detection results by the intelligent monitoring platform include calculating the failure rate of each cable device according to the following formula and displaying the failure probability and alarming when exceeding the preset threshold: ， in, is the failure probability, is the number of cable equipment failures, It is the set failure rate statistical period.

8. A cable tunnel multi-source inspection and sensing system, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the cable tunnel multi-source inspection and sensing method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the cable tunnel multi-source inspection and perception method described in any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the cable tunnel multi-source inspection and perception method described in any one of claims 1 to 7 through a processor.

Images