Method and System for Processing Cable Well Image Data Based on SLAM Device

Abstract

The present invention relates to the technical field of image data processing, and more particularly to a method and system for processing cable well image data based on a slam device. The method includes: using the slam device to perform a 360-degree rotation scan on the cable well to obtain three-dimensional point cloud data and a first image of the cable well, using a first algorithm to perform preliminary cropping on the three-dimensional point cloud data, and using a second algorithm to perform automatic denoising on the three-dimensional point cloud data; obtaining a geometric body that completely encloses the three-dimensional point cloud data as a bounding box and obtaining the measurement result of the cable well; collecting images in historical cable wells and using them as input data to train a corresponding learning model, and the learning model outputs a first image with annotation information; based on the measurement result of the cable well and the first image with annotation information that is consistent with the time of the cable well measurement result, constructing a three-dimensional model of the cable well. The present invention can improve the efficiency and safety of cable well surveying and mapping, and can also provide strong support for the digital management of underground pipe trenches.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and more particularly to a method and system for processing cable well image data based on a SLAM device. Background Art

[0002] With the acceleration of the urbanization process, the management and maintenance of underground cables and pipe trenches have become increasingly important. The workload of the province-wide distribution network cable census is huge. The existing technical means mainly rely on manual cable surveying and mapping in the well. Due to the complex underground environment, there may also be dangers such as toxic gases and collapses, which pose a serious threat to the life safety of the operators. On the other hand, the cable census is implemented by different surveying and mapping units, and the census results are often stored offline, with inconsistent data formats and a lack of a unified data management platform, resulting in difficulties in data integration and analysis. Summary of the Invention

[0003] Aiming at the disadvantages of the prior art, the present invention provides a method and system for processing cable well image data based on a SLAM device, aiming to provide a safe, efficient, and automated cable well surveying and mapping solution, which can improve the efficiency and safety of cable well surveying and mapping and also provide strong support for the digital management of underground pipe trenches.

[0004] In order to achieve the above invention purpose, a method for processing cable well image data based on a SLAM device is realized by performing the following method:

[0005] Step S1: Use a SLAM device to perform a 360-degree rotational scan of the cable well to obtain the three-dimensional point cloud data and the first image of the cable well, and establish a correspondence between the three-dimensional point cloud data and the first image according to the shooting time, and transmit it to the control unit in real time through a wireless transmission module;

[0006] Step S2: In the control unit, use a first algorithm to perform a preliminary cut on the three-dimensional point cloud data to remove the redundant point cloud outside the cable well, and use a second algorithm to perform automated denoising on the three-dimensional point cloud data to identify and remove the noise points and outliers in the three-dimensional point cloud data;

[0007] Step S3: Obtain a geometric body that completely encloses the three-dimensional point cloud data as a bounding box, calculate the size of the bounding box, and obtain the cable well measurement result;

[0008] Step S4: Collect the images in the historical cable wells and use them as input data to train the corresponding learning model, and input the first image into the trained learning model, and the learning model outputs the first image with annotation information;

[0009] Step S5: Based on the measurement results of the cable well and the first image with annotated information that is consistent with the time of the measurement results of the cable well, a three-dimensional model of the cable well is constructed. After the three-dimensional model of the cable well is constructed, the three-dimensional model of the cable well is stored, and a panoramic viewing function of the cable well is provided to the staff.

[0010] As a preferred technical solution of the present invention, step S4 further includes the following steps:

[0011] Step S41: collecting historical images of the cable well as second images, wherein the second images include images containing pipe holes, images of well body collapse, images of cable breakage, and images of water accumulation in the well. The images containing pipe holes also include images of occupied and unoccupied pipe holes.

[0012] Step S42: creating and training a first learning model to classify and label the occupancy status of the pipe holes in the second image, and identifying whether the cable in the pipe hole is broken when the pipe hole is marked as occupied;

[0013] Step S43: Create and train a second learning model to identify and label whether the second image contains features of well collapse, and create and train a third learning model to identify and label whether the second image contains features of water accumulation in the well.

[0014] As a preferred technical solution of the present invention, step S42 further includes:

[0015] Creating a first learning model based on the deep learning model, the first learning model having a first output branch and a second output branch, the first output branch being used to classify and label the occupancy status of the pipe holes in the second image, and the second output branch being used to identify whether the second image contains a broken cable and label the identification result;

[0016] The second image is input into the first learning model. The first layer of the first learning model extracts and identifies the pipe holes and the pipe hole positions in the second image, and uses the pipe hole positions as the output result of the first layer. Based on the pipe hole positions, the second layer of the first learning model identifies whether the pipe hole positions are occupied. The occupancy status of the pipe hole positions is classified and labeled and used as the output result of the first output branch. If the output result of the first output branch is that the pipe hole is occupied, the third layer of the first learning model identifies whether the cable in the occupied pipe hole is broken, labels the identification result, and uses it as the output result of the second output branch.

[0017] As a preferred technical solution of the present invention, step S43 further includes:

[0018] Step S431: Calculate the number of images in the second image that do not contain the wellbore collapse feature image as the first quantity, and the number of images in the second image that contain the wellbore collapse feature image as the second quantity. Compare the first quantity and the second quantity. When the first quantity is N times or more of the second quantity, based on the images without the wellbore collapse feature and the images with the wellbore collapse feature, use data augmentation technology to generate virtual images with wellbore collapse features as virtual images. Use the virtual images and the second image together as the input data of the second learning model to train the second learning model, and use the annotation result of the wellbore collapse as the output result of the second learning model, where N represents a positive integer greater than or equal to 3;

[0019] Step S432: Repeat the step S431 to train the third learning model.

[0020] As a preferred technical solution of the present invention, the step S3 further includes:

[0021] Find the points with the maximum and minimum coordinates along the X-axis, the points with the maximum and minimum coordinates along the Y-axis, and the points with the maximum and minimum coordinates along the Z-axis in the three-dimensional point cloud data as the 8 vertices of the bounding box;

[0022] Construct an axis-aligned bounding box based on the 8 vertices of the bounding box, where each side of the axis-aligned bounding box is parallel to the coordinate axis;

[0023] Calculate the difference between the maximum coordinate value and the minimum coordinate value of the X-axis as the length of the bounding box, calculate the difference between the maximum coordinate value and the minimum coordinate value of the Y-axis as the width of the bounding box, calculate the difference between the maximum coordinate value and the minimum coordinate value of the Z-axis as the height of the bounding box. Based on the length, width and height of the bounding box, construct the bounding box, and use the size of the bounding box as the measurement result of the cable well.

[0024] As a preferred technical solution of the present invention, after constructing the bounding box, it further includes:

[0025] Convert the axis-aligned bounding box into an oriented bounding box, apply the principal component analysis method to calculate the main axis of the three-dimensional point cloud data and use it as the axis vector of the oriented bounding box, and also calculate the centroid of the three-dimensional point cloud data as the center point of the oriented bounding box. For each axis vector, calculate the maximum value of the perpendicular distance from all points in the three-dimensional point cloud data to the first axis, and use it as the semi-axis length of the first axis;

[0026] Use the gradient descent algorithm to fine-tune the center point, the axis vector and the semi-axis length of the oriented bounding box to obtain the minimum volume of the oriented bounding box.

[0027] As a preferred technical solution of the present invention, a cable shaft is scanned 360 degrees using a SLAM device, including:

[0028] The SLAM device is safely sent into the cable shaft through a robotic arm, and scanning is started during the descent of the SLAM device to obtain environmental images taken in real time, and the speed and direction of the descent of the SLAM device are adjusted in a timely manner based on the environmental images to ensure that the SLAM device does not collide with the well wall or other structural objects;

[0029] After the SLAM device descends to the bottom of the cable shaft, initial positioning and attitude adjustment are performed, the rotation mechanism of the SLAM device is turned on, and at the same time, the lidar and the camera are activated to start a full - range scan.

[0030] As a preferred technical solution of the present invention, obtaining three - dimensional point cloud data and a first image of the cable shaft includes:

[0031] The lidar emits laser pulses at a preset frequency, measures the time when the laser pulses are reflected back, obtains the distance information of the structure inside the cable shaft, and generates three - dimensional point cloud data. The camera captures the first image during the rotation of the SLAM device and records the video information inside the cable shaft.

[0032] As a preferred technical solution of the present invention, the step S2 further includes:

[0033] The first algorithm is to use CloudCompare software to import the three - dimensional point cloud data, create a cube based on the cropping area, define the corner coordinates of the cube, and crop the three - dimensional point cloud data based on the cube to obtain the cropped three - dimensional point cloud data;

[0034] The second algorithm is the RANSAC algorithm. Set the parameters of the RANSAC algorithm and execute the RANSAC algorithm to denoise and filter outliers from the three - dimensional point cloud data. The parameters of the RANSAC algorithm include the number of iterations, model type, distance threshold, and confidence level.

[0035] The present invention also provides a processing system for cable shaft image data based on a SLAM device as described above, including the following modules:

[0036] A data acquisition unit for scanning the cable shaft 360 degrees using a SLAM device, obtaining three - dimensional point cloud data and a first image of the cable shaft, establishing a correspondence between the three - dimensional point cloud data and the first image according to the shooting time, and transmitting it to the control unit in real time through a wireless transmission module;

[0037] A data preprocessing unit is configured to initially crop the three-dimensional point cloud data in the control unit using a first algorithm to remove redundant point clouds outside the cable shaft, and perform automatic denoising on the three-dimensional point cloud data using a second algorithm to identify and remove noise points and outliers in the three-dimensional point cloud data;

[0038] A data measurement unit is configured to obtain a geometric body that completely encloses the three-dimensional point cloud data as a bounding box, calculate the size of the bounding box, and obtain the measurement result of the cable shaft;

[0039] An image annotation unit is configured to collect images in historical cable shafts and use them as input data to train a corresponding learning model, input the first image into the trained learning model, and the learning model outputs the first image with annotation information;

[0040] A three-dimensional modeling unit is configured to construct a three-dimensional model of the cable shaft based on the measurement result of the cable shaft and the first image with annotation information that is consistent with the time of the measurement result of the cable shaft. After the three-dimensional model of the cable shaft is constructed, store the three-dimensional model of the cable shaft, and also provide a panoramic viewing function of the cable shaft to the staff.

[0041] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0042] The technical solution of the present invention obtains the three-dimensional point cloud data and the first image of the cable shaft by using a slam device to perform a 360-degree rotation scan on the cable shaft, and establishes the correspondence between the three-dimensional point cloud data and the first image according to the shooting time, and transmits it to the control unit in real time through a wireless transmission module. It realizes the rapid and comprehensive scanning of the internal environment of the cable shaft, eliminates the need for manual entry into the shaft, and improves the safety of the operation; in the control unit, a first algorithm is used to preliminarily crop the three-dimensional point cloud data to remove the redundant point cloud outside the cable shaft, and a second algorithm is used to automatically denoise the three-dimensional point cloud data to identify and remove the noise points and outliers in the three-dimensional point cloud data, providing a high-quality data basis for subsequent measurement and modeling. A geometric body that completely encloses the three-dimensional point cloud data is obtained as a bounding box, the size of the bounding box is calculated, and the measurement result of the cable shaft is obtained, improving the speed and accuracy of the measurement. Images in historical cable shafts are collected and used as input data to train the corresponding learning model, and the first image is input into the trained learning model, and the learning model outputs the first image with annotation information; it provides basic data for constructing the three-dimensional model of the cable shaft. Based on the measurement result of the cable shaft and the first image with annotation information that is consistent with the time of the cable shaft measurement result, a three-dimensional model of the cable shaft is constructed, and after the three-dimensional model of the cable shaft is constructed, the three-dimensional model of the cable shaft is stored, and a panoramic viewing function of the cable shaft is also provided to the staff, so that the staff can quickly understand the details of the cable shaft without being on-site, improving the work efficiency and safety. Through the cooperation between the above steps, the present invention can improve the efficiency and safety of cable shaft surveying and mapping, and can also provide strong support for the digital management of underground pipe trenches. Brief Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0044] Figure 1 It is the flowchart of the steps of the method for processing cable shaft image data based on a slam device in the present invention;

[0045] Figure 2 It is the composition structure diagram of the system for processing cable shaft image data based on a slam device in the present invention. Detailed Embodiments

[0046] In order to make the purpose, technical solution and advantages of the present invention clearer, the following further details the present invention in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] It will be understood that the terms "first", "second", etc. used in the present application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of the present application, the first xx script may be referred to as the second xx script, and similarly, the second xx script may be referred to as the first xx script.

[0048] The workload of the province-wide distribution network cable census is huge. The existing technical means mainly rely on manual downhole cable surveying and mapping. Due to the complex underground environment, there may also be dangers such as toxic gases and collapses, which pose a serious threat to the life safety of the operators; on the other hand, the cable census is implemented by different surveying and mapping units, and the census results are often stored offline, with inconsistent data formats and a lack of a unified data management platform, resulting in difficulties in data integration and analysis.

[0049] In view of the above technical problems, the present invention proposes a Figure 1 processing method for cable well image data based on a slam device as shown in

[0050] Step S1: Use a slam device to perform a 360-degree rotation scan on the cable well to obtain three-dimensional point cloud data and a first image of the cable well, establish a correspondence between the three-dimensional point cloud data and the first image according to the shooting time, and transmit it to the control unit in real time through a wireless transmission module.

[0051] Specifically, SLAM (Simultaneous Localization and Mapping), that is, simultaneous localization and mapping, is a technology that enables a robot to autonomously navigate in an unknown environment and build a map of the environment. Before going down the well, check whether the SLAM device is in good condition, including whether sensors such as lidar, panoramic camera, and inertial measurement unit are working properly, and set parameters such as the resolution of the camera and the frame rate of the video according to the specific environment and requirements of the cable well to ensure that the acquired data meets the needs of subsequent processing. Send the SLAM device into the cable well through a rope or robotic arm. The SLAM device automatically rotates 360 degrees for scanning underground. The lidar and panoramic camera work simultaneously. The lidar obtains point cloud data in space, and the panoramic camera obtains the first image data. The lidar scans the inside of the cable well at a high frequency to generate a large amount of point cloud data, which contains the spatial position information of the well structure; the panoramic camera takes the first image to record the visual information in the well and provides data support for subsequent image recognition and analysis. The SLAM device is usually equipped with a wireless transmission module, which can transmit the acquired point cloud data and the first image to the control center in real time for subsequent processing. Through the above steps, a rapid and comprehensive scan of the internal environment of the cable well is achieved, eliminating the need for manual entry into the well, improving the safety of the operation, and enabling the rapid processing of image data in real time, thus improving work efficiency.

[0052] Step S2: In the control unit, use the first algorithm to preliminarily crop the three-dimensional point cloud data to remove the redundant point cloud outside the cable well, and use the second algorithm to automatically denoise the three-dimensional point cloud data to identify and remove the noise points and outliers in the three-dimensional point cloud data.

[0053] Specifically, in the control unit, use the first algorithm, namely the CloudCompare algorithm, to preliminarily crop the received three-dimensional point cloud data to remove the redundant point cloud outside the cable well. Then, use the second algorithm, namely the RANSAC algorithm, to automatically denoise the point cloud data to identify and remove the noise points and outliers. The RANSAC algorithm improves the accuracy and robustness of the data, especially in the presence of noise and outliers. By cropping and denoising the three-dimensional point cloud data, the point cloud data becomes more accurate, providing a high-quality data basis for subsequent measurement and modeling.

[0054] Step S3: Obtain the geometric body that completely encloses the three-dimensional point cloud data as the bounding box, calculate the size of the bounding box, and obtain the measurement result of the cable well.

[0055] Specifically, the bounding box calculation method is adopted to determine a geometric body that can completely enclose the point cloud data, such as a cuboid. By calculating the maximum and minimum coordinate values of the geometric body along each coordinate axis in the three-dimensional space, the dimensions (length, width, height) of the bounding box are determined, thereby obtaining the measurement results of the cable shaft. The dimensions of the bounding box provide the accurate spatial dimensions of the cable shaft, while automating the measurement process of the cable shaft and improving the speed and accuracy of the measurement.

[0056] Step S4: Collect the images in the historical cable shafts and use them as input data to train the corresponding learning model. Input the first image into the trained learning model, and the learning model outputs the first image with annotation information.

[0057] Specifically, collect the images in the historical cable shafts and use these images as input data to train a deep learning model for identifying the occupancy of pipe holes and underground defects. Through the intelligent identification of the deep learning model, the annotation process of the occupancy status of pipe holes and underground defects is automated, improving the efficiency and accuracy of the identification. Input the first image into the trained learning model, and the model outputs the first image with annotation information. The first image with annotation information provides the basic data for constructing the three-dimensional model of the cable shaft.

[0058] Step S5: Based on the measurement results of the cable shaft and the first image with annotation information that is consistent with the measurement results of the cable shaft in time, construct the three-dimensional model of the cable shaft. After the three-dimensional model of the cable shaft is constructed, store the three-dimensional model of the cable shaft and also provide the staff with a panoramic viewing function of the cable shaft.

[0059] Specifically, combine the measurement results of the cable shaft and the first image with annotation information, and use three-dimensional modeling software or algorithms to construct the three-dimensional model of the cable shaft. The three-dimensional model provides an intuitive view of the cable shaft, which helps the staff better understand and analyze the structure and state of the cable shaft. After the three-dimensional model of the cable shaft is constructed, store the model in the database to realize the unified management of the cable shaft data. Also provide a panoramic viewing function for the staff to use. The panoramic viewing function enhances the ability of remote exploration, enabling the staff to quickly understand the detailed situation of the cable shaft without having to visit the site, improving the work efficiency and safety.

[0060] Through the cooperation between the above steps, the present invention can improve the efficiency and safety of cable shaft surveying and mapping, and also provide strong support for the digital management of underground pipe trenches.

[0061] Further, the above step S4 further includes the following steps:

[0062] Step S41: collecting historical images of the cable well as second images, wherein the second images include images containing pipe holes, images of well body collapse, images of cable breakage, and images of water accumulation in the well. The images containing pipe holes also include images of occupied and unoccupied pipe holes.

[0063] Step S42: creating and training a first learning model to classify and label the occupancy status of the pipe holes in the second image, and identifying whether the cable in the pipe hole is broken when the pipe hole is marked as occupied;

[0064] Step S43: Create and train a second learning model to identify and label whether the second image contains features of well collapse, and create and train a third learning model to identify and label whether the second image contains features of water accumulation in the well.

[0065] Specifically, secondary images are collected from historical cable well images. These images should include images of pipe holes, well collapses, cable breaks, and water accumulation within the well. Images containing pipe holes are further subdivided to include images of occupied and unoccupied pipe holes. This expanded range of image types ensures that the model can learn the characteristics of different cable well states, improving its ability to recognize diverse image features. A primary learning model, such as a convolutional neural network (CNN), is created and trained specifically to identify and classify pipe hole occupancy status. For pipe holes marked as occupied, the model can further identify whether the cable within the hole is broken. Through precise image segmentation and object detection, the model provides more detailed information on pipe hole and cable status, automating pipe hole occupancy identification and cable break detection, improving recognition efficiency and accuracy.

[0066] Create and train a second learning model, using deep learning techniques (such as CNN) to identify and annotate the second image for features indicating a well collapse. Create and train a third learning model, also using deep learning techniques to identify and annotate the second image for features indicating well water accumulation. For both the second and third learning models, transfer learning techniques can be used, leveraging pre-trained models to improve training efficiency and model performance. These specialized models can more accurately identify key issues such as well collapse and well water accumulation, facilitating timely action. This not only reduces manual inspection workload but also improves detection efficiency and response speed.

[0067] Furthermore, the above step S42 further includes:

[0068] Create a first learning model based on a deep learning model. The first learning model has a first output branch and a second output branch. The first output branch is used to classify and label the occupancy status of the pipe holes in the second image, and the second output branch is used to identify whether the second image contains a severed cable and label the recognition result.

[0069] Input the second image into the first learning model. In the first layer of the first learning model, extract and identify the pipe holes and their positions in the second image, and use the pipe hole positions as the output results of the first layer. Based on the pipe hole positions, in the second layer of the first learning model, identify whether the pipe hole positions are occupied, classify and label the occupancy status of the pipe hole positions, and use it as the output result of the first output branch. When the output result of the first output branch is that the pipe hole is occupied, in the third layer of the first learning model, identify whether the cable in the occupied pipe hole is severed and label the recognition result, which is also used as the output result of the second output branch.

[0070] Specifically, design a deep learning model that includes two output branches: the first output branch is used for the classification and labeling of the pipe hole occupancy status, and the second output branch is used to identify severed cables and label the recognition results. Use a convolutional neural network (CNN) as the basic architecture, and through multiple levels of feature extraction and learning, achieve the recognition of the pipe hole occupancy status and cable breaks in the image. Input the second image into the first learning model. The first layer of the model is responsible for extracting the pipe holes and their position information in the image. Use image segmentation techniques, such as U-Net or Mask R-CNN, to accurately locate the pipe hole areas and transfer the coordinates and features of these areas to the next layer. In the second layer of the model, based on the pipe hole positions output by the first layer, use a classification model (such as a fully connected layer) to identify whether the pipe holes are occupied and label them. For each pipe hole, output its occupancy status (occupied or unoccupied), and use a bounding box or pixel-level mask to label the occupancy status of the pipe hole on the image. When the output result of the first output branch indicates that the pipe hole is occupied, it means that a cable has been installed in the pipe hole. The third layer of the model further identifies whether the cable in the pipe hole is severed. Use object detection algorithms, such as Faster R-CNN or YOLO, to identify and locate the cable breaks and label them on the image. Through the multi-level processing of the deep learning model, high-precision recognition and labeling of the pipe hole occupancy status and cable breaks in the cable well are achieved. The step-by-step processing from the first layer to the third layer of the model makes the recognition process more refined, improves the recognition accuracy, and provides an efficient and accurate solution for the intelligent management of underground cables and cable trenches.

[0071] Furthermore, the above step S43 further includes:

[0072] Step S431: Calculate the number of images without the well collapse feature image in the second image as the first quantity, and the number of images with the well collapse feature image in the second image as the second quantity. Compare the first quantity and the second quantity. When the first quantity is N times or more of the second quantity (where N represents a positive integer greater than or equal to 3), based on the images without the well collapse feature and the images with the well collapse feature, use data augmentation techniques to generate virtual images with the well collapse feature as virtual images. Use the virtual images and the second image together as the input data for the second learning model to train the second learning model, and use the annotation result of the well collapse as the output result of the second learning model.

[0073] Step S432: Repeat Step S431 to train the third learning model.

[0074] Specifically, extract the second image from the historical data, and classify these images using the annotated well collapse features. Calculate the number of images without the well collapse feature, that is, the first quantity, and the number of images with the well collapse feature, that is, the second quantity. Set a threshold N (N≥3), compare the first quantity and the second quantity. If the first quantity is N times or more of the second quantity, it means that the number of images with the well collapse feature is small, and data augmentation techniques need to be used to balance the dataset. Select the images without the well collapse feature and the images with the well collapse feature as the basis for generating virtual images, and apply data augmentation techniques such as image rotation and scaling to generate virtual images with the well collapse feature, that is, virtual images. For virtual images, it is necessary to ensure that the virtual images are visually similar to the real images and at the same time contain the features of the well collapse to improve the generalization ability of the model. Use the virtual images and the original second image together as input data to train the second learning model. Use the supervised learning method, and use the annotation result of the well collapse as the output result of the model to train the model to recognize the well collapse feature. Through data augmentation techniques, the problem of unbalanced training data is effectively solved, the output result of the model is prevented from being biased to one side, and the recognition ability of the model for the well collapse feature is enhanced. For identifying the feature of water accumulation in the well in the image, the same scheme as the above method can be adopted. Through the implementation of the above steps, the data augmentation technique can be effectively used to solve the problem of insufficient number of specific feature images and improve the recognition ability of the learning model. This method not only improves the performance of the model, but also enhances the sensitivity of the model to abnormal situations.

[0075] Furthermore, the above Step S3 further includes:

[0076] Find the points with the maximum and minimum coordinates along the X-axis, the points with the maximum and minimum coordinates along the Y-axis, and the points with the maximum and minimum coordinates along the Z-axis in the 3D point cloud data as the 8 vertices of the bounding box;

[0077] Construct an axis-aligned bounding box based on the 8 vertices of the bounding box, where each side of the axis-aligned bounding box is parallel to the coordinate axes;

[0078] Calculate the difference between the maximum and minimum coordinate values of the X-axis as the length of the bounding box, calculate the difference between the maximum and minimum coordinate values of the Y-axis as the width of the bounding box, calculate the difference between the maximum and minimum coordinate values of the Z-axis as the height of the bounding box, and construct the bounding box based on the length, width, and height of the bounding box, and use the size of the bounding box as the measurement result of the cable shaft.

[0079] Specifically, by accurately calculating the size of the bounding box, the spatial size of the cable shaft can be accurately obtained, providing important data for design and construction. The axis-aligned bounding box simplifies the calculation process and improves the calculation efficiency. This method automates the measurement process of the cable shaft, reducing the errors and costs of manual measurement. Through the above embodiments, the accurate measurement results of the cable shaft can be effectively obtained, providing strong data support for the intelligent management of underground cables and pipe trenches.

[0080] Furthermore, after constructing the bounding box, the following steps are also included:

[0081] Convert the axis-aligned bounding box to an oriented bounding box, apply the principal component analysis method to calculate the main axis of the three-dimensional point cloud data and use it as the axis vector of the oriented bounding box, and also calculate the centroid of the three-dimensional point cloud data as the center point of the oriented bounding box. For each axis vector, calculate the maximum value of the perpendicular distance from all points in the three-dimensional point cloud data to the first axis, and use it as the semi-axis length of the first axis;

[0082] Use the gradient descent algorithm to fine-tune the center point, axis vector, and semi-axis length of the oriented bounding box to obtain the minimum volume of the oriented bounding box.

[0083] Specifically, further optimize the bounding box, the purpose is to more accurately match the shape of the three-dimensional point cloud data, while minimizing the volume of the bounding box as much as possible. During the above optimization process, the axis vectors of the OBB should be kept orthogonal to ensure that they always form a rectangular box. Through the above method, the bounding box can be effectively optimized to make it more closely match the shape of the three-dimensional point cloud data of the cable shaft, while reducing the consumption of computing resources.

[0084] In this step, the gradient descent algorithm should be used to fine-tune the center point, axis vector, and semi-axis length of the oriented bounding box (OBB) to obtain the minimum volume of the oriented bounding box. Define the objective function: The goal is to minimize the volume of the oriented bounding box. The volume of the oriented bounding box can be calculated from its semi-axis lengths (a, b, c), and the volume formula is

[0085] V = 8abc (since the OBB volume is half of the axis - aligned bounding box (AABB) volume). Thus, our objective function can be defined as f(a, b, c)=8abc.

[0086] Parameter initialization: Initialize the center point (x, y, z), axis vectors (u, v, w), and semi - axis lengths (a, b, c) of the oriented bounding box.

[0087] Calculate the gradient: Calculate the gradient of the objective function f(a, b, c) with respect to each parameter. For the semi - axis lengths, the gradient is: ;

[0088] For the center point and axis vectors, since they affect the orientation and position of the bounding box but not the volume size, their gradients are zero during the process of minimizing the volume.

[0089] Update the parameters: Update the parameters using the gradient descent algorithm. The update rule is:

[0090] ;

[0091] where α is the learning rate.

[0092] Iterative optimization: Repeat the above parameter update steps until the convergence condition is met, such as the gradient is small enough or the number of iterations reaches a preset value.

[0093] Implementation details: In actual implementation, numerical stability and computational efficiency may need to be considered. For example, the PCA method can be used to create the OBB bounding box, which involves calculating the covariance matrix and eigenvectors.

[0094] Code implementation: The mathematical derivation and code implementation of the gradient descent algorithm, as well as how to use the PCA method to create the OBB, can be referred to.

[0095] Through the above steps, we can fine - tune the parameters of the oriented bounding box to obtain the minimized volume of the oriented bounding box. This process requires a combination of mathematical derivation, algorithm implementation, and programming skills to ensure efficient and accurate achievement of the optimization goal.

[0096] Furthermore, by improving the learning rate α to make the training model more accurate, the size of the learning rate directly affects the convergence speed of the algorithm. A larger learning rate can accelerate the convergence speed, but it may also increase the risk of overfitting because the model may oscillate near the minimum value instead of stabilizing. A smaller learning rate can increase the stability of the model but slow down the convergence speed. If the learning rate is set properly, the gradient descent algorithm may jump out of the local minimum and thus have the opportunity to find the global minimum. The learning rate can be adjusted dynamically according to past gradient information to adapt to different parameters and training stages.

[0097] In the present invention, the specific steps are as follows:

[0098] Based on the first and second moments of historical gradients (i.e., the mean and uncentered variance of gradients), adaptively adjust the learning rate for each parameter:

[0099] ,

[0100] wherein, is the first moment estimate (exponentially weighted average of gradients), is the second moment estimate (exponentially weighted average of squared gradients), and are the decay rates controlling these two estimates.

[0101] Add a momentum term to smooth the gradients and accelerate convergence:

[0102] ,

[0103] Use a bias correction step to correct the first and second moment estimates from biasing towards zero in the initial stage:

[0104] ,

[0105] where а is the learning rate, is a small constant to avoid division by zero errors.

[0106] The above steps result in a better а that can converge faster and achieve a better optimization effect for the deep learning model.

[0107] Furthermore, use a slam device to perform a 360-degree rotational scan of the cable shaft, including:

[0108] Safely send the slam device into the cable shaft through a robotic arm, start scanning during the descent of the slam device, and obtain the captured environmental images in real time, and timely adjust the descent speed and direction of the slam device based on the environmental images to ensure that the slam device does not collide with the well wall or other structural objects.

[0109] After the slam device descends to the bottom of the cable shaft, perform initial positioning and attitude adjustment, activate the rotation mechanism of the slam device, and at the same time activate the lidar and camera to start a full-round scan.

[0110] Specifically, the real-time image acquisition and control mechanism in the above solution ensures the safety of the SLAM device during the descent process and reduces the risk of device damage. Through all-round scanning, the SLAM device can efficiently collect complete data of the cable shaft, including three-dimensional structures and internal environment images. Initial positioning and attitude adjustment ensure the accuracy of the scanned data, providing high-quality input for subsequent data processing and analysis. The entire scanning process is automated, reducing the risk of manual operations, improving the operation efficiency and the consistency of data acquisition. Through the implementation of the above steps, the SLAM device can safely and efficiently perform 360-degree rotational scanning of the cable shaft, providing strong technical support for the intelligent management and maintenance of the cable shaft.

[0111] Further, obtaining the three-dimensional point cloud data and the first image of the cable shaft includes:

[0112] The lidar emits laser pulses at a preset frequency, measures the time when the laser pulses are reflected back, obtains the distance information of the structure inside the cable shaft, and generates three-dimensional point cloud data. The camera captures the first image during the rotation of the slam device and records the video information inside the cable shaft.

[0113] Specifically, by the lidar emitting at high frequency and precisely measuring the reflection time of the laser pulses, high-precision three-dimensional point cloud data of the internal structure of the cable shaft can be obtained, providing an accurate data basis for subsequent three-dimensional modeling.

[0114] Further, the above step S2 further includes:

[0115] The first algorithm is to use CloudCompare software, import the three-dimensional point cloud data, create a cube based on the cropping area, define the corner coordinates of the cube, and crop the three-dimensional point cloud data based on the cube to obtain the cropped three-dimensional point cloud data;

[0116] The second algorithm is the RANSAC algorithm. Set the parameters of the RANSAC algorithm and execute the RANSAC algorithm to denoise and filter outliers from the three-dimensional point cloud data. The parameters of the RANSAC algorithm include the number of iterations, model type, distance threshold, and confidence level.

[0117] The present invention also provides a Figure 2 processing system for cable shaft image data based on a slam device as shown, including the following modules:

[0118] The data acquisition unit is used to perform 360-degree rotational scanning of the cable shaft using the slam device, obtain the three-dimensional point cloud data and the first image of the cable shaft, establish the correspondence between the three-dimensional point cloud data and the first image according to the shooting time, and transmit them to the control unit in real time through the wireless transmission module;

[0119] A data preprocessing unit, which is used to preliminarily crop the three-dimensional point cloud data in the control unit by using a first algorithm, remove the redundant point cloud outside the cable shaft, and perform automatic denoising on the three-dimensional point cloud data by using a second algorithm to identify and remove the noise points and outliers in the three-dimensional point cloud data;

[0120] A data measurement unit, which is used to obtain a geometric body that completely encloses the three-dimensional point cloud data as a bounding box, calculate the size of the bounding box, and obtain the cable shaft measurement result;

[0121] An image annotation unit, which is used to collect the images in the historical cable shaft and use them as input data to train the corresponding learning model, input the first image into the trained learning model, and the learning model outputs the first image with annotation information;

[0122] A three-dimensional modeling unit, which is used to construct a three-dimensional model of the cable shaft based on the measurement result of the cable shaft and the first image with annotation information that is consistent with the time of the cable shaft measurement result. After the three-dimensional model of the cable shaft is constructed, store the three-dimensional model of the cable shaft, and also provide the staff with a panoramic viewing function of the cable shaft.

[0123] In summary, the present invention uses a slam device to perform a 360-degree rotational scan on the cable shaft, obtains the three-dimensional point cloud data and the first image of the cable shaft, and establishes a corresponding relationship between the three-dimensional point cloud data and the first image according to the shooting time, and transmits them to the control unit in real time through a wireless transmission module. It realizes the rapid and comprehensive scanning of the internal environment of the cable shaft, eliminates the need for manual entry into the well, and improves the safety of the operation; in the control unit, a first algorithm is used to preliminarily crop the three-dimensional point cloud data, remove the redundant point cloud outside the cable shaft, and a second algorithm is used to perform automatic denoising on the three-dimensional point cloud data to identify and remove the noise points and outliers in the three-dimensional point cloud data, providing a high-quality data basis for subsequent measurement and modeling. Obtain a geometric body that completely encloses the three-dimensional point cloud data as a bounding box, calculate the size of the bounding box, and obtain the cable shaft measurement result, improving the speed and accuracy of the measurement. Collect the images in the historical cable shaft and use them as input data to train the corresponding learning model, input the first image into the trained learning model, and the learning model outputs the first image with annotation information; providing basic data for constructing the three-dimensional model of the cable shaft. Based on the measurement result of the cable shaft and the first image with annotation information that is consistent with the time of the cable shaft measurement result, construct a three-dimensional model of the cable shaft. After the three-dimensional model of the cable shaft is constructed, store the three-dimensional model of the cable shaft, and also provide the staff with a panoramic viewing function of the cable shaft, enabling the staff to quickly understand the detailed situation of the cable shaft without being on-site, improving the work efficiency and safety. Through the cooperation between the above steps, the present invention can improve the efficiency and safety of cable shaft surveying and mapping, and also provide strong support for the digital management of underground pipe trenches.

[0124] It should be understood that although the steps in the flowcharts of the embodiments of the present invention are shown in sequence according to the indications of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0125] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The above program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0126] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0127] The above-mentioned embodiments only represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent for the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several deformations and improvements can be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention shall be subject to the appended claims.

[0128] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A processing method for cable well image data based on a slam device, characterized in that, The method includes: Step S1: Use a slam device to perform a 360-degree rotational scan on the cable shaft to obtain the three-dimensional point cloud data and the first image of the cable shaft, establish the correspondence between the three-dimensional point cloud data and the first image according to the shooting time, and transmit them to the control unit in real time through a wireless transmission module; Step S2: In the control unit, use a first algorithm to preliminarily crop the three-dimensional point cloud data, remove the redundant point cloud outside the cable shaft, and use a second algorithm to automatically denoise the three-dimensional point cloud data, identify and remove the noise points and outliers in the three-dimensional point cloud data; Step S3: Obtain the geometric body that completely encloses the three-dimensional point cloud data as the bounding box, calculate the size of the bounding box, and obtain the measurement result of the cable shaft; Step S4: Collect the images in the historical cable shafts and use them as input data to train the corresponding learning model, input the first image into the trained learning model, and the learning model outputs the first image with annotation information; Step S4 further includes the following steps: Step S41: Collect the images in the historical cable shafts as the second images, where the second images include the images containing pipe holes, the images of the shaft body collapse, the images of cable breakage, and the images of water accumulation in the shaft. The images containing pipe holes also include the situations where the pipe holes are occupied and unoccupied; Step S42: Create and train a first learning model to classify and annotate the occupancy status of the pipe holes in the second images, and when the pipe holes are annotated as occupied, identify whether the cables in the pipe holes are broken; Step S43: Create and train a second learning model to identify and annotate whether the second images contain the features of shaft body collapse, and create and train a third learning model to identify and annotate whether the second images contain the features of water accumulation in the shaft; Step S5: Based on the measurement result of the cable shaft and the first image with annotation information that is consistent with the time of the measurement result of the cable shaft, construct the three-dimensional model of the cable shaft. After the three-dimensional model of the cable shaft is constructed, store the three-dimensional model of the cable shaft and provide the panoramic viewing function of the cable shaft to the staff.

2. The method according to claim 1, characterized in that Step S42 further includes: Create a first learning model based on a deep learning model. The first learning model has a first output branch and a second output branch. The first output branch is used to classify and annotate the occupancy status of the pipe holes in the second images, and the second output branch is used to identify whether the second images contain broken cables and annotate the recognition results; Input the second image into the first learning model. Extract and identify the holes and their positions in the second image in the first layer of the first learning model, and use the hole positions as the output results of the first layer. Based on the hole positions, identify whether the hole positions are occupied in the second layer of the first learning model, classify and label the occupancy status of the hole positions and use it as the output results of the first output branch. When the output results of the first output branch indicate that the holes are occupied, identify whether the cables in the occupied holes are broken ends in the third layer of the first learning model, and label the identification results, which are also used as the output results of the second output branch.

3. The method according to claim 1, characterized in that, The step S43 further includes: Step S431: Calculate the number of images in the second image that do not contain the feature image of well body collapse as the first number, and the number of images in the second image that contain the feature image of well body collapse as the second number. Compare the first number and the second number. When the first number is N times or more of the second number, based on the images that do not contain the feature of well body collapse and the images that contain the feature of well body collapse, use data augmentation technology to generate virtual images with the feature of well body collapse as virtual images. Use the virtual images and the second image together as the input data of the second learning model to train the second learning model, and use the annotation results of well body collapse as the output results of the second learning model, where N represents a positive integer greater than or equal to 3; Step S432: Repeat the step S431 to train the third learning model.

4. The method according to claim 1, characterized in that, The step S3 further includes: Find the points with the maximum and minimum coordinates along the X-axis, the maximum and minimum coordinates along the Y-axis, and the maximum and minimum coordinates along the Z-axis in the three-dimensional point cloud data as the 8 vertices of the bounding box; Construct an axis-aligned bounding box based on the 8 vertices of the bounding box, where each side of the axis-aligned bounding box is parallel to the coordinate axes; Calculate the difference between the maximum coordinate value and the minimum coordinate value of the X-axis as the length of the bounding box, calculate the difference between the maximum coordinate value and the minimum coordinate value of the Y-axis as the width of the bounding box, calculate the difference between the maximum coordinate value and the minimum coordinate value of the Z-axis as the height of the bounding box. Based on the length, width, and height of the bounding box, construct the bounding box, and use the size of the bounding box as the measurement result of the cable well.

5. The method according to claim 4, wherein After constructing the bounding box, it further includes: Convert the axis-aligned bounding box into an oriented bounding box. Apply the principal component analysis method to calculate the main axis of the three-dimensional point cloud data as the axis vector of the oriented bounding box, and also calculate the centroid of the three-dimensional point cloud data as the center point of the oriented bounding box. For each axis vector, calculate the maximum value of the perpendicular distance from all points in the three-dimensional point cloud data to the first axis, and use it as the semi-axis length of the first axis; Use the gradient descent algorithm to fine-tune the center point, the axis vector, and the semi-axis length of the oriented bounding box to obtain the minimum volume of the oriented bounding box.

6. The method according to claim 1, wherein Use a slam device to perform a 360-degree rotational scan of the cable well, including: Safely send the SLAM device into the cable shaft through the robotic arm, start scanning during the descent of the SLAM device, and obtain the captured environmental images in real time. Based on the environmental images, adjust the descent speed and direction of the SLAM device in a timely manner to ensure that the SLAM device does not collide with the well wall or other structural objects; After the SLAM device descends to the bottom of the cable shaft, perform initial positioning and attitude adjustment, activate the rotation mechanism of the SLAM device, and at the same time activate the lidar and camera to start omnidirectional scanning.

7. The method according to claim 6, characterized in that, Obtain the three-dimensional point cloud data and the first image of the cable shaft, including: The lidar emits laser pulses at a preset frequency, measures the time when the laser pulses are reflected back, obtains the distance information of the structures inside the cable shaft, and generates three-dimensional point cloud data. The camera captures the first image during the rotation of the SLAM device and records the video information inside the cable shaft.

8. The method according to claim 1, wherein Step S2 further includes: The first algorithm is to use CloudCompare software to import the three-dimensional point cloud data, create a cube based on the cropping area, define the corner coordinates of the cube, and crop the three-dimensional point cloud data based on the cube to obtain the cropped three-dimensional point cloud data; The second algorithm is the RANSAC algorithm. Set the parameters of the RANSAC algorithm and execute the RANSAC algorithm to denoise and filter outliers from the three-dimensional point cloud data. The parameters of the RANSAC algorithm include the number of iterations, model type, distance threshold, and confidence level.

9. A processing system for cable well image data based on a SLAM device, which is used to implement the method according to any one of claims 1-8, characterized in that, The system includes the following modules: The data acquisition unit is used to use the SLAM device to perform a 360-degree rotation scan of the cable shaft, obtain the three-dimensional point cloud data and the first image of the cable shaft, establish the correspondence between the three-dimensional point cloud data and the first image according to the shooting time, and transmit it to the control unit in real time through the wireless transmission module; The data preprocessing unit is used to preliminarily crop the three-dimensional point cloud data using the first algorithm in the control unit, remove the redundant point cloud outside the cable shaft, and perform automatic denoising on the three-dimensional point cloud data using the second algorithm to identify and remove the noise points and outliers in the three-dimensional point cloud data; The data measurement unit is used to obtain the geometric body that completely encloses the three-dimensional point cloud data as the bounding box, calculate the size of the bounding box, and obtain the measurement result of the cable shaft; The image annotation unit is used to collect the images in the historical cable shafts and use them as input data to train the corresponding learning model. Input the first image into the trained learning model, and the learning model outputs the first image with annotation information; The three-dimensional modeling unit is used to construct the three-dimensional model of the cable shaft based on the measurement result of the cable shaft and the first image with annotation information that is consistent with the time of the measurement result of the cable shaft. After the three-dimensional model of the cable shaft is constructed, store the three-dimensional model of the cable shaft and provide the panoramic viewing function of the cable shaft to the staff.

Images