Cable line refined monitoring method and system based on three-dimensional digital model

Abstract

The present invention relates to a cable line refined monitoring method and system based on a three-dimensional digital model, comprising the following steps: S1: using laser scanning, unmanned aerial vehicles, and total station equipment to map the cable and its surrounding environment to obtain three-dimensional point cloud data, two-dimensional image data, and spatial coordinate data; S2: extracting the depth features of the cable and its ancillary facilities in the image; S3: recovering the position of the camera from the multi-view image through the SfM algorithm and generating three-dimensional point cloud data; S4: integrating the point cloud data generated by laser scanning and the SfM algorithm, unifying them into a coordinate system, and constructing a refined three-dimensional model; S5: installing sensors at key nodes, and synchronizing the data with the three-dimensional model; S6: based on a machine learning algorithm, intelligently predicting the operation data collected by the sensor, dynamically adjusting the alarm threshold, and intelligently generating alarm information. The present invention can effectively improve the refinement of cable line operation monitoring and realize real-time and accurate monitoring of cables and their ancillary facilities.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring, and in particular to a cable line refined monitoring method and system based on a three-dimensional digital model. Background Art

[0002] With the development of intelligent distribution networks and Internet of Things technology, the number of cable wells and cables continues to increase, and the lack of cable environment monitoring devices has led to unclear detailed cable paths and missing cable well monitoring information. The cable operation and maintenance work is faced with problems such as large inspection and maintenance workload, insufficient human resources, and increased equipment risks. Summary of the invention

[0003] In order to solve the above problems, the purpose of the present invention is to provide a cable line refined monitoring method based on a three-dimensional digital model, which can effectively improve the refinement of cable line operation monitoring and realize real-time, accurate monitoring and intelligent analysis of cables and their ancillary facilities.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A cable line refined monitoring method based on a three-dimensional digital model comprises the following steps:

[0006] S1: Use laser scanning, drones, and total station equipment to survey and map the cable and its surrounding environment to obtain three-dimensional point cloud data, two-dimensional image data, and spatial coordinate data, and pre-process the obtained two-dimensional image data and three-dimensional point cloud data;

[0007] Collect basic data, including cable type, laying method, burial depth, entry and exit nodes, path coordinates, etc.

[0008] S2: Use the pre-trained deep learning model to perform target detection on the cables and their ancillary facilities, and extract the deep features of the cables and their ancillary facilities in the image based on the feature extraction model;

[0009] S3: Based on the depth features of the cables and their ancillary facilities in the extracted images, the camera position is restored from the multi-view images through the SfM algorithm and three-dimensional point cloud data is generated;

[0010] S4: Integrate the point cloud data generated by laser scanning and SfM algorithm into a coordinate system, use 3D visualization tools to display the integrated point cloud data in three dimensions, and construct a refined 3D model;

[0011] S5: Install sensors at key nodes, including temperature sensors, humidity sensors, current sensors, and voltage sensors, to monitor power parameters and environmental parameters, and synchronize the data with the three-dimensional model;

[0012] S6: Based on machine learning algorithms, intelligent predictions are made on the operating data collected by sensors, and combined with the cable environment (such as temperature and humidity), the alarm threshold is dynamically adjusted to intelligently generate alarm information.

[0013] Furthermore, preprocessing includes using the Voxel Grid Filte filtering algorithm to remove noise points in the three-dimensional point cloud data, graying the image data, and integrating the spatial coordinate data obtained by the station equipment with the three-dimensional point cloud data to ensure that all data are in a unified coordinate system.

[0014] Furthermore, the pre-trained deep learning model is used to detect the cables and their ancillary facilities. The YOLO3 deep learning model is used. The specific steps are as follows:

[0015] Normalize the pixel values ​​of the input image:

[0016]

[0017] in, is the input image, N O is the pixel value of the input image; is the normalized image;

[0018] Scale the input image to the input size expected by the model :

[0019] ;

[0020] Among them, Resize() represents the scaling operation; is the scaled input image;

[0021] The scaled input image is fed into the pre-trained YOLOv3 model. YOLOv3 divides the input image into S×S grid cells, and predicts B bounding boxes for each grid cell. Each bounding box contains the following information: the offset of the center of the bounding box relative to the grid cell （t x ,t y ） , logarithmic adjustment of the bounding box width and height （t w ,t h ） ; Bounding box confidence p c , category probability distribution p class ;

[0022] Using the bounding box parameters (t x ,ty， t w ,t h ) and the location information of the grid unit (c x ,c y ), which is decoded into the bounding box of the actual image:

[0023] Use the sigmoid function to offset the network output （t x ,t y ） Convert to relative position, then add the grid unit position to get the actual coordinates （b x ,b y ） :

[0024] ;

[0025] ;

[0026] Use exponential function to convert the network output （t w ,t h ） Convert to actual width and height （b w ,b h ） :

[0027] ;

[0028] ;

[0029] in, （p w ,p h ） is the width and height of the prior box;

[0030] Convert the actual coordinates and width and height obtained in the previous step into the actual coordinates in the image:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] in, （xmin ,y min ） is the actual coordinate of the lower left corner of the bounding box, （x max ,y max ） is the actual coordinate of the upper right corner of the bounding box;

[0036] According to the bounding box confidence p c And non-maximum suppression is used to filter out valid bounding boxes, and output the bounding box coordinates and category information.

[0037] Furthermore, the feature extraction model uses VGG16 to extract the deep features of the cables and their ancillary facilities in the image based on the output bounding box coordinates and category information of the cables and their ancillary facilities, as follows:

[0038] For each detected object, extract the corresponding image region:

[0039] ;

[0040] According to the requirements of the VGG model, the image is preprocessed, including:

[0041] Standardized operations:

[0042]

[0043] in, is the standardized image, μ is the mean of the RGB color channels in the training set, and σ is the standard deviation;

[0044] Resize the image to the fixed size required by VGG16:

[0045] ;

[0046] in, is the preprocessed image;

[0047] The preprocessed image is fed into the VGG model to extract low- to mid-level features of the image:

[0048] ;

[0049] ;

[0050] Among them, Conv1-1, Conv1-2.., Conv1-k represent convolution operations at different levels, f 1、 f2...f kRepresents the output feature map of the corresponding convolutional layer; Max represents the pooling operation; P1 represents the output feature map of the pooling layer;

[0051] Repeat the above convolution and pooling operations until the final fully connected layer, where the feature vector is flattened into a one-dimensional vector;

[0052] Finally, deep features are extracted from the penultimate fully connected layer of the VGG16 model.

[0053] Furthermore, step S3 is specifically as follows:

[0054] Based on step S2, feature extraction is performed on the multi-view image, and the extracted feature points are matched using the depth feature points:

[0055] ;

[0056] in, Represents the matching feature point pairs; Indicates Features of an image; Indicates Features of an image;

[0057] Use the RANSAC algorithm to perform robust estimation of the matching feature points and obtain the basic matrix:

[0058] ;

[0059] in, is a set of matching feature points; The function represents the estimation of the basic matrix from the set of matching feature points by the least squares method or other robust estimation methods.

[0060] Combine the basic matrix with the camera intrinsic parameter matrix to get the essential matrix:

[0061] ;

[0062] Among them, K1 and K2 are the intrinsic parameter matrices of the camera;

[0063] Get the camera rotation matrix from the essential matrix decomposition and displacement vector :

[0064] ;

[0065] Among them, the decomposeEssentialMatrix function extracts the rotation matrix between cameras through singular value decomposition of the essential matrix E and translation vector ;

[0066] Use matching points in multi-view images to triangulate and recover 3D points:

[0067] ;

[0068] Among them, P1 and P2 are the camera projection matrices:

[0069]

[0070] Where I is the identity matrix;

[0071] The triangulatePoints function uses the triangulation method in multi-view geometry to calculate the coordinates of a three-dimensional point through matching point pairs in two images and the camera's projection matrix.

[0072] Furthermore, step S4 is specifically as follows:

[0073] The laser scanning directly outputs the 3D point cloud data, represented as P LiDAR , the SfM algorithm outputs the 3D point cloud data generated by multi-view images, represented as P SfM ;

[0074] Use the ICP algorithm to calculate the transformation matrix T ICP , so that P SfM With P LiDAR Alignment;

[0075] ;

[0076] Apply the calculated ICP transformation matrix to P SfM , get the same as P LiDAR Aligned point cloud ;

[0077] ;

[0078] Integrate the aligned point cloud data to form a unified point cloud data :

[0079] ;

[0080] Load the integrated point cloud data into the 3D visualization tool, set the rendering parameters, and perform three-dimensional display to build a refined 3D model.

[0081] Furthermore, the ICP algorithm is used to calculate the transformation matrix T ICP , so that P SfM With P LiDAR Alignment, as follows:

[0082] According to two sets of point cloud data and ; Use nearest neighbor search to find P SfM The closest point to each point in P LiDAR The symmetric point in

[0083] In each iteration, the following steps are performed:

[0084] ① Calculate the centroid of the two point clouds respectively:

[0085] ;

[0086] Where N is the number of point clouds;

[0087] ②Convert the point cloud to the centroid coordinate system:

[0088] ;

[0089] in, is P after removing the centroid SfM Points of a point cloud; is P after removing the centroid LiDAR Points of a point cloud;

[0090] ③Construct the covariance matrix:

[0091] ;

[0092] Among them, the superscript T represents transposition;

[0093] ④ Perform SVD decomposition on the covariance matrix:

[0094] ;

[0095] Where U and V are orthogonal matrices of the singular value decomposition of the covariance matrix H; Σ is a diagonal matrix;

[0096] ⑤Calculate the rotation matrix:

[0097] ;

[0098] ⑥Calculate the rotation matrix:

[0099] ;

[0100] ⑦ Apply rotation and translation to update P SfM :

[0101] ;

[0102] ⑧If the change of the transformation matrix is ​​less than the set threshold, it is considered converged, otherwise return to step ① to continue iterating;

[0103] After the ICP iteration, the rigid transformation matrix T ICP It is expressed as:

[0104] .

[0105] Furthermore, S6 is specifically:

[0106] S61: preprocessing the sensor data, extracting features from the preprocessed data, extracting time and frequency domain features, and storing them in combination with environmental data;

[0107] S62: Train a deep neural network based on historical data to obtain a prediction model, and use cross-validation and hyperparameter optimization to improve model performance;

[0108] S63: Obtain real-time sensor data and make real-time predictions based on the trained deep neural network;

[0109] S64: Calculate a dynamic threshold based on the environmental data, and generate warning information according to the prediction result and the dynamic threshold;

[0110] S65: Synchronize alarm information and real-time data to the 3D model, providing real-time visualization and query functions.

[0111] Furthermore, cross-validation and hyperparameter optimization are used to improve model performance, as follows:

[0112] Divide the historical data into K mutually exclusive subsets, use K-1 subsets for training each time, and use the remaining 1 subset for verification;

[0113] Repeat K times, each time selecting a different subset as the validation set, and finally get the average performance:

[0114] ;

[0115] in, is the cross validation score; Indicates that the current model is evaluated at Compromised performance; Indicates that in the K-fold cross validation Compromise, the model obtained by training with K-1 subsets; Indicates that in the K-fold cross validation The compromise used to verify subset datasets;

[0116] And grid search is used to perform exhaustive search on a predefined grid to find the best hyperparameter combination.

[0117] A cable line refined monitoring system based on a three-dimensional digital model includes a processor, a memory and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the cable line refined monitoring method based on a three-dimensional digital model as described above.

[0118] A computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method steps described above.

[0119] The present invention has the following beneficial effects:

[0120] 1. The present invention can effectively improve the refinement of cable line operation monitoring and realize real-time, accurate monitoring and intelligent analysis of cables and their ancillary facilities;

[0121] 2. The present invention uses the SfM algorithm to restore the camera position from multi-view images and generate a three-dimensional point cloud to achieve high-precision stereo model reconstruction, and integrates the point cloud data generated by laser scanning and the SfM algorithm into a coordinate system to ensure data consistency and model integrity. Finally, a visualization tool is used to display the integrated point cloud data in three dimensions, and a refined three-dimensional model is constructed to provide intuitive visual support for monitoring and analysis.

[0122] 3. The present invention makes intelligent predictions on the operating data collected by the sensor based on a machine learning algorithm and dynamically adjusts the alarm threshold in combination with the cable environment (such as temperature and humidity), intelligently generates alarm information, reduces false alarms and missed alarms, and improves the intelligence level of the alarm system. BRIEF DESCRIPTION OF THE DRAWINGS

[0123] Figure 1 is a flow chart of the method of the present invention;

[0124] Figure 2 It is a detection flow chart in one embodiment of the present invention. DETAILED DESCRIPTION

[0125] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0126] refer to Figure 1 In this embodiment, a cable line refined monitoring method based on a three-dimensional digital model is provided, comprising the following steps:

[0127] S1: Use laser scanning, drones, and total station equipment to survey and map the cable and its surrounding environment to obtain three-dimensional point cloud data, two-dimensional image data, and spatial coordinate data, and pre-process the obtained two-dimensional image data and three-dimensional point cloud data;

[0128] Collect basic data, including cable type, laying method, burial depth, entry and exit nodes, path coordinates, etc.

[0129] S2: Use the pre-trained deep learning model to perform target detection on the cables and their ancillary facilities, and extract the deep features of the cables and their ancillary facilities in the image based on the feature extraction model;

[0130] S3: Based on the depth features of the cables and their ancillary facilities in the extracted images, the camera position is restored from the multi-view images through the SfM algorithm and three-dimensional point cloud data is generated;

[0131] S4: Integrate the point cloud data generated by laser scanning and SfM algorithm into a coordinate system, use 3D visualization tools to display the integrated point cloud data in three dimensions, and construct a refined 3D model;

[0132] In this embodiment, a 3D modeling software (such as AutoCAD, Revit, 3ds Max) is used to build a refined 3D model of the cable and its channel (cable well, junction box). The model should include the cable path, laying method, laying depth, cross-section information, and auxiliary equipment related to the cable.

[0133] S5: Install sensors at key nodes, including temperature sensors, humidity sensors, current and voltage sensors, to monitor power parameters and environmental parameters, and synchronize the data with the three-dimensional model;

[0134] S6: Based on machine learning algorithms, intelligent predictions are made on the operating data collected by sensors, and combined with the cable environment (such as temperature and humidity), the alarm threshold is dynamically adjusted to intelligently generate alarm information.

[0135] Preferably, in this embodiment, preprocessing includes using a Voxel Grid Filte filtering algorithm to remove noise points in the three-dimensional point cloud data, graying the image data, and integrating the spatial coordinate data obtained by the total station device with the three-dimensional point cloud data to ensure that all data are in a unified coordinate system.

[0136] Preferably, in this embodiment, a pre-trained deep learning model is used to perform target detection on the cable and its ancillary facilities, and a YOLO3 deep learning model is adopted. The specific steps are as follows:

[0137] Normalize the pixel values ​​of the input image:

[0138]

[0139] in, is the input image, N O is the pixel value of the input image; is the normalized image;

[0140] Scale the input image to the input size expected by the model :

[0141] ;

[0142] Among them, Resize() represents the scaling operation; is the scaled input image;

[0143] The scaled input image is fed into the pre-trained YOLOv3 model. YOLOv3 divides the input image into S×S grid cells, and predicts B bounding boxes for each grid cell. Each bounding box contains the following information: the offset of the center of the bounding box relative to the grid cell （t x ,t y ） , logarithmic adjustment of the bounding box width and height （t w ,t h ） ; Bounding box confidence p c , category probability distribution p class ;

[0144] Using the bounding box parameters (t x ,t y， t w ,t h ) and the location information of the grid unit (c x ,c y ), which is decoded into the bounding box of the actual image:

[0145] Use the sigmoid function to offset the network output （t x ,t y ） Convert to relative position, then add the grid unit position to get the actual coordinates （b x ,b y ） :

[0146] ;

[0147] ;

[0148] Use exponential function to convert the network output （t w ,th ） Convert to actual width and height （b w ,b h ） :

[0149] ;

[0150] ;

[0151] in, （p w ,p h ） is the width and height of the prior box;

[0152] Convert the actual coordinates and width and height obtained in the previous step into the actual coordinates in the image:

[0153] ;

[0154] ;

[0155] ;

[0156] ;

[0157] in, （x min ,y min ） is the actual coordinate of the lower left corner of the bounding box, （x max ,y max ） is the actual coordinate of the upper right corner of the bounding box;

[0158] According to the bounding box confidence p c And non-maximum suppression is used to filter out valid bounding boxes, and output the bounding box coordinates and category information.

[0159] Preferably, in this embodiment, the feature extraction model uses VGG16, and based on the outputted bounding box coordinates and category information of the cable and its ancillary facilities, the deep features of the cable and its ancillary facilities in the image are extracted, as follows:

[0160] For each detected object, extract the corresponding image region:

[0161] ;

[0162] According to the requirements of the VGG model, the image is preprocessed, including:

[0163] Standardized operations:

[0164] ;

[0165] in, is the standardized image, μ is the mean of the RGB color channels in the training set, and σ is the standard deviation;

[0166] Resize the image to the fixed size required by VGG16:

[0167] ;

[0168] in, is the preprocessed image;

[0169] The preprocessed image is fed into the VGG model to extract low- to mid-level features of the image:

[0170] ;

[0171] ;

[0172] Among them, Conv1-1, Conv1-2.., Conv1-k represent convolution operations at different levels, f 1、 f2...f k Represents the output feature map of the corresponding convolutional layer; Max represents the pooling operation; P1 represents the output feature map of the pooling layer;

[0173] Repeat the above convolution and pooling operations until the final fully connected layer, where the feature vector is flattened into a one-dimensional vector;

[0174] Finally, deep features are extracted from the penultimate fully connected layer of the VGG16 model.

[0175] Preferably, in this embodiment, step S3 is specifically:

[0176] Based on step S2, feature extraction is performed on the multi-view image, and the extracted feature points are matched using the depth feature points:

[0177] ;

[0178] in, Represents the matching feature point pairs; Indicates Features of an image; Indicates Features of an image;

[0179] Use the RANSAC algorithm to perform robust estimation of the matching feature points and obtain the basic matrix:

[0180] ;

[0181] in, is a set of matching feature points; The function represents the estimation of the basic matrix from the set of matching feature points by the least squares method or other robust estimation methods.

[0182] Combine the basic matrix with the camera intrinsic parameter matrix to get the essential matrix:

[0183] ;

[0184] Among them, K1 and K2 are the intrinsic parameter matrices of the camera;

[0185] Get the camera rotation matrix from the essential matrix decomposition and displacement vector :

[0186] ;

[0187] Among them, the decomposeEssentialMatrix function extracts the rotation matrix between cameras through singular value decomposition of the essential matrix E and translation vector ;

[0188] Use matching points in multi-view images to triangulate and recover 3D points:

[0189] ;

[0190] Among them, P1 and P2 are the camera projection matrices:

[0191]

[0192] Where I is the identity matrix;

[0193] The triangulatePoints function uses the triangulation method in multi-view geometry to calculate the coordinates of a three-dimensional point through matching point pairs in two images and the camera's projection matrix.

[0194] Preferably, in this embodiment, step S4 is specifically:

[0195] The three-dimensional point cloud data output by laser scanning is represented as P LiDAR , the SfM algorithm outputs the 3D point cloud data generated by multi-view images, represented as P SfM ;

[0196] Use the ICP algorithm to calculate the transformation matrix T ICP , so that P SfM With P LiDAR Alignment;

[0197] ;

[0198] The calculated T ICP The transformation matrix is ​​applied to P SfM , get the same as P LiDAR Aligned point cloud ;

[0199] ;

[0200] Integrate the aligned point cloud data to form a unified point cloud data :

[0201] ;

[0202] Load the integrated point cloud data into the 3D visualization tool, set the rendering parameters, and perform three-dimensional display to build a refined 3D model.

[0203] Preferably, in this embodiment, the ICP algorithm is used to calculate the transformation matrix T ICP , so that P SfM With P LiDAR Alignment, as follows:

[0204] According to two sets of point cloud data and ; Use nearest neighbor search to find P SfM The closest point to each point in P LiDAR The symmetric point in

[0205] In each iteration, the following steps are performed:

[0206] ① Calculate the centroid of the two point clouds respectively:

[0207] ;

[0208] Where N is the number of point clouds;

[0209] ②Convert the point cloud to the centroid coordinate system:

[0210] ;

[0211] in, is P after removing the centroid SfM Points of a point cloud; is P after removing the centroid LiDAR Points of a point cloud;

[0212] ③Construct the covariance matrix:

[0213] ;

[0214] Among them, the superscript T represents transposition;

[0215] ④ Perform SVD decomposition on the covariance matrix:

[0216] ;

[0217] Where U and V are orthogonal matrices of the singular value decomposition of the covariance matrix H; Σ is a diagonal matrix;

[0218] ⑤Calculate the rotation matrix:

[0219] ;

[0220] ⑥Calculate the rotation matrix:

[0221] ;

[0222] ⑦ Apply rotation and translation to update P SfM :

[0223] ;

[0224] ⑧If the change of the transformation matrix is ​​less than the set threshold, it is considered converged, otherwise return to step ① to continue iterating;

[0225] After the ICP iteration, the rigid transformation matrix T ICP It is expressed as:

[0226] .

[0227] Preferably, in this embodiment, S6 is specifically:

[0228] S61: preprocessing the sensor data, extracting features from the preprocessed data, extracting time and frequency domain features, and storing them in combination with environmental data;

[0229] S62: Train a deep neural network based on historical data to obtain a prediction model, and use cross-validation and hyperparameter optimization to improve model performance;

[0230] S63: Obtain real-time sensor data and make real-time predictions based on the trained deep neural network;

[0231] S64: Calculate a dynamic threshold based on the environmental data, and generate warning information according to the prediction result and the dynamic threshold;

[0232] S65: Synchronize alarm information and real-time data to the 3D model, providing real-time visualization and query functions.

[0233] Preferably, in this embodiment, cross validation and hyperparameter optimization are used to improve model performance, as follows:

[0234] Divide the historical data into K mutually exclusive subsets, use K-1 subsets for training each time, and use the remaining 1 subset for verification;

[0235] Repeat K times, each time selecting a different subset as the validation set, and finally get the average performance:

[0236]

[0237] in, is the cross validation score; Indicates that the current model is evaluated at Compromised performance; Indicates that in the K-fold cross validation Compromise, the model obtained by training with K-1 subsets; Indicates that in the K-fold cross validation The compromise used to verify subset datasets;

[0238] And grid search is used to perform exhaustive search on a predefined grid to find the best hyperparameter combination.

[0239] In another embodiment of the present invention, a cable line refined monitoring system based on a three-dimensional digital model is also provided, including a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically executes the steps of a cable line refined monitoring method based on a three-dimensional digital model as described above.

[0240] In another embodiment of the present invention, a computer storage medium is provided. The computer storage medium stores a plurality of instructions. The instructions are suitable for being loaded by a processor and executing the method steps described above.

[0241] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0242] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0243] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0244] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0245] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A cable line refined monitoring method based on a three-dimensional digital model, characterized in that: The following steps are involved: S1: Use laser scanning, drones, and total station equipment to survey and map the cable and its surrounding environment to obtain three-dimensional point cloud data, two-dimensional image data, and spatial coordinate data, and pre-process the obtained two-dimensional image data and three-dimensional point cloud data; S2: Use the pre-trained deep learning model to perform target detection on the cables and their ancillary facilities, and extract the deep features of the cables and their ancillary facilities in the image based on the feature extraction model; S3: Based on the depth features of the cables and their ancillary facilities in the extracted images, the camera position is restored from the multi-view images through the SfM algorithm and three-dimensional point cloud data is generated; S4: Integrate the point cloud data generated by laser scanning and SfM algorithm into a coordinate system, use 3D visualization tools to display the integrated point cloud data in three dimensions, and construct a refined 3D model; S5: Install sensors at key nodes, including temperature sensors, humidity sensors, current sensors, and voltage sensors, to monitor power parameters and environmental parameters, and synchronize the data with the three-dimensional model; S6: Based on machine learning algorithms, intelligent predictions are made on the operating data collected by sensors, and combined with the environmental data of the cables, the alarm threshold is adjusted dynamically, and alarm information is generated intelligently. The pre-trained deep learning model is used to detect the cable and its ancillary facilities, and the YOLO3 deep learning model is used. The specific steps are as follows: Normalize the pixel values ​​of the input image: in, is the input image, N O is the pixel value of the input image; is the normalized image; Scale the input image to the input size expected by the model : ； Among them, Resize() represents the scaling operation; is the scaled input image; The scaled input image is fed into the pre-trained YOLOv3 model. YOLOv3 divides the input image into S×S grid cells. Each grid cell predicts B bounding boxes. Each bounding box contains the following information: the offset of the center of the bounding box relative to the grid cell （t x ,t y ） , logarithmic adjustment of the bounding box width and height （t w ,t h ） ; Bounding box confidence p c , category probability distribution p class ; Using the bounding box parameters (t x ,t y， t w ,t h ) and the location information of the grid unit (c x ,c y ), which is decoded into the bounding box of the actual image: Use the sigmoid function to offset the network output （t x ,t y ） Convert to relative position, then add the grid unit position to get the actual coordinates （b x ,b y ） : ； ； Use exponential function to convert the network output （t w ,t h ） Convert to actual width and height （b w ,b h ） : ; ; in, （p w ,p h ） is the width and height of the prior box; Convert the actual coordinates and width and height obtained in the previous step into the actual coordinates in the image: ; ; ; ; in, （x min ,y min ） is the actual coordinate of the lower left corner of the bounding box, （x max ,y max ） is the actual coordinate of the upper right corner of the bounding box; According to the bounding box confidence p c And non-maximum suppression is used to filter out valid bounding boxes, and output the bounding box coordinates and category information.

2. According to claim 1, a cable line refined monitoring method based on a three-dimensional digital model is characterized in that: The preprocessing includes using a Voxel Grid Filte filtering algorithm to remove noise points in the three-dimensional point cloud data, graying the image data, and integrating the spatial coordinate data obtained by the total station device with the three-dimensional point cloud data to ensure that all data are in a unified coordinate system.

3. The cable line refined monitoring method based on a three-dimensional digital model according to claim 1 is characterized in that: The feature extraction model uses VGG16 to extract the deep features of the cables and their ancillary facilities in the image based on the output bounding box coordinates and category information of the cables and their ancillary facilities, as follows: For each detected object, extract the corresponding image region: ； According to the requirements of the VGG model, the image is preprocessed, including: Standardized operations: ; in, is the standardized image, μ is the mean of the RGB color channels in the training set, and σ is the standard deviation; Resize the image to the fixed size required by VGG16: ； in, is the preprocessed image; The preprocessed image is fed into the VGG model to extract low- to mid-level features of the image: ； ； Among them, Conv1-1, Conv1-2.., Conv1-k represent convolution operations at different levels, f 1、 f2...f k Represents the output feature map of the corresponding convolutional layer; Max represents the pooling operation; P1 represents the output feature map of the pooling layer; Repeat the above convolution and pooling operations until the final fully connected layer, where the feature vector is flattened into a one-dimensional vector; Finally, deep features are extracted from the penultimate fully connected layer of the VGG16 model.

4. The cable line refined monitoring method based on a three-dimensional digital model according to claim 1 is characterized in that: The step S3 is specifically as follows: Based on step S2, feature extraction is performed on the multi-view image, and the extracted feature points are matched using the depth feature points: ; in, Represents the matching feature point pairs; Indicates Features of an image; Indicates Features of an image; Use the RANSAC algorithm to perform robust estimation of the matching feature points and obtain the basic matrix: ； in, is a set of matching feature points; The function represents estimating the basic matrix from the set of matching feature points by the least squares method; Combine the basic matrix with the camera intrinsic parameter matrix to get the essential matrix: ; Among them, K1 and K2 are the intrinsic parameter matrices of the camera; Get the camera rotation matrix from the essential matrix decomposition and displacement vector : ; Among them, the decomposeEssentialMatrix function extracts the rotation matrix between cameras through singular value decomposition of the essential matrix E and translation vectors ; Use matching points in multi-view images to triangulate and recover 3D points: ; Among them, P1 and P2 are the camera projection matrices: Where I is the identity matrix; The triangulatePoints function uses the triangulation method in multi-view geometry to calculate the coordinates of a three-dimensional point through matching point pairs in two images and the camera's projection matrix.

5. The cable line refined monitoring method based on a three-dimensional digital model according to claim 1 is characterized in that: The step S4 is specifically as follows: The three-dimensional point cloud data output by laser scanning is represented as P LiDAR , the SfM algorithm outputs the 3D point cloud data generated by multi-view images, represented as P SfM ; Use the ICP algorithm to calculate the transformation matrix T ICP , so that P SfM With P LiDAR Alignment; ; The calculated T ICP The transformation matrix is ​​applied to P SfM , get the same as P LiDAR Aligned point cloud ; ； Integrate the aligned point cloud data to form a unified point cloud data : ； Load the integrated point cloud data into the 3D visualization tool, set the rendering parameters, and perform three-dimensional display to build a refined 3D model.

6. A cable line refined monitoring method based on a three-dimensional digital model according to claim 5, characterized in that: The ICP algorithm is used to calculate the transformation matrix T ICP , so that P SfM With P LiDAR Alignment, as follows: According to two sets of point cloud data and ; Use nearest neighbor search to find P SfM The closest point to each point in P LiDAR The symmetric point in In each iteration, the following steps are performed: ① Calculate the centroid of the two point clouds respectively: ； Where N is the number of point clouds; ②Convert the point cloud to the centroid coordinate system: ； in, is P after removing the centroid SfM Points of a point cloud; is P after removing the centroid LiDAR Points of a point cloud; ③Construct the covariance matrix: ； Among them, the superscript T represents transposition; ④ Perform SVD decomposition on the covariance matrix: ； Where U and V are orthogonal matrices of the singular value decomposition of the covariance matrix H; Σ is a diagonal matrix; ⑤Calculate the rotation matrix: ； ⑥Calculate the rotation matrix: ； ⑦ Apply rotation and translation to update P SfM : ； ⑧If the change of the transformation matrix is ​​less than the set threshold, it is considered converged, otherwise return to step ① to continue iterating; After the ICP iteration, the rigid transformation matrix T ICP It is expressed as: 。 7. The cable line refined monitoring method based on a three-dimensional digital model according to claim 1 is characterized in that: The S6 is specifically: S61: preprocessing the sensor data, extracting features from the preprocessed data, extracting time and frequency domain features, and storing them in combination with environmental data; S62: Train a deep neural network based on historical data to obtain a prediction model, and use cross-validation and hyperparameter optimization to improve model performance; S63: Obtain real-time sensor data and make real-time predictions based on the trained deep neural network; S64: Calculate a dynamic threshold based on the environmental data, and generate warning information according to the prediction result and the dynamic threshold; S65: Synchronize alarm information and real-time data to the 3D model, providing real-time visualization and query functions.

8. The cable line refined monitoring method based on a three-dimensional digital model according to claim 7 is characterized in that: Improve model performance using cross-validation and hyperparameter optimization as follows: Divide the historical data into K mutually exclusive subsets, use K-1 subsets for training each time, and use the remaining 1 subset for verification; Repeat K times, each time selecting a different subset as the validation set, and finally get the average performance: ; in, is the cross validation score; Indicates that the current model is evaluated at Compromised performance; Indicates that in the K-fold cross validation Compromise, the model obtained by training with K-1 subsets; Indicates that in the K-fold cross validation The compromise used to verify subset datasets; And grid search is used to perform exhaustive search on a predefined grid to find the best hyperparameter combination.

9. A cable line refined monitoring system based on a three-dimensional digital model, characterized in that: It includes a processor, a memory and a computer program stored in the memory. When the processor executes the computer program, it specifically executes the steps in the cable line refined monitoring method based on a three-dimensional digital model as described in any one of claims 1-8.

Images