Intelligent tunnel inspection method and system based on robot
Through the robot intelligent inspection method, path planning and deep learning technology are used to identify cracks in the tunnel, solving the problems of low efficiency and high safety risks in traditional manual inspections, and achieving efficient and safe tunnel management.
Patent Information
- Application Number
- CN202410955185.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-08
AI Technical Summary
Traditional manual tunnel inspections are inefficient, costly and have high safety risks, making it difficult to meet the needs of tunnel safety management, especially in harsh conditions such as high temperature, high humidity and harmful gases, it is difficult to effectively monitor and identify damage and failure of tunnel facilities.
Using robot-based intelligent tunnel inspection method, using path planning algorithms and expansion node strategies to plan patrol paths, combining lidar and visual sensors to build a panoramic map, a deep neural network integrating attention mechanisms to identify cracks, and based on detected cracks Information dynamically adjusts the inspection speed.
It improves patrol efficiency and accuracy, reduces costs, improves the safety and comprehensiveness of tunnel management, and ensures timely identification and handling of tunnel facilities damage and faults.
Smart Images

Figure CN118915732A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of tunnel inspection, and in particular to a robot-based intelligent tunnel inspection method and system. Background Art
[0002] As an important part of the modern urban transportation system, tunnels undertake the important task of connecting various areas of the city and ensuring smooth traffic. However, due to the complex internal environment of tunnels and long-term exposure to harsh conditions such as high temperature, high humidity, and harmful gases, tunnel facilities are easily damaged or malfunction, posing a great hidden danger to the safe operation of tunnels. Traditional manual inspection methods are inefficient, costly, and have great safety risks, making it difficult to meet the needs of tunnel safety management. To address this problem, this patent proposes a robot-based intelligent tunnel inspection method, which uses artificial intelligence technology to conduct intelligent tunnel inspections, improve inspection efficiency, reduce costs, enhance safety, and improve tunnel management. Summary of the invention
[0003] In view of this, the present invention provides a robot-based intelligent tunnel inspection method, the purpose of which is to: 1) use a path planning algorithm to plan the robot's inspection path, and adopt an expansion node strategy to better cover the entire tunnel, reduce blind spots and missed inspection areas in path planning, and improve inspection efficiency and comprehensiveness; 2) use data collected by lidar and visual sensors to build a panoramic map of the interior of the tunnel according to the planned inspection path, and provide more detailed information and a more comprehensive perspective for subsequent crack identification; 3) construct a deep neural network that integrates an attention mechanism to identify cracks on the tunnel surface, which can better capture the characteristic information of the cracks and improve the accuracy and robustness of crack identification; 4) dynamically adjust the inspection speed according to the detected crack information, so that the robot can respond to the crack situation more intelligently, optimize the inspection efficiency, and improve the timeliness and accuracy of crack detection.
[0004] To achieve the above object, the present invention provides a robot-based intelligent tunnel inspection method, comprising the following steps:
[0005] S1: Based on the tunnel structure design diagram, the robot's inspection path is calculated using a path planning algorithm to cover the entire tunnel and achieve rapid tunnel inspection, wherein graph search based on an extended node strategy is the main implementation method of the path planning;
[0006] S2: The robot is equipped with a laser radar, a visual sensor, and an inertial navigation system. The laser radar is used to obtain the three-dimensional point cloud data inside the tunnel. The visual sensor is used to collect image information. The inertial navigation system is used to collect images according to the planned inspection path to build a panoramic map of the tunnel.
[0007] S3: Construct a tunnel surface crack recognition model, analyze the images and point cloud data collected in the tunnel, and identify tunnel surface cracks, wherein a deep neural network integrating an attention mechanism is the main implementation method for identifying tunnel surface cracks;
[0008] S4: Dynamically adjust the inspection speed according to the detected crack information to complete the inspection task.
[0009] As a further improvement method of the present invention:
[0010] Optionally, in step S1, calculating the inspection path of the robot using a path planning algorithm based on the tunnel structure design drawing includes:
[0011] S11: Acquire tunnel structure design drawing data, where the design drawing data is an electronic map in a CAD file format;
[0012] S12: performing rasterization processing on the acquired map data to obtain rasterized map data;
[0013] S13: representing the rasterized map as a graph data structure, where a node represents a location point in the tunnel, and an edge represents a connectivity relationship between two location points;
[0014] S14: Perform path search on the graph structure to obtain a coarse-grained inspection path, wherein the graph search based on the extended node strategy is the main implementation method of the path search, and the specific process includes:
[0015] S141: Add the starting node to the list of nodes to be expanded, and initialize the distance function value and the heuristic function value;
[0016] S142: Select the node with the minimum heuristic function value among the nodes to be expanded according to the heuristic function calculation, calculate and update the distance function of the adjacent nodes according to the adjacent nodes of the node, and add the updated node to the list of nodes to be expanded. The distance update calculation formula is:
[0017] d(v)=d(u)+c(u,v)
[0018] in:
[0019] d(v) represents the distance from the starting node to node v, d(u) represents the distance from the starting node to node u, and c(u,v) represents the distance from node u to node v;
[0020] The heuristic function calculation formula is:
[0021]
[0022] in:
[0023] h(v) represents the estimated cost from node v to the target node;
[0024] (x v ,y v ) represents the coordinates of node v;
[0025] (x goal ,y goal ) represents the coordinates of the target node;
[0026] S143: If the target node is found, the start node is found from the target node along the parent node pointer by backtracking to obtain the optimal path; otherwise, the process returns to step S142 to continue searching until the optimal path is found and returned as the coarse-grained inspection path;
[0027] S15: Smoothing the coarse-grained inspection path obtained through searching, removing the jagged parts in the path, and smoothing the path to obtain a fine-grained inspection path.
[0028] Optionally, in step S15, the coarse-grained inspection path obtained by searching is smoothed to remove the jagged parts in the path, so as to smooth the path to obtain a fine-grained inspection path, including:
[0029] The coarse-grained inspection path obtained by searching is used as input to perform Bezier curve fitting to obtain a smooth curve as the fine-grained inspection path. The calculation formula is:
[0030]
[0031] in:
[0032] B(t) represents a point on the Bezier curve;
[0033] n represents the order of the Bezier curve;
[0034] Indicates the number of combinations;
[0035] t represents the point parameter, and its value range is [0,1];
[0036] P i Represents the coordinates of the i-th control point.
[0037] Optionally, in step S2, three-dimensional point cloud data of the interior of the tunnel is acquired by using a laser radar, image information is collected by a visual sensor, and image acquisition is performed according to the planned inspection path in combination with an inertial navigation system to realize construction of a panoramic map of the interior of the tunnel, including:
[0038] S21: Scanning the interior of the tunnel using a laser radar to obtain three-dimensional point cloud data, where the point cloud data includes coordinate information of various positions inside the tunnel; using a visual sensor to collect images inside the tunnel according to the planned inspection path and using an inertial navigation system to obtain position information of the images inside the tunnel;
[0039] S22: Fusing the three-dimensional point cloud data acquired by the laser radar and the image information acquired by the visual sensor, the specific process includes: performing denoising and coordinate system conversion operations on the point cloud data acquired by the laser radar and the image acquired by the visual sensor to obtain preprocessed point cloud data and image data; extracting features from the preprocessed image and point cloud data, and matching them, using SIFT, SURF and ORB algorithms to extract features from the preprocessed image data; extracting features from the preprocessed point cloud data using curvature and normal vector attributes, matching the extracted point cloud features with image features through descriptor matching, and performing coordinate conversion between the matched feature points in the image and the point cloud so that they are represented in the same coordinate system; fusing the matched feature points to obtain fused data, and the feature point fusion is optimized by minimizing the reprojection error so that the distance between the projection of the point cloud feature in the image and the feature point in the actual image is minimized;
[0040] S23: Using the fused data, a voxel grid method is used to perform three-dimensional reconstruction to obtain a panoramic map of the interior of the tunnel.
[0041] Optionally, in step S3, a tunnel surface crack recognition model is constructed to analyze the images and point cloud data collected in the tunnel to identify tunnel surface cracks, including:
[0042] S31: Use the pre-trained ResNet model to extract features from the pre-processed tunnel surface image to obtain an image feature vector;
[0043] S32: Use the PointNet network to extract features from the preprocessed point cloud data to obtain a point cloud feature vector;
[0044] S33: fusing the extracted image feature vector and the point cloud feature vector to obtain a fused feature vector;
[0045] S34: Construct a tunnel surface crack recognition model integrating an attention mechanism, wherein the model includes a convolutional layer, an attention module, a pooling module, and a fully connected layer. The detailed structure of the image crack recognition model integrating an attention mechanism is as follows:
[0046] The convolution layer includes: convolution layer 1 and convolution layer 2, wherein the convolution kernel size of convolution layer 1 is 3*3, the number of output feature maps is 32, and the ReLU activation function is used; the convolution kernel size of convolution layer 2 is 3*3, the number of output feature maps is 64, and the ReLU activation function is used;
[0047] The attention module obtains the final attention feature map by fusing spatial attention and channel attention;
[0048] The pooling window size of the pooling layer is 2*2, which is used to reduce the size of the feature map;
[0049] The fully connected layer includes a fully connected layer 1 and a fully connected layer 2, wherein the number of neurons in the fully connected layer 1 is 512, and a ReLU activation function is used; the Dropout layer is used to prevent overfitting and randomly discard some neurons; the number of neurons in the fully connected layer 2 is 2, and a Softmax activation function is used to output the probability distribution of whether there is a crack;
[0050] S35: Parameter optimization is performed on the constructed tunnel surface crack identification model to obtain an optimal model instance for tunnel crack identification, wherein a whale algorithm integrating multiple population strategies is an implementation method for the parameter optimization.
[0051] In the step S35, the constructed tunnel surface crack identification model is optimized for parameters to obtain an optimal model instance for tunnel crack identification, including:
[0052] The whale algorithm integrating multiple population strategies introduces multiple independent whale groups and shares information among them to increase the diversity of global search. The specific process includes:
[0053] S351: Generate a preset number of independent whale groups, each group contains a preset number of whale individuals, and randomly initialize the position and speed of each whale individual;
[0054] S352: Calculate the fitness of each individual whale, and determine the fitness value of each individual according to the objective function of the tunnel surface crack recognition model;
[0055] S353: Perform an update operation on each whale group, including: updating the position and speed of individual whales, and obtaining new positions and speeds through formula calculation; updating the optimal solution in the group, and the position update formula of individual whales is:
[0056] x i (t+1)=x i (t)+v i (t+1)
[0057] The speed update formula of individual whales is:
[0058] v i (t+1)=v i (t)+A·C
[0059] in:
[0060] x i (t) represents the position of the i-th whale individual at iteration t of round t;
[0061] v i (t) represents the speed of the i-th whale individual in the t-th iteration;
[0062] A represents the acceleration coefficient;
[0063] C represents the exploration coefficient;
[0064] S354: Iterate steps S352 and S353 until the maximum number of iterations is reached and output the optimal model parameters.
[0065] Optionally, in step S4, the inspection speed is dynamically adjusted according to the detected crack information to complete the inspection task, including:
[0066] According to the crack information, the inspection speed is dynamically adjusted to speed up the inspection frequency of the crack area to ensure that more time and resources are used for the inspection of key areas. The calculation formula is:
[0067] V new =V base +log(N)
[0068] in:
[0069] V base Indicates the basic inspection speed:
[0070] N represents the number of tunnel cracks identified at the current position:
[0071] V new Indicates the new inspection speed adjusted according to the crack information.
[0072] In order to solve the above problems, the present invention provides a robot-based intelligent tunnel inspection system, the system comprising:
[0073] Inspection path planning module, which is used to calculate the robot's inspection path based on the tunnel structure design drawing using the path planning algorithm to cover the entire tunnel and achieve rapid tunnel inspection;
[0074] The tunnel map construction module is used to carry laser radar, visual sensor and inertial navigation system on the robot, use laser radar to obtain three-dimensional point cloud data inside the tunnel, collect image information through visual sensor, and combine with the inertial navigation system to collect images according to the planned inspection path to realize the construction of a panoramic map of the tunnel;
[0075] The tunnel surface crack recognition module is used to build a tunnel surface crack recognition model, analyze the images and point cloud data collected in the tunnel, identify tunnel surface cracks, dynamically adjust the inspection speed according to the detected crack information, and complete the inspection task.
[0076] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:
[0077] A memory storing at least one instruction;
[0078] Communication interface, enabling electronic equipment to communicate; and
[0079] The processor executes the instructions stored in the memory to implement the above-mentioned robot-based intelligent tunnel inspection method.
[0080] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned robot-based intelligent tunnel inspection method.
[0081] Compared with the prior art, the present invention proposes a robot-based intelligent tunnel inspection method, which has the following advantages:
[0082] First, this scheme proposes a robot inspection path planning method, which uses a path planning algorithm to plan the robot's inspection path and adopts an expansion node strategy to better cover the entire tunnel, reduce blind spots and missed inspection areas in path planning, and improve inspection efficiency and comprehensiveness. By using data collected by lidar and visual sensors and following the planned inspection path, a panoramic map of the interior of the tunnel is constructed, providing more detailed information and a more comprehensive perspective for subsequent crack identification.
[0083] At the same time, this scheme constructs a deep neural network that integrates the attention mechanism to identify cracks on the tunnel surface. It can better capture the characteristic information of the cracks, improve the accuracy and robustness of crack identification, and dynamically adjust the inspection speed according to the detected crack information, so that the robot can respond to crack conditions more intelligently, optimize inspection efficiency, and improve the timeliness and accuracy of crack detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0084] Figure 1 A schematic diagram of a flow chart of a robot-based intelligent tunnel inspection method provided by one embodiment of the present invention;
[0085] Figure 2 A functional module diagram of a robot-based intelligent tunnel inspection system provided in one embodiment of the present invention;
[0086] Figure 3 A schematic diagram of the structure of an electronic device for implementing a robot-based intelligent tunnel inspection method provided by an embodiment of the present invention.
[0087] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0088] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0089] The embodiment of the present application provides a robot-based intelligent tunnel inspection method. The execution subject of the robot-based intelligent tunnel inspection method includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the robot-based intelligent tunnel inspection method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.
[0090] Embodiment 1:
[0091] S1: Based on the tunnel structure design drawing, the robot's inspection path is calculated using a path planning algorithm to cover the entire tunnel and achieve rapid tunnel inspection, wherein graph search based on an expansion node strategy is the implementation method of the path planning.
[0092] In the step S1, the inspection path of the robot is calculated using a path planning algorithm based on the tunnel structure design drawing, including:
[0093] S11: Acquire tunnel structure design drawing data, where the design drawing data is an electronic map in a CAD file format;
[0094] S12: performing rasterization processing on the acquired map data to obtain rasterized map data;
[0095] S13: representing the rasterized map as a graph data structure, where a node represents a location point in the tunnel, and an edge represents a connectivity relationship between two location points;
[0096] S14: Perform path search on the graph structure to obtain a coarse-grained inspection path, wherein the graph search based on the extended node strategy is the main implementation method of the path search, and the specific process includes:
[0097] S141: Add the starting node to the list of nodes to be expanded, and initialize the distance function value and the heuristic function value;
[0098] S142: Select the node with the minimum heuristic function value among the nodes to be expanded according to the heuristic function calculation, calculate and update the distance function of the adjacent nodes according to the adjacent nodes of the node, and add the updated node to the list of nodes to be expanded. The distance update calculation formula is:
[0099] d(v)=d(u)+c(u,v)
[0100] in:
[0101] d(v) represents the distance from the starting node to node v, d(u) represents the distance from the starting node to node u, and c(u,v) represents the distance from node u to node v;
[0102] The heuristic function calculation formula is:
[0103]
[0104] in:
[0105] h(v) represents the estimated cost from node v to the target node;
[0106] (x v ,y v ) represents the coordinates of node v;
[0107] (x goal ,y goal ) represents the coordinates of the target node;
[0108] S143: If the target node is found, the start node is found from the target node along the parent node pointer by backtracking to obtain the optimal path; otherwise, the process returns to step S142 to continue searching until the optimal path is found and returned as the coarse-grained inspection path;
[0109] S15: Smoothing the coarse-grained inspection path obtained through searching, removing the jagged parts in the path, and smoothing the path to obtain a fine-grained inspection path.
[0110] In the step S15, the coarse-grained inspection path obtained by searching is smoothed to remove the jagged parts in the path, so as to smooth the path to obtain a fine-grained inspection path, including:
[0111] The coarse-grained inspection path obtained by searching is used as input to perform Bezier curve fitting to obtain a smooth curve as the fine-grained inspection path. The calculation formula is:
[0112]
[0113] in:
[0114] B(t) represents a point on the Bezier curve;
[0115] n represents the order of the Bezier curve;
[0116] Indicates the number of combinations;
[0117] t represents the point parameter, and its value range is [0,1];
[0118] P i Represents the coordinates of the i-th control point.
[0119] S2: The robot is equipped with a laser radar, a visual sensor and an inertial navigation system. The laser radar is used to obtain the three-dimensional point cloud data inside the tunnel. The visual sensor is used to collect image information. The inertial navigation system is used to collect images according to the planned inspection path to realize the construction of a panoramic map of the inside of the tunnel.
[0120] In the step S2, the three-dimensional point cloud data inside the tunnel is obtained by using a laser radar, image information is collected by a visual sensor, and image collection is performed according to the planned inspection path in combination with an inertial navigation system to realize the construction of a panoramic map inside the tunnel, including:
[0121] S21: Scanning the interior of the tunnel using a laser radar to obtain three-dimensional point cloud data, where the point cloud data includes coordinate information of various positions inside the tunnel; using a visual sensor to collect images inside the tunnel according to the planned inspection path and using an inertial navigation system to obtain position information of the images inside the tunnel;
[0122] S22: Fusing the three-dimensional point cloud data acquired by the laser radar and the image information acquired by the visual sensor, the specific process includes: performing denoising and coordinate system conversion operations on the point cloud data acquired by the laser radar and the image acquired by the visual sensor to obtain preprocessed point cloud data and image data; extracting features from the preprocessed image and point cloud data, and matching them, using SIFT, SURF and ORB algorithms to extract features from the preprocessed image data; extracting features from the preprocessed point cloud data using curvature and normal vector attributes, matching the extracted point cloud features with image features through descriptor matching, and performing coordinate conversion between the matched feature points in the image and the point cloud so that they are represented in the same coordinate system; fusing the matched feature points to obtain fused data, and the feature point fusion is optimized by minimizing the reprojection error so that the distance between the projection of the point cloud feature in the image and the feature point in the actual image is minimized;
[0123] S23: Using the fused data, a voxel grid method is used to perform three-dimensional reconstruction to obtain a panoramic map of the interior of the tunnel.
[0124] S3: Construct a tunnel surface crack recognition model, analyze the images and point cloud data collected in the tunnel, and identify tunnel surface cracks, wherein a deep neural network integrating an attention mechanism is an implementation method for identifying tunnel surface cracks.
[0125] In the step S3, a tunnel surface crack recognition model is constructed, and images and point cloud data collected in the tunnel are analyzed to identify tunnel surface cracks, including:
[0126] S31: Use the pre-trained ResNet model to extract features from the pre-processed tunnel surface image to obtain an image feature vector;
[0127] S32: Use the PointNet network to extract features from the preprocessed point cloud data to obtain a point cloud feature vector;
[0128] S33: fusing the extracted image feature vector and the point cloud feature vector to obtain a fused feature vector;
[0129] S34: Construct a tunnel surface crack recognition model integrating an attention mechanism, wherein the model includes a convolutional layer, an attention module, a pooling module, and a fully connected layer. The detailed structure of the image crack recognition model integrating an attention mechanism is as follows:
[0130] The convolution layer includes: convolution layer 1 and convolution layer 2, wherein the convolution kernel size of convolution layer 1 is 3*3, the number of output feature maps is 32, and the ReLU activation function is used; the convolution kernel size of convolution layer 2 is 3*3, the number of output feature maps is 64, and the ReLU activation function is used;
[0131] The attention module obtains the final attention feature map by fusing spatial attention and channel attention;
[0132] The pooling window size of the pooling layer is 2*2, which is used to reduce the size of the feature map;
[0133] The fully connected layer includes a fully connected layer 1 and a fully connected layer 2, wherein the number of neurons in the fully connected layer 1 is 512, and a ReLU activation function is used; the Dropout layer is used to prevent overfitting and randomly discard some neurons; the number of neurons in the fully connected layer 2 is 2, and a Softmax activation function is used to output the probability distribution of whether there is a crack;
[0134] S35: Parameter optimization is performed on the constructed tunnel surface crack identification model to obtain an optimal model instance for tunnel crack identification, wherein a whale algorithm integrating multiple population strategies is an implementation method for the parameter optimization.
[0135] In the step S35, the constructed tunnel surface crack identification model is optimized for parameters to obtain an optimal model instance for tunnel crack identification, including:
[0136] The whale algorithm integrating multiple population strategies introduces multiple independent whale groups and shares information among them to increase the diversity of global search. The specific process includes:
[0137] S351: Generate a preset number of independent whale groups, each group contains a preset number of whale individuals, and randomly initialize the position and speed of each whale individual;
[0138] S352: Calculate the fitness of each individual whale, and determine the fitness value of each individual according to the objective function of the tunnel surface crack recognition model;
[0139] S353: Perform an update operation on each whale group, including: updating the position and speed of individual whales, and obtaining new positions and speeds through formula calculation; updating the optimal solution in the group, and the position update formula of individual whales is:
[0140] x i (t+1)=x i (t)+v i (t+1)
[0141] The speed update formula of individual whales is:
[0142] v i (t+1)=v i (t)+A·C
[0143] in:
[0144] x i (t) represents the position of the i-th whale individual at iteration t of round t;
[0145] v i (t) represents the speed of the i-th whale individual in the t-th iteration;
[0146] A represents the acceleration coefficient;
[0147] C represents the exploration coefficient;
[0148] S354: Iterate steps S352 and S353 until the maximum number of iterations is reached and output the optimal model parameters.
[0149] S4: Dynamically adjust the inspection speed according to the detected crack information to complete the inspection task.
[0150] In step S4, the inspection speed is dynamically adjusted according to the detected crack information to complete the inspection task, including:
[0151] According to the crack information, the inspection speed is dynamically adjusted to speed up the inspection frequency of the crack area to ensure that more time and resources are used for the inspection of key areas. The calculation formula is:
[0152] V new =V base +log(N)
[0153] in:
[0154] V base Indicates the basic inspection speed:
[0155] N represents the number of tunnel cracks identified at the current position:
[0156] V new Indicates the new inspection speed adjusted according to the crack information.
[0157] Embodiment 2:
[0158] like Figure 2 , is a functional module diagram of a robot-based intelligent tunnel inspection system provided in one embodiment of the present invention, which can implement the robot-based intelligent tunnel inspection method in Example 1.
[0159] The robot-based tunnel intelligent inspection system 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the robot-based tunnel intelligent inspection system can include an inspection path planning module 101, a tunnel map building module 102, and a tunnel surface crack identification module 103. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.
[0160] The inspection path planning module 101 is used to calculate the inspection path of the robot using a path planning algorithm based on the tunnel structure design diagram to cover the entire tunnel and realize rapid tunnel inspection;
[0161] The tunnel map construction module 102 is used to carry a laser radar, a visual sensor and an inertial navigation system on the robot, use the laser radar to obtain three-dimensional point cloud data inside the tunnel, collect image information through the visual sensor, and combine the inertial navigation system to collect images according to the planned inspection path to realize the construction of a panoramic map inside the tunnel;
[0162] The tunnel surface crack recognition module 103 is used to build a tunnel surface crack recognition model, analyze the images and point cloud data collected in the tunnel, identify the tunnel surface cracks, and dynamically adjust the inspection speed according to the detected crack information to complete the inspection task.
[0163] In detail, each module in the robot-based tunnel intelligent inspection system 100 in the embodiment of the present invention is used in the same manner as described above. Figure 1 The same technical means are used as the robot-based intelligent tunnel inspection method described in the text, and can produce the same technical effects, so I will not go into details here.
[0164] Embodiment 3:
[0165] like Figure 3 , is a schematic diagram of the structure of an electronic device for implementing a robot-based intelligent tunnel inspection method provided by an embodiment of the present invention.
[0166] The electronic device 1 may include a processor 10 , a memory 11 , a communication interface 13 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10 , such as a program 12 .
[0167] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may be used not only to store application software and various types of data installed in the electronic device 1, such as the code of the program 12, etc., but also to temporarily store data that has been output or is to be output.
[0168] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (such as the program 12 for implementing the robot-based intelligent tunnel inspection method), and calls the data stored in the memory 11 to execute various functions of the electronic device 1 and process data.
[0169] The communication interface 13 may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices, and to achieve connection and communication between internal components of the electronic device.
[0170] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.
[0171] Figure 3 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0172] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
[0173] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.
[0174] The program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:
[0175] Based on the tunnel structure design, the robot's inspection path is calculated using a path planning algorithm to cover the entire tunnel and achieve rapid tunnel inspection.
[0176] The robot is equipped with a laser radar, a visual sensor and an inertial navigation system. The laser radar is used to obtain the three-dimensional point cloud data inside the tunnel, and the visual sensor is used to collect image information. The inertial navigation system is combined with the planned inspection path to collect images and build a panoramic map of the tunnel.
[0177] Construct a tunnel surface crack recognition model, analyze the images and point cloud data collected in the tunnel, and identify tunnel surface cracks;
[0178] The inspection speed is dynamically adjusted according to the detected crack information to complete the inspection task.
[0179] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figures 1 to 3 The description of the relevant steps in the corresponding embodiments will not be repeated here.
[0180] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.
[0181] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A robot-based intelligent tunnel inspection method, characterized in that: The method comprises: S1: Based on the tunnel structure design diagram, the robot's inspection path is calculated using a path planning algorithm to cover the entire tunnel and achieve rapid tunnel inspection, wherein graph search based on an extended node strategy is the main implementation method of the path planning; S2: The robot is equipped with a laser radar, a visual sensor, and an inertial navigation system. The laser radar is used to obtain the three-dimensional point cloud data inside the tunnel. The visual sensor is used to collect image information. The inertial navigation system is used to collect images according to the planned inspection path to build a panoramic map of the tunnel. S3: Construct a tunnel surface crack recognition model, analyze the images and point cloud data collected in the tunnel, and identify tunnel surface cracks, wherein a deep neural network integrating an attention mechanism is the main implementation method for identifying tunnel surface cracks; S4: Dynamically adjust the inspection speed according to the detected crack information to complete the inspection task.
2. A robot-based intelligent tunnel inspection method as claimed in claim 1, characterized in that: In the step S1, the inspection path of the robot is calculated using a path planning algorithm based on the tunnel structure design drawing, including: S11: Acquire tunnel structure design drawing data, where the design drawing data is an electronic map in a CAD file format; S12: performing rasterization processing on the acquired map data to obtain rasterized map data; S13: representing the rasterized map as a graph data structure, where a node represents a location point in the tunnel, and an edge represents a connectivity relationship between two location points; S14: Perform path search on the graph structure to obtain a coarse-grained inspection path, wherein the graph search based on the extended node strategy is the main implementation method of the path search, and the specific process includes: S141: Add the starting node to the list of nodes to be expanded, and initialize the distance function value and the heuristic function value; S142: Select the node with the minimum heuristic function value among the nodes to be expanded according to the heuristic function calculation, calculate and update the distance function of the adjacent nodes according to the adjacent nodes of the node, and add the updated node to the list of nodes to be expanded. The distance update calculation formula is: d(v)=d(u)+c(u,v) in: d(v) represents the distance from the starting node to node v, d(u) represents the distance from the starting node to node u, and c(u,v) represents the distance from node u to node v; The heuristic function calculation formula is: in: h(v) represents the estimated cost from node v to the target node; (x v ,y v ) represents the coordinates of node v; (x goal ,y goal ) represents the coordinates of the target node; S143: If the target node is found, the start node is found from the target node along the parent node pointer by backtracking to obtain the optimal path; otherwise, the process returns to step S142 to continue searching until the optimal path is found and returned as the coarse-grained inspection path; S15: Smoothing the coarse-grained inspection path obtained through searching, removing the jagged parts in the path, and smoothing the path to obtain a fine-grained inspection path.
3. A robot-based intelligent tunnel inspection method as claimed in claim 2, characterized in that: In the step S15, the coarse-grained inspection path obtained by searching is smoothed to remove the jagged parts in the path, so as to smooth the path to obtain a fine-grained inspection path, including: The coarse-grained inspection path obtained by searching is used as input to perform Bezier curve fitting to obtain a smooth curve as the fine-grained inspection path. The calculation formula is: in: B(t) represents a point on the Bezier curve; n represents the order of the Bezier curve; Indicates the number of combinations; t represents the point parameter, and its value range is [0,1]; P i Represents the coordinates of the i-th control point.
4. A robot-based intelligent tunnel inspection method as claimed in claim 1, characterized in that: In the step S2, the three-dimensional point cloud data inside the tunnel is obtained by using a laser radar, image information is collected by a visual sensor, and image collection is performed according to the planned inspection path in combination with an inertial navigation system to realize the construction of a panoramic map inside the tunnel, including: S21: Scanning the interior of the tunnel using a laser radar to obtain three-dimensional point cloud data, where the point cloud data includes coordinate information of various positions inside the tunnel; using a visual sensor to collect images inside the tunnel according to the planned inspection path and using an inertial navigation system to obtain position information of the images inside the tunnel; S22: Fusing the three-dimensional point cloud data acquired by the laser radar and the image information acquired by the visual sensor, the specific process includes: performing denoising and coordinate system conversion operations on the point cloud data acquired by the laser radar and the image acquired by the visual sensor to obtain preprocessed point cloud data and image data; extracting features from the preprocessed image and point cloud data, and matching them, using SIFT, SURF and ORB algorithms to extract features from the preprocessed image data; extracting features from the preprocessed point cloud data using curvature and normal vector attributes, matching the extracted point cloud features with image features through descriptor matching, and performing coordinate conversion between the matched feature points in the image and the point cloud so that they are represented in the same coordinate system; fusing the matched feature points to obtain fused data, and the feature point fusion is optimized by minimizing the reprojection error so that the distance between the projection of the point cloud feature in the image and the feature point in the actual image is minimized; S23: Using the fused data, a voxel grid method is used to perform three-dimensional reconstruction to obtain a panoramic map of the interior of the tunnel.
5. The robot-based intelligent tunnel inspection method according to claim 1, characterized in that: In the step S3, a tunnel surface crack recognition model is constructed, and images and point cloud data collected in the tunnel are analyzed to identify tunnel surface cracks, including: S31: Use the pre-trained ResNet model to extract features from the pre-processed tunnel surface image to obtain an image feature vector; S32: Use the PointNet network to extract features from the preprocessed point cloud data to obtain a point cloud feature vector; S33: fusing the extracted image feature vector and the point cloud feature vector to obtain a fused feature vector; S34: Construct a tunnel surface crack recognition model integrating an attention mechanism, wherein the model includes a convolutional layer, an attention module, a pooling module, and a fully connected layer. The detailed structure of the image crack recognition model integrating an attention mechanism is as follows: The convolution layer includes: convolution layer 1 and convolution layer 2, wherein the convolution kernel size of convolution layer 1 is 3*3, the number of output feature maps is 32, and the ReLU activation function is used; the convolution kernel size of convolution layer 2 is 3*3, the number of output feature maps is 64, and the ReLU activation function is used; The attention module obtains the final attention feature map by fusing spatial attention and channel attention; The pooling window size of the pooling layer is 2*2, which is used to reduce the size of the feature map; The fully connected layer includes a fully connected layer 1 and a fully connected layer 2, wherein the number of neurons in the fully connected layer 1 is 512, and a ReLU activation function is used; the Dropout layer is used to prevent overfitting and randomly discard some neurons; the number of neurons in the fully connected layer 2 is 2, and a Softmax activation function is used to output the probability distribution of whether there is a crack; S35: Parameter optimization is performed on the constructed tunnel surface crack identification model to obtain an optimal model instance for tunnel crack identification, wherein a whale algorithm integrating multiple population strategies is an implementation method for the parameter optimization.
6. A robot-based intelligent tunnel inspection method as claimed in claim 5, characterized in that: In the step S35, the constructed tunnel surface crack identification model is optimized for parameters to obtain an optimal model instance for tunnel crack identification, including: The whale algorithm integrating multiple population strategies introduces multiple independent whale groups and shares information among them to increase the diversity of global search. The specific process includes: S351: Generate a preset number of independent whale groups, each group contains a preset number of whale individuals, and randomly initialize the position and speed of each whale individual; S352: Calculate the fitness of each individual whale, and determine the fitness value of each individual according to the objective function of the tunnel surface crack recognition model; S353: Perform an update operation on each whale group, including: updating the position and speed of individual whales, and obtaining new positions and speeds through formula calculation; updating the optimal solution in the group, and the position update formula of individual whales is: x i (t+1)=x i (t)+v i (t+1) The speed update formula of individual whales is: v i (t+1)=v i (t)+A·C in: x i (t) represents the position of the i-th whale individual at iteration t of round t; v i (t) represents the speed of the i-th whale individual in the t-th iteration; A represents the acceleration coefficient; C represents the exploration coefficient; S354: Iterate steps S352 and S353 until the maximum number of iterations is reached and output the optimal model parameters.
7. The robot-based intelligent tunnel inspection method according to claim 1, characterized in that: In step S4, the inspection speed is dynamically adjusted according to the detected crack information to complete the inspection task, including: According to the crack information, the inspection speed is dynamically adjusted to speed up the inspection frequency of the crack area to ensure that more time and resources are used for the inspection of key areas. The calculation formula is: V new =V base +log(N) in: V base Indicates the basic inspection speed: N represents the number of tunnel cracks identified at the current position: V new Indicates the new inspection speed adjusted according to the crack information.
8. A robot-based intelligent tunnel inspection system, characterized in that: The system comprises: Inspection path planning module, which is used to calculate the robot's inspection path based on the tunnel structure design drawing using the path planning algorithm to cover the entire tunnel and achieve rapid tunnel inspection; The tunnel map construction module is used to carry laser radar, visual sensor and inertial navigation system on the robot, use laser radar to obtain three-dimensional point cloud data inside the tunnel, collect image information through visual sensor, and combine with the inertial navigation system to collect images according to the planned inspection path to realize the construction of a panoramic map of the tunnel; A tunnel surface crack recognition module is used to construct a tunnel surface crack recognition model, analyze the images and point cloud data collected in the tunnel, identify tunnel surface cracks, dynamically adjust the inspection speed according to the detected crack information, and complete the inspection task to realize a robot-based tunnel intelligent inspection method as described in any one of claims 1 to 7.
Citation Information
Cited By
Route planning method and device based on subway tunnel inspection
CN119290004A
Unmanned aerial vehicle system for transformer substation inspection
CN119596993A
