Three-dimensional point cloud pipeline detection method based on space-time perception SLAM

Abstract

The invention discloses a three-dimensional point cloud pipeline detection method based on space-time perception SLAM, and the method mainly comprises the steps: collecting the multi-time-sequence real-time features of a pipeline through a sensor, including the flow, speed, material, impurities and foreign matters in the pipeline; the graph attention network carries out aggregation operation on the multi-time-sequence real-time features so as to realize adaptive distribution of the multi-time-sequence real-time features of the pipeline; carrying out forgetting and memory storage on the multi-time sequence real-time features of the pipeline by adopting an LSTM (Long Short Term Memory) long-short term neural network; compensation operation is carried out on errors caused by light change, object shielding, motion blur and noise problems when the point cloud is generated; performing back-end local optimization on the sparse point cloud map by adopting a stochastic gradient descent algorithm; and generating a three-dimensional point cloud map, and detecting and monitoring the water supply and drainage pipeline according to the three-dimensional point cloud map. According to the method, the operation state of the pipe network system at the next moment can be predicted through space-time perception, and leakage detection is effectively controlled.

Description

Technical Field

[0001] The present invention belongs to the technical field of pipeline detection, and in particular relates to a three-dimensional point cloud pipeline detection method based on spatiotemporal perception SLAM. Background Art

[0002] With the rapid development of cities, underground pipelines have become the capillaries of urban systems. A reliable pipeline network system is conducive to improving the level of urban infrastructure, and its safe, stable and reliable operation has a far-reaching impact on the city. At present, water supply and drainage pipelines are often prone to leakage, which not only causes waste of water resources, but also pollutes the environment and affects people's daily life. Leak detection usually uses methods such as acoustic wave detection, infrared thermal imaging, pressure sensors, and pipeline CCTV detection, but the above methods are easily affected by the objective environment and require a high level of professionalism from the detection personnel.

[0003] With the rapid development of robotics, SLAM technology is increasingly being used in autonomous navigation, three-dimensional reconstruction, object recognition, and other fields. SLAM, short for Simultaneous Localization and Mapping, is a research field that targets two major technologies: positioning and mapping. At present, the mainstream SLAM technology applications are laser SLAM (based on laser radar) and visual SLAM (based on single / binocular cameras), which are mainly divided into filter-based SLAM and graph-based SLAM in terms of implementation. In complex environments, especially in narrow spaces such as pipelines, data collection is difficult for existing SLAM methods, resulting in incomplete three-dimensional reconstruction maps. Chinese patent CN115824208A discloses "A pipeline environment SLAM method based on multi-robot collaboration" and CN117635957A discloses "A visual SLAM method for underground pipelines based on point-line feature fusion". These methods are both methods for locating pipeline fault locations, which are mainly used to detect the spatial location of fault points and are applicable to smaller scales. Summary of the invention

[0004] The purpose of the present invention is to propose a SLAM three-dimensional point cloud pipeline detection method based on time and space perception to improve data acquisition capabilities, generate historical data graphs, detect and monitor pipelines according to three-dimensional point cloud maps, achieve effective leak control and detection, and ensure the safe and stable operation of water supply and drainage pipeline systems.

[0005] The technical solution of the present invention is as follows:

[0006] A three-dimensional point cloud pipeline detection method based on time-space perception SLAM uses a time-space perception SLAM to construct a three-dimensional point cloud map of the water supply and drainage pipeline, and detects and monitors the pipeline system. The main steps are:

[0007] (1) Collect multi-time real-time characteristics of the pipeline through sensors, including flow rate, velocity, material, and impurities in the pipeline;

[0008] (2) The graph attention network aggregates the multiple time series real-time features to achieve adaptive allocation of the multiple time series real-time features of the pipeline;

[0009] (3) Using LSTM long short time neural network to forget and memorize the multi-time series real-time features of the pipeline;

[0010] (4) Compensate for errors caused by light changes, object occlusion, motion blur, and noise when generating point clouds;

[0011] (5) Using stochastic gradient descent algorithm to perform backend local optimization on sparse point cloud maps;

[0012] (6) Generate a three-dimensional point cloud map, and detect and monitor the water supply and drainage pipelines based on the three-dimensional point cloud map.

[0013] The present invention has the following preferred design:

[0014] Since pipeline flow data is typical spatiotemporal data, it is closely related to spatial position and time changes. The inspection robot SLAM technology uses visual sensors to collect feature data. For the analysis and processing of spatiotemporal data, this paper proposes to use LSTM long short-term neural network and graph attention network to model the two dimensions of time and space.

[0015] In step (2) of the present invention, the pipeline geometry is recorded as G = (V, A), where V represents the sensor node set, |V| = N, and A = {A k |k=1,2,...,N} represents the adjacency matrix of different relationships between nodes, N is a positive integer;

[0016] The multivariate flow data at a certain time step can be expressed as Represents the information observed by node i at time step t, including flow, speed, material, and impurities;

[0017] In order to better allocate weights, the graph attention network activation function adopts the Leaky ReLu activation function, so as to adaptively allocate the weights of each multi-time series real-time feature, which is specifically expressed as follows:

[0018]

[0019] Where, X t ' represents the output feature information, σ represents the activation factor, α represents the function of calculating the correlation of adjacent features, α ijrepresents the attention coefficient, N represents the number of features, and v j 、v i represents adjacent features, W is the weight parameter matrix, X i represents the characteristics of node i, X j Represents the characteristics of node j.

[0020] In step (3) of the present invention, the LSTM long short time neural network converts the time series data X output by the graph attention network into t 'Memory is stored or forgotten for a long time, allowing the persistence of information and the memory of multi-step time series data;

[0021] The long short-term memory neural network saves long-term memory at the next state h. The input is the time series data group after the graph attention network aggregation operation. At time t, the input x t With the previous state h t-1 Multiply their weights respectively and add them together to get the state h at the next moment t , using forward propagation, the principle is: h t =x t W xh +h t-1 W hh ,

[0022] Among them, W xh For input x t and the state quantity h at the previous time t-1 The connection weight, W hh Indicates the connection weight at the last moment;

[0023] The update gate of the long short-term memory neural network memorizes and forgets the input time series data to realize the screening of input features. The update gate is actually an activation function. The input is a vector and the output is a real vector between 0 and 1. When the gate is 0, no vector can pass through. When the gate is 1, any vector can pass through. Therefore, the update gate is controlled to choose to forget or save the feature information: X t '=σ(W i (h t-1 ,x t )+b i ),

[0024] The update phase uses the update gate z t Forgetting and remembering at the same time:

[0025] g t =(1-z t )(W f (h t-1 ,x t )+b f )+z ttanh(W g (h t-1 ,x t )+b g ),

[0026] Finally, the activation function is used to get the output of the spatiotemporal graph neural network at time t:

[0027] o′ t =σ(W o (h t-1 ,x t )+b o ),

[0028] In the formula, X′ t , g t , o t Respectively represent input, update, and output; W i , W f , W g , W o is the weight; b i 、b f 、b g 、b o is the bias vector; σ is the sigmoid activation function.

[0029] The compensation operation performed when generating the point cloud in step (4) of the present invention includes: when the sensor collects feature information, the error and deformation in the point cloud data caused by light changes, object occlusion, motion blur, noise and other problems are used to improve the accuracy and consistency of the point cloud data by using a point cloud compensation method. The point cloud compensation method projects the dynamic feature information into a point cloud of corresponding color. The inspection robot tracks and updates the current state of the node pipeline, and constructs a three-dimensional dynamic map of the surrounding environment according to the positioning position. For the error and deformation in the point cloud data, the output of the pipeline at the next moment is predicted by the following formula, and a compensation operation is performed. t+1 =(o′ t-1 +o′ t ) / 2.

[0030] In step (5) of the present invention, the point cloud map generated by optimizing the front end of the SLAM framework is a sparse point cloud map, which cannot fully represent the description of environmental information. Therefore, the three-dimensional point cloud map is locally optimized by using a stochastic gradient descent algorithm on the back end: In the formula, η is the learning rate, Represents the gradient.

[0031] In step (6) of the present invention, a three-dimensional point cloud map is established through the SLAM framework to effectively detect and monitor the water supply and drainage pipelines. By analyzing historical data, the area and time where leakage will occur at the next moment are predicted, abnormal situations are discovered in time and early warning processing is carried out, so as to achieve effective leak control and detection, improve water supply and drainage efficiency, and ensure the safe and stable operation of the water supply and drainage pipeline system.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The feature points are collected and aggregated through the graph attention network to improve the data collection capability. The feature points are saved for a long time through the LSTM long-short time neural network to generate a historical data graph. Finally, the pipeline is inspected and monitored based on the three-dimensional point cloud map to achieve effective leak control and detection, and ensure the safe and stable operation of the water supply and drainage pipeline system.

[0034] 2. The present invention monitors the pipe network system in the entire area by constructing a time-space perception three-dimensional point cloud map. Through time-space perception, the operating status of the pipe network system at the next moment can be predicted, and abnormal situations that are about to occur can be early warned and the pipelines can be inspected. This can achieve effective leak control and detection in large-scale pipe networks and ensure the safe and stable operation of the water supply and drainage pipe system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A basic framework diagram of a three-dimensional point cloud pipeline detection method for spatiotemporal perception SLAM provided by the present invention;

[0036] Figure 2 This is a graph attention network structure diagram of the present invention;

[0037] Figure 3 This is the structure diagram of the LSTM long-short time neural network of the present invention;

[0038] Figure 4 It is a flow chart of the spatiotemporal perception SLAM algorithm of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0040] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0041] It should be understood that the present invention can be implemented in different forms and should not be interpreted as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to make the disclosure thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0042] The purpose of the terms used herein is only to describe specific embodiments and is not intended to be limiting of the present invention. When used herein, the singular forms "one", "an" and "said / the" are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "consisting of" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0043] In order to fully understand the present invention, a detailed structure will be proposed in the following description to illustrate the technical solution proposed by the present invention. The optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementations.

[0044] A three-dimensional point cloud pipeline detection method based on time-space perception SLAM uses a time-space perception SLAM to construct a three-dimensional point cloud map of the water supply and drainage pipeline, and detects and monitors the pipeline system. The main steps are:

[0045] (1) Collect multi-time real-time characteristics of the pipeline through sensors, including flow rate, velocity, material, and impurities in the pipeline;

[0046] (2) The graph attention network aggregates the multiple time series real-time features to achieve adaptive allocation of the multiple time series real-time features of the pipeline;

[0047] (3) Using LSTM long short time neural network to forget and memorize the multi-time series real-time features of the pipeline;

[0048] (4) Compensate for errors caused by light changes, object occlusion, motion blur, and noise when generating point clouds;

[0049] (5) Using stochastic gradient descent algorithm to perform backend local optimization on sparse point cloud maps;

[0050] (6) Generate a three-dimensional point cloud map, and detect and monitor the water supply and drainage pipelines based on the three-dimensional point cloud map.

[0051] Since pipeline flow data is typical spatiotemporal data, it is closely related to spatial position and time changes. The inspection robot SLAM technology uses visual sensors to collect feature data. For the analysis and processing of spatiotemporal data, this paper proposes to use LSTM long short-term neural network and graph attention network to model the two dimensions of time and space.

[0052] In step (2), the pipeline geometry is denoted as G = (V, A), where V represents the set of sensor nodes, |V| = N, and A = {A k |k=1,2,...,N} represents the adjacency matrix of different relationships between nodes, N is a positive integer;

[0053] The multivariate flow data at a certain time step can be expressed as Represents the information observed by node i at time step t, including flow, speed, material, and impurities;

[0054] In order to better allocate weights, the graph attention network activation function adopts the Leaky ReLu activation function, so as to adaptively allocate the weights of each multi-time series real-time feature, which is specifically expressed as follows:

[0055]

[0056] In the formula, X t ' represents the output feature information, σ represents the activation factor, α represents the function of calculating the correlation of adjacent features, α ij represents the attention coefficient, N represents the number of features, and v j 、v i represents adjacent features, W is the weight parameter matrix, X i represents the characteristics of node i, X j Represents the characteristics of node j.

[0057] In step (3), the LSTM long short-term neural network converts the time series data X output by the graph attention network into t 'Memory is stored or forgotten for a long time, allowing the persistence of information and the memory of multi-step time series data;

[0058] The long short-term memory neural network saves long-term memory at the next state h. The input is the time series data group after the graph attention network aggregation operation. At time t, the input x t With the previous state h t-1 Multiply their weights and add them together to get the state h at the next moment. t, using forward propagation, the principle is: h t =x t W xh +h t-1 W hh ,

[0059] Among them, W xh For input x t and the state quantity h at the previous time t-1 The connection weight, W hh Indicates the connection weight at the last moment;

[0060] The update gate of the long short-term memory neural network memorizes and forgets the input time series data to realize the screening of input features. The update gate is actually an activation function. The input is a vector and the output is a real vector between 0 and 1. When the gate is 0, no vector can pass through. When the gate is 1, any vector can pass through. Therefore, the update gate is controlled to choose to forget or save the feature information: X t '=σ(W i (h t-1 ,x t )+b i ),

[0061] The update phase uses the update gate z t Forgetting and remembering at the same time:

[0062] g t =(1-z t )(W f (h t-1 ,x t )+b f )+z t tanh(W g (h t-1 ,x t )+b g ),

[0063] Finally, the activation function is used to get the output of the spatiotemporal graph neural network at time t:

[0064] o′ t =σ(W o (h t-1 ,x t )+b o ),

[0065] In the formula, X′ t , g t , o t Respectively represent input, update, and output; W i , W f , W g , W ois the weight; b i 、b f 、b g 、b o is the bias vector; σ is the sigmoid activation function.

[0066] The compensation operation performed when generating the point cloud in step (4) includes: when the sensor collects feature information, the point cloud data is compensated for errors and deformations caused by light changes, object occlusion, motion blur, noise, etc. The accuracy and consistency of the point cloud data are improved by the point cloud compensation method. The point cloud compensation method projects the dynamic feature information into a point cloud of corresponding color. The inspection robot tracks and updates the current state of the node pipeline, and constructs a three-dimensional dynamic map of the surrounding environment according to the positioning position. For the errors and deformations in the point cloud data, the output of the pipeline at the next moment is predicted by the following formula, and the compensation operation is performed, o′ t+1 =(o′ t-1 +o′ t ) / 2.

[0067] In step (5), the point cloud map generated by optimizing the front end of the SLAM framework is a sparse point cloud map, which cannot fully represent the description of the environmental information. Therefore, the three-dimensional point cloud map is locally optimized by the stochastic gradient descent algorithm on the back end: o′ t+1 ←o′ t+1 -η▽o′ t , where η is the learning rate and ▽ represents the gradient.

[0068] In one embodiment, Figure 1 This is a diagram of the visual attention network structure of the method of the present invention. The attention network extracts and aggregates the image features collected by the sensor, and stores and forgets them through the LSTM long short-term neural network. However, due to errors such as light changes, object occlusion, motion blur, and noise, compensation operations are required when generating point cloud maps. The point cloud map obtained by optimizing the SLAM front end is a sparse point cloud map, and the three-dimensional point cloud map is locally optimized on the back end using a random gradient descent algorithm. Finally, a three-dimensional point cloud map consistent with the environment is constructed.

[0069] Figure 2 This is a graph attention network structure diagram of this embodiment. First, the inspection robot's sensor collects pipeline features (flow, speed, material, foreign matter and impurities), and divides the entire graph data structure from the inside to the outside into k layers with the node as the center; then the graph attention network iteratively aggregates adjacent nodes layer by layer, splices the aggregated adjacent node information with the previous layer information, and adaptively allocates weights through the activation function; finally, the feature information after the aggregation operation is output.

[0070] Figure 3It is the LSTM long short-term neural network of this embodiment. The long short-term memory neural network memorizes and forgets the input time series data through the update gate to achieve the screening of input features; the input gate represents x at time t t Save to memory cell state c t The output gate updates the memory cell state c t , and get the final cell output h t .

[0071] Figure 4 It is a calculation flow chart of the spatiotemporal perception SLAM algorithm of the method of the present invention. The following is a description of the accompanying drawings:

[0072] Step 1: The sensor collects multi-time series real-time characteristics of the pipeline, including flow rate, speed, material, impurities and foreign matter in the pipeline;

[0073] Step 2: The graph attention network aggregates the above features to achieve adaptive allocation of pipeline features;

[0074] Step 3: The LSTM long short-term neural network memorizes and forgets pipeline features;

[0075] Step 4: For errors such as light changes, object occlusion, motion blur, and noise, compensation operations need to be performed when generating the point cloud;

[0076] Step 5: Determine whether the historical characteristics are fitted;

[0077] Step 6: For the sparse point cloud map, use the stochastic gradient descent algorithm to perform backend local optimization;

[0078] Step 7: Generate 3D point cloud map;

[0079] Step 8: Detect and monitor the pipeline based on the 3D point cloud map.

[0080] The present invention establishes a three-dimensional point cloud map through the SLAM framework to effectively detect and monitor the water supply and drainage pipelines. It also predicts the area and time where leakage will occur at the next moment through the analysis of historical data, promptly discovers abnormal situations and conducts early warning processing, thereby achieving effective leak control and detection, improving water supply and drainage efficiency, and ensuring the safe and stable operation of the water supply and drainage pipeline system.

[0081] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A three-dimensional point cloud pipeline detection method based on spatiotemporal perception SLAM, characterized in that: A spatiotemporal perception SLAM is used to construct a 3D point cloud map of the water supply and drainage pipelines, and to detect and monitor the pipeline system. The main steps are: (1) Collect multi-time real-time characteristics of the pipeline through sensors, including flow rate, velocity, material, and impurities in the pipeline; (2) The graph attention network aggregates the multiple time series real-time features to achieve adaptive allocation of the multiple time series real-time features of the pipeline; (3) Using LSTM long short time neural network to forget and memorize the multi-time series real-time features of the pipeline; (4) Compensate for errors caused by light changes, object occlusion, motion blur, and noise when generating point clouds; (5) Using stochastic gradient descent algorithm to perform backend local optimization on sparse point cloud maps; (6) Generate a three-dimensional point cloud map, and detect and monitor the water supply and drainage pipelines based on the three-dimensional point cloud map.

2. The three-dimensional point cloud pipeline detection method of spatiotemporal perception SLAM according to claim 1 is characterized in that: In step (2), the pipeline geometry is denoted as G = (V, A), where V represents the set of sensor nodes, |V| = N, and A = {A k |k=1,2,...,N} represents the adjacency matrix of different relationships between nodes, N is a positive integer; The multivariate flow data at a certain time step can be expressed as Represents the information observed by node i at time step t, including flow, speed, material, and impurities; The activation function of the graph attention network adopts the Leaky ReLu activation function to adaptively allocate the weights of each multi-time series real-time feature, which is specifically expressed as follows: Where, X t ' represents the output feature information, σ represents the activation factor, α represents the function of calculating the correlation of adjacent features, α ij represents the attention coefficient, N represents the number of features, and v j 、v i represents adjacent features, W is the weight parameter matrix, X i represents the characteristics of node i, X j Represents the characteristics of node j.

3. The three-dimensional point cloud pipeline detection method of spatiotemporal perception SLAM according to claim 2 is characterized in that: In step (3), the LSTM long short time neural network converts the time series data X output by the graph attention network into t 'Memory is stored or forgotten for a long time, allowing the persistence of information and the memory of multi-step time series data; The long short-term memory neural network saves long-term memory at the next state h. The input is the time series data group after the graph attention network aggregation operation. At time t, the input x t With the previous state h t-1 Multiply their weights respectively and add them together to get the state h at the next moment t , using the forward propagation method, the principle is: h t =x t W xh +h t-1 W hh , Among them, W xh For input x t and the state quantity h at the previous time t-1 The connection weight, W hh Indicates the connection weight at the last moment; The update gate of the long short-term memory neural network memorizes and forgets the input time series data to achieve the screening of input features. The update gate is an activation function. The input is a vector and the output is a real vector between 0 and 1. When the gate is 0, no vector can pass through. When the gate is 1, any vector can pass through. The update gate is controlled to choose to forget or save the feature information: X t '=σ(W i (h t-1 ,x t )+b i ), The update phase uses the update gate z t Forgetting and remembering at the same time: g t =(1-z t )(W f (h t-1 ,x t )+b f )+z t tanh(W g (h t-1 ,x t )+b g ), Finally, the activation function is used to get the output of the spatiotemporal graph neural network at time t: oh t =σ(W o (h t-1 ,x t )+b o ), In the formula, X′ t , g t , o′ t Respectively represent input, update, and output; W i , W f , W g , W o is the weight; b i , b f , b g , b o is the bias vector; σ is the sigmoid activation function.

4. The three-dimensional point cloud pipeline detection method of spatiotemporal perception SLAM according to claim 3 is characterized in that: The compensation operation performed when generating the point cloud in step (4) includes: when the sensor collects feature information, the point cloud data is compensated for errors and deformations caused by light changes, object occlusion, motion blur, noise, etc., and the accuracy and consistency of the point cloud data are improved by the point cloud compensation method. The point cloud compensation method projects the dynamic feature information into a point cloud of corresponding color. The inspection robot tracks and updates the current state of the node pipeline, and constructs a three-dimensional dynamic map of the surrounding environment according to the positioning position. For the errors and deformations in the point cloud data, the output of the pipeline at the next moment is predicted by the following formula, and the compensation operation is performed, o′ t+1 =(o′ t-1 +o′ t ) / 2.

5. The three-dimensional point cloud pipeline detection method of spatiotemporal perception SLAM according to claim 4 is characterized in that: In step (5), the point cloud map generated by optimizing the front end of the SLAM framework is a sparse point cloud map, which cannot fully represent the description of environmental information. Therefore, the three-dimensional point cloud map is locally optimized on the back end using a random gradient descent algorithm: In the formula, η is the learning rate, Represents the gradient.

6. The three-dimensional point cloud pipeline detection method of spatiotemporal perception SLAM according to any one of claims 1 to 5, characterized in that: In step (6), a three-dimensional point cloud map is established through the SLAM framework to effectively detect and monitor the water supply and drainage pipelines, and through the analysis of historical data, the area and time where leakage will occur at the next moment are predicted, abnormal situations are discovered in time and early warning processing is carried out.

Images