An obstacle detection method and system for a wheeled inspection robot of an unmanned intelligent power station based on multi-sensor fusion

Abstract

The application provides a kind of obstacle detection method and system of unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion, belongs to the technical field of obstacle detection.A kind of detection method steps are: first, obtain the sensor data of inspection robot, type conversion and coordinate transformation are carried out to the depth data therein, finally, depth data and laser radar data are fused using BP neural network algorithm, to obtain more accurate obstacle information;A kind of detection system, comprising data acquisition module, the depth data and laser radar point cloud data of the region in front of robot are collected, first data processing module is used to convert depth data into point cloud data, second data processing module, coordinate transformation is carried out to the depth data processed, data fusion module, depth data and point cloud data are fused using BP neural network algorithm, to obtain more accurate obstacle information.The beneficial effects of the application are: it can improve the range and precision of obstacle detection.

Description

Technical Field

[0001] This invention belongs to the field of obstacle detection technology, specifically relating to an obstacle detection method and system for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion. Background Technology

[0002] Power station equipment is a crucial component in ensuring the safe and stable operation of the power grid. However, power station equipment is often subjected to erosion from the natural environment, such as strong winds, sunlight, and rain. This accelerates the aging of the equipment, increases the failure rate, and seriously affects the safety and stability of the power grid. Therefore, it is essential to conduct regular inspections of power stations to eliminate potential hazards in advance.

[0003] The traditional method of power plant inspection is manual inspection, which is not only inefficient and costly, but also cannot guarantee the safety of inspection personnel. In case of severe weather, the difficulty of inspection will be further increased. Therefore, the use of safer and more efficient inspection robots has gradually become a research hotspot.

[0004] During the inspection process, inspection robots inevitably need to avoid obstacles, so obstacle detection is crucial. However, the commonly used detection sensors are limited by their detection range and accuracy, which cannot meet the needs of inspection. Summary of the Invention

[0005] To address the problems mentioned above, this invention provides an obstacle detection method and system for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion, which can overcome the limitations of a single sensor and improve the detection range and accuracy.

[0006] To achieve the above objectives, the present invention provides the following technical solution: Firstly, it provides an obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion, the method comprising the following steps:

[0007] S1. Acquire sensor data from the inspection robot, including LiDAR and depth camera data;

[0008] S2. Convert the depth data from the depth camera into point cloud data, and transform the converted depth data from the depth camera coordinate system to the lidar coordinate system.

[0009] S3. Use the BP neural network algorithm to fuse depth camera data and LiDAR data to obtain more accurate obstacle information.

[0010] In the preferred embodiment, in step S1, the lidar selected is the SLAMTEC laser scanning ranging radar.

[0011] The depth camera used is the Lebe medium-light depth camera.

[0012] Furthermore, in step S2, the conversion between depth data and point cloud data involves the conversion between the image coordinate system O-xyz and the world coordinate system O. w -x w y w z w The transformation relationship is as follows:

[0013]

[0014] Where f is the camera focal length and d is the depth value.

[0015] Furthermore, in step S2, the transformation relationship between the depth camera coordinate system and the lidar coordinate system is as follows:

[0016] With depth camera optical center O c Establish a depth camera coordinate system O with the origin. c -x c y c z c With lidar center O r Establish a lidar coordinate system O with the origin. c -x r y r z r O c With O r The height difference is H, and the lateral distance is L;

[0017] For point P, its coordinates in the depth camera coordinate system are (x... c y c , z c Its coordinates in the lidar coordinate system are (x r y r , z r The two have the following relationship:

[0018]

[0019] Rewriting the above equation in matrix form, we have:

[0020]

[0021] Furthermore, in step S3, the BP neural network algorithm includes an input layer, a hidden layer, and an output layer. The input layer includes depth camera data and LiDAR data, the hidden layer includes m1 neurons, and the output layer outputs updated obstacle information.

[0022] The steps of updating obstacle information using the BP neural network algorithm are as follows:

[0023] S1 imports input variables, with x being the i-th neuron in the input layer.i Here, x1 represents depth camera data, and x2 represents LiDAR data; together, they form the input vector X = (x1, x2). T ;

[0024] S2 calculates h of the i-th neuron in the hidden layer. i input value and output value The calculation process is as follows:

[0025]

[0026] Where n is the number of neurons in the input layer. b1 is the hidden layer weight, f is the hidden layer bias, and f is the hidden layer weight. in (·) is the activation function of the hidden layer;

[0027] S3 calculates the input value y of neuron y in the output layer. in and output value y out The calculation process is as follows:

[0028]

[0029] in, b1 is the output layer weight, b2 is the output layer bias, and f is the output layer weight. out (·) is the activation function of the output layer;

[0030] S4 outputs the value y when the set number of iterations is reached. out This refers to the updated obstacle information.

[0031] Furthermore, in step S3, the gradient descent method is used to train the neural network, and the iterative formula for the weights and biases of the output layer is as follows:

[0032]

[0033] Among them, w out b' and b2' are the output layer weights and biases to be updated after iteration, η is the learning rate, E(m) is the evaluation function, O(m) is the output value of the m-th output layer neuron, and O1(m) is the expected output value of the m-th output layer neuron.

[0034] We determine the magnitude of E(m). If E(m) ≥ 0.5, we update the weights and biases of the output layer, i.e., we set:

[0035]

[0036] Furthermore, in step S32, the activation function f of the hidden layer in (·) Choosing the hyperbolic tangent function, its expression is:

[0037]

[0038] Furthermore, in step S33, the activation function f of the hidden layer out (·) The sigmoid function is selected, and its expression is:

[0039]

[0040] Furthermore, the BP neural network algorithm is first optimized for its initial parameter values ​​using the fireworks algorithm;

[0041] The optimization steps are as follows:

[0042] S1) The parameters that need to be optimized in the selected BP neural network are: hidden layer weights and biases, output layer weights and biases, and learning rate.

[0043] S2) Encode the above parameters into individual firework forms, and determine each firework x fi The dimension D is:

[0044] D = n IW(1,1) +n b(1,1) +n IW(2,1) +n b(2,1) ;

[0045] Where, n IW (1,1) represents the number of weights between the input layer and hidden layers of the neural network, n. b (1,1) represents the number of thresholds for neurons in the hidden layer, n IW (2,1) is the number of weights between the hidden layer and the output layer of the neural network, n b (2,1) represents the number of thresholds for each output layer neuron, for each firework x. fi Perform random initialization, setting the dimension range of each firework to [-1, 1];

[0046] S3) The fitness value f(x) for each firework is calculated using the sum of squared errors. fi ):

[0047]

[0048] Where t is the expected output of the neural network parameters, y is the output value of the neural network parameters, and s is the number of neurons in the output layer;

[0049] S4) The blast radius of the fireworks explosion A i and the number of sparks S produced i for:

[0050]

[0051] Where N is the initial number of fireworks species, E r and E n These are parameters that control the explosion radius and quantity, f min and f max These are the minimum and maximum fitness, respectively, and ε is used to avoid extremely small real numbers with a denominator of 0;

[0052] S5) Randomly select a firework and perform Gaussian mutation operations on its various dimensions:

[0053] x fik =x fik g;

[0054] Where, x fik is the position of the i-th individual in the k-dimensional space, and g is a random number that follows a Gaussian distribution with mean 1 and variance 1;

[0055] Map fireworks that are outside the boundary area back to within the boundary area:

[0056]

[0057] in, These are the upper and lower bounds of the k-th dimension, respectively.

[0058] S6) Form a population set K from the initial fireworks, the explosion sparks, and the Gaussian mutation sparks. In each iteration, retain the optimal solution and select the remaining N-1 sparks based on the Euclidean distance between each firework and the optimal firework. The selection strategy is as follows:

[0059]

[0060] Where R(x) fi P(x) is the Euclidean distance between the remaining fireworks and the optimal fireworks. fi () is fireworks x fi The probability of being selected;

[0061] S7) The optimization ends when the number of iterations reaches the set value.

[0062] Secondly, a multi-sensor fusion-based obstacle detection system for an unmanned intelligent power station wheeled inspection robot is provided to implement the multi-sensor fusion-based obstacle detection method for an unmanned intelligent power station wheeled inspection robot as described above. The system includes a data acquisition module, a first data processing module, a second data processing module, and a data fusion module.

[0063] The data acquisition module uses a depth camera to collect depth data of the area in front of the robot and a lidar to collect point cloud data of the area in front of the robot, and sends the collected data to the first data processing module.

[0064] The first data processing module is used to receive the data collected by the data acquisition module, convert the depth data from the depth camera into point cloud data, and then send the data to the second data processing module.

[0065] The second data processing module is used to receive data from the first data processing module, transform the converted depth data from the depth camera coordinate system to the lidar coordinate system, and then send the data to the data fusion module.

[0066] The data fusion module receives data from the second data processing module and fuses the converted depth data with the point cloud data from the LiDAR using an optimized BP neural network algorithm to obtain more accurate obstacle information.

[0067] The beneficial effects of this invention are: by fusing data from multiple sensors, it is possible to overcome the detection limitations of a single sensor and improve the range and accuracy of obstacle detection. Attached Figure Description

[0068] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0069] Figure 1 This is a schematic flowchart of the detection method of the present invention;

[0070] Figure 2 This is a block diagram of the BP neural network fusion of the present invention;

[0071] Figure 3 This is a block diagram of the detection system of the present invention. Detailed Implementation

[0072] Example 1

[0073] like Figures 1-2 As shown in the figure, an obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion includes the following steps:

[0074] S1. Acquire sensor data from the inspection robot, including data from LiDAR and depth cameras;

[0075] S2. Convert the depth data from the depth camera into point cloud data. The conversion between depth data and point cloud data involves changing the image coordinate system O-xyz to the world coordinate system O. w -x w y w z w The transformation relationship is as follows:

[0076]

[0077] Where f is the camera focal length and d is the depth value.

[0078] The transformed depth data is converted from the depth camera coordinate system to the LiDAR coordinate system, with the depth camera optical center O... c Establish a depth camera coordinate system O with the origin. c -x c y c z c Establish a lidar coordinate system O with the lidar center Or as the origin. r -x r y r z r O c With O r The height difference is H, and the lateral distance is L;

[0079] For point P, its coordinates in the depth camera coordinate system are (x... c y c , z c Its coordinates in the lidar coordinate system are (x r y r , z r The two have the following relationship:

[0080]

[0081] Rewriting the above equation in matrix form, we have:

[0082]

[0083] S3. First, use the fireworks algorithm to optimize the initial values ​​of the parameters of the BP neural network. The process is as follows:

[0084] S1) The parameters that need to be optimized in the selected BP neural network are: hidden layer weights and biases, output layer weights and biases, and learning rate.

[0085] S2) Encode the above parameters into individual firework forms, and determine each firework x fi The dimension D is:

[0086] D = n IW(1,1) +n b(1,1) +n IW(2,1) +n b(2,1) ;

[0087] Where, n IW (1,1) represents the number of weights between the input layer and hidden layers of the neural network, n. b (1,1) represents the number of thresholds for neurons in the hidden layer, n IW (2,1) is the number of weights between the hidden layer and the output layer of the neural network, n b (2,1) represents the number of thresholds for each output layer neuron, for each firework x.fi Perform random initialization, setting the dimension range of each firework to [-1, 1];

[0088] S3) The fitness value f(x) for each firework is calculated using the sum of squared errors. fi ):

[0089]

[0090] Where t is the expected output of the neural network parameters, y is the output value of the neural network parameters, and s is the number of neurons in the output layer;

[0091] S4) The blast radius of the fireworks explosion A i and the number of sparks S produced i for:

[0092]

[0093] Where N is the initial number of fireworks species, E r and E n These are parameters that control the explosion radius and quantity, f min and f max These are the minimum and maximum fitness, respectively, and ε is used to avoid extremely small real numbers with a denominator of 0;

[0094] S5) Randomly select a firework and perform Gaussian mutation operations on its various dimensions:

[0095] x ik =x ik g;

[0096] Where, x fik is the position of the i-th individual in the k-dimensional space, and g is a random number that follows a Gaussian distribution with mean 1 and variance 1;

[0097] Map fireworks that are outside the boundary area back to within the boundary area:

[0098]

[0099] in, These are the upper and lower bounds of the k-th dimension, respectively.

[0100] S6) Form a population set K from the initial fireworks, the explosion sparks, and the Gaussian mutation sparks. In each iteration, retain the optimal solution and select the remaining N-1 sparks based on the Euclidean distance between each firework and the optimal firework. The selection strategy is as follows:

[0101]

[0102] Where R(x) fiP(x) is the Euclidean distance between the remaining fireworks and the optimal fireworks. fi () is fireworks x fi The probability of being selected;

[0103] S7) The optimization ends when the number of iterations reaches the set value;

[0104] The optimized parameter values ​​are used as the initial values ​​for the BP neural network to fuse the transformed depth camera data and LiDAR point cloud data. The steps are as follows:

[0105] S31 Imports input variables, with x being the i-th neuron in the input layer. i Here, x1 represents depth camera data, and x2 represents LiDAR data; together, they form the input vector X = (x1, x2). T ;

[0106] S32 calculates h of the i-th neuron in the hidden layer. i input value and output value The calculation process is as follows:

[0107]

[0108] Where n is the number of neurons in the input layer. b1 is the hidden layer weight, f is the hidden layer bias, and f is the hidden layer weight. in (·) is the activation function of the hidden layer;

[0109] f in (·) Choosing the hyperbolic tangent function, its expression is:

[0110]

[0111] S33 calculates the input value y of neuron y in the output layer. in and output value y out The calculation process is as follows:

[0112]

[0113] Where m1 is the number of neurons in the hidden layer. b1 is the output layer weight, b2 is the output layer bias, and f is the output layer weight. out (·) is the activation function of the output layer;

[0114] f out (·) The sigmoid function is selected, and its expression is:

[0115]

[0116] S34 outputs the value y when the set number of iterations is reached.out This refers to the updated obstacle information.

[0117] Furthermore, in step S3, the gradient descent method is used to train the neural network, and the iterative formula for the weights and biases of the output layer is as follows:

[0118]

[0119] Among them, w out b' and b2' are the output layer weights and biases to be updated after iteration, η is the learning rate, E(m) is the evaluation function, O(m) is the output value of the m-th output layer neuron, and O1(m) is the expected output value of the m-th output layer neuron.

[0120] We check the magnitude of E(m). When E(m) ≥ 0.5, we update the weights and biases of the output layer, that is, we set:

[0121]

[0122] In the preferred embodiment, in step 1, the lidar selected is the SLAMTEC laser scanning ranging radar.

[0123] The depth camera used is the Lebe medium-light depth camera.

[0124] Example 2

[0125] like Figure 3 As shown in the figure, in conjunction with Embodiment 1, a multi-sensor fusion-based obstacle detection system for unmanned intelligent power station wheeled inspection robots is provided to implement the multi-sensor fusion-based obstacle detection method for unmanned intelligent power station wheeled inspection robots described in the above embodiment. The system includes a data acquisition module, a first data processing module, a second data processing module, and a data fusion module.

[0126] The data acquisition module uses a depth camera to collect depth data of the area in front of the robot and a lidar to collect point cloud data of the area in front of the robot, and sends the collected data to the first data processing module.

[0127] The first data processing module is used to receive the data collected by the data acquisition module, convert the depth data from the depth camera into point cloud data, and then send the data to the second data processing module.

[0128] The second data processing module is used to receive data from the first data processing module, transform the converted depth data from the depth camera coordinate system to the lidar coordinate system, and then send the data to the data fusion module.

[0129] The data fusion module receives data from the second data processing module and fuses the converted depth data with the point cloud data from the LiDAR using an optimized BP neural network algorithm to obtain more accurate obstacle information.

[0130] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be defined as the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. An obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion, characterized in that: The method includes the following steps: S1. Acquire sensor data from the inspection robot, including LiDAR and depth camera data; S2. Convert the depth data from the depth camera into point cloud data, and transform the converted depth data from the depth camera coordinate system to the lidar coordinate system. S3. Use the BP neural network algorithm to fuse depth camera data and LiDAR data to obtain more accurate obstacle information; The BP neural network algorithm includes an input layer, a hidden layer, and an output layer. The input layer includes depth camera data and LiDAR data, and the hidden layer includes... m One neuron, the output layer outputs the updated obstacle information; The steps of updating obstacle information using the BP neural network algorithm are as follows: S31 imports input variables, input layer number i One neuron is ,in, For depth camera data, The input vector consists of LiDAR data and other data. X =( , ) T ; S32 Calculate the first hidden layer i one neuron h i input value and output value The calculation process is as follows: （1）； in, n The number of neurons in the input layer. For hidden layer weights, b 1 represents the hidden layer bias. f in (·) is the activation function of the hidden layer; S33 Calculate the neurons in the output layer y input value and output value The calculation process is as follows: （2）； in, For output layer weights, b 2 is the output layer bias. f out (·) is the activation function of the output layer; S34 When the set number of iterations is reached, output the value. This refers to the updated obstacle information; The BP neural network algorithm described above first uses the fireworks algorithm to optimize its initial parameter values; The optimization steps are as follows: S1) The parameters that need to be optimized in the selected BP neural network are: hidden layer weights and biases, output layer weights and biases, and learning rate. S2) Encode the above parameters into individual firework forms, and determine the characteristics of each firework. x fi Dimensions D for: （3）； in, n IW (1,1) represents the number of weights between the input layer and the hidden layer of the neural network. n b (1,1) is the number of thresholds for neurons in the hidden layer. n IW (2,1) represents the number of weights between the hidden layer and the output layer of the neural network. n b (2,1) is the number of thresholds for the output layer neurons, for each firework. x fi Perform random initialization, setting the dimension range of each firework to [-1, 1]; S3) The fitness value of each firework is calculated using the sum of squared errors. f ( x fi ): （4）； in, t It is the expected output of the neural network parameters. y These are the output values ​​of the neural network parameters. s It is the number of neurons in the output layer; S4) Explosion radius of fireworks A i and the number of sparks produced S i for: （5）； in, N This represents the initial number of fireworks species. E r and E n These are parameters that control the explosion radius and quantity. f min and f max These are the minimum and maximum fitness, respectively. It is used to avoid extremely small real numbers with a denominator of 0; S5) Randomly select a firework and perform Gaussian mutation operations on its various dimensions: （6）； in, It is the first i Individuals k Position on the dimension g It is a random number that follows a Gaussian distribution with a mean of 1 and a variance of 1; Map fireworks that are outside the boundary area back to within the boundary area: （7）； in, They are the first k The upper and lower bounds of a dimension. S6) Form a population set from the initial fireworks, the explosion sparks, and the Gaussian mutation sparks. K In each iteration, the optimal solution is retained, and the remaining solutions are selected based on the Euclidean distance between each firework and the optimal firework. N -1 spark, the selection strategy is as follows: （8）； in, R ( x fi ) is the Euclidean distance between the remaining fireworks and the best fireworks. P ( x fi Fireworks x fi The probability of being selected; S7) The optimization ends when the number of iterations reaches the set value.

2. The obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion according to claim 1, characterized in that: In step S1, the lidar selected is the Slammer laser scanning ranging radar. The depth camera used is the Lebe medium-light depth camera.

3. The obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion according to claim 1, characterized in that: In step S2, the conversion between depth data and point cloud data is the image coordinate system transformation. O - xyz With world coordinate system O w -x w y w z w The transformation relationship is as follows: （9）； in, f For camera focal length, d This represents the depth value.

4. The obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion according to claim 1, characterized in that: In step S2, the transformation relationship between the depth camera coordinate system and the lidar coordinate system is as follows: With depth camera optical center O c Establish a depth camera coordinate system with the origin. O c -x c y c z c With lidar center O r Establish a lidar coordinate system with the origin. O r -x r y r z r , O c and O r The height difference is H The horizontal distance is L ; For point P, its coordinates in the depth camera coordinate system are ( x c , y c , z c Its coordinates in the lidar coordinate system are ( x r , y r , z r The two have the following relationship: （10）； Rewriting equation (2) in matrix form, we have: （11）。 5. The obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion according to claim 1, characterized in that: In step S3, the gradient descent method is used to train the neural network, and the iterative formulas for the weights and biases of the output layer are as follows: （12）； in, These are the output layer weights and biases to be updated after iteration. It's the learning rate. E ( m ) is the evaluation function. O ( m ) is the first m Output values ​​of each output layer neuron. O 1( m ) is the first m The expected output value of each output layer neuron; right E(m The size of ) is used to determine when Sometimes, we update the weights and biases of the output layer, that is: （13）。 6. The obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion according to claim 1, characterized in that: In step S32, the activation function of the hidden layer f in (·) Choosing the hyperbolic tangent function, its expression is: （14）。 7. The obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion according to claim 1, characterized in that: In step S33, the activation function of the hidden layer f out (·) The sigmoid function is selected, and its expression is: （15）。 8. An obstacle detection system for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion, used to implement the obstacle detection method for an unmanned intelligent power station wheeled inspection robot based on multi-sensor fusion as described in any one of claims 1 to 7, characterized in that: It includes a data acquisition module, a first data processing module, a second data processing module, and a data fusion module; The data acquisition module uses a depth camera to collect depth data of the area in front of the robot and a lidar to collect point cloud data of the area in front of the robot, and sends the collected data to the first data processing module. The first data processing module is used to receive the data collected by the data acquisition module, convert the depth data from the depth camera into point cloud data, and then send the data to the second data processing module. The second data processing module is used to receive data from the first data processing module, transform the converted depth data from the depth camera coordinate system to the lidar coordinate system, and then send the data to the data fusion module. The data fusion module receives data from the second data processing module and fuses the converted depth data with the point cloud data from the LiDAR using an optimized BP neural network algorithm to obtain more accurate obstacle information.

Images