An autonomous navigation system and method for a blast furnace scene based on point cloud fusion

Abstract

The application discloses a kind of based on point cloud fusion's tuyere scene autonomous navigation system and method, belong to blast furnace production process control technical field.It includes: robot trolley, robot trolley is sequentially provided with 16 line laser radar, IMU inertial measuring instrument, depth camera, holder, temperature and humidity sensor, gas sensor, sound intensity sensor and database server.The present application solves the problem that existing artificial handheld temperature measuring gun is used to measure temperature at tuyere, which causes greater damage to the human body by the environment, the point cloud fusion technology is used to better obtain the tuyere and obstacle information, and the odometer fusion algorithm is used to give the robot trolley more accurate coarse positioning;Through IMU inertial measuring instrument, the rotation angle, horizontal angle information of robot trolley is collected, high-precision positioning work is completed, and accurate temperature measurement of temperature measurement point is ensured;Using positioning autonomous navigation scheme ensures that the robot trolley accurately reaches the temperature measurement point in high temperature and high pressure environment, and ensures high frequency work every day.

Description

Technical Field

[0001] This invention relates to the field of blast furnace production process control technology, specifically to an autonomous navigation system and method for tuyere scenes based on point cloud fusion. Background Technology

[0002] The tuyere is located on the blast furnace in an ironmaking plant. High-temperature hot air blown into the tuyere oxidizes and burns the coke at the bottom of the furnace, producing CO. As the CO rises at high temperatures, it reduces iron, which was originally in oxide form. This is a crucial step in steelmaking and ironmaking. The primary material for blast furnace tuyeres is high-purity copper. To consider strength, rigidity, and crack resistance, the material is available in forged, rolled copper plate, and cast states. Rolled copper plate tuyeres are lightweight and inexpensive, but if the wall thickness is too thin, insufficient rigidity makes them prone to deformation. Because blast furnace tuyeres are kept at a low temperature by water cooling to maintain strength and rigidity, the purity of the copper is paramount.

[0003] The traditional method of checking air vents involves manually measuring the temperature of the air vents on-site with a handheld temperature gun. However, the on-site environment is quite harmful to the human body, thus reducing the feasibility of current on-site air vent temperature measurement.

[0004] A search revealed a robot recognition and navigation system based on the ROS system, disclosed in publication number CN217687263U. This solution runs the ROS operating system on an industrial control computer and integrates multiple sensors, including image recognition, UWB positioning, LiDAR, infrared detection, and ultrasonic detection. The industrial control computer then controls the robot chassis to move. Software design related to target recognition, autonomous obstacle avoidance, autonomous localization, mapping, and path planning is implemented on the ROS system to achieve robot recognition and autonomous navigation. However, the radar and sensor fusion methods used in this invention are different, making its autonomous temperature monitoring on the wind-prone platform more intelligent. Summary of the Invention

[0005] The purpose of this invention is to provide an autonomous navigation system and method for windy scenes based on point cloud fusion. By employing point cloud fusion technology, it can better acquire information about wind vents and obstacles, and use an odometry fusion algorithm to provide a more accurate coarse positioning for the robot vehicle. By collecting the rotation angle and horizontal angle information of the robot vehicle through an IMU inertial measurement unit, it can complete high-precision positioning and ensure accurate temperature measurement of the temperature measurement point. The autonomous navigation scheme ensures that the robot vehicle can accurately reach the temperature measurement point in a high-temperature and high-pressure environment, ensuring high-frequency operation every day, thus solving the problems mentioned in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An autonomous navigation system for windy scenes based on point cloud fusion includes a robot car, characterized in that: the robot car is used for autonomous navigation to the temperature measurement point in the windy scene, and the robot car is sequentially equipped with a 16-line lidar, an IMU inertial measurement unit, a depth camera, a gimbal, a temperature and humidity sensor, a gas sensor, a sound intensity sensor, and a database server. The database server is loaded with programming software, image editing software, and a database. The programming software includes point cloud fusion software and autonomous navigation software.

[0008] Furthermore, the point cloud fusion software includes:

[0009] The point cloud fusion module is used to fuse the information acquired by the 16-line LiDAR and the depth camera to obtain multi-dimensional fused point cloud data of the windy scene.

[0010] The point cloud optimization module is used to optimize multi-dimensional fused point cloud data from windy scenarios into point cloud information suitable for the environment.

[0011] The point cloud matching module is used to match the matched point cloud information with the established map point cloud information to determine the correct temperature measurement point location.

[0012] The path planning module is used to plan a local path for the robot car based on the established temperature measurement points and the 3D real-time situation of the air vent scene. The autonomous navigation software performs real-time movement control of the robot car based on the local path and controls the robot car to perform obstacle avoidance actions based on the point cloud data of mismatched obstacle protrusions.

[0013] Furthermore, the multi-dimensional fusion of point cloud data from trending scenarios will be optimized, specifically as follows:

[0014] Remove point cloud data from the ground, point cloud data that is too far away, useless point cloud data that overlaps at 90° in the back, and point cloud data that overlaps too much. Then compress the simplified point cloud back into 2D data.

[0015] Furthermore, after matching the matched point cloud information with the recorded map point cloud information, the confidence level is represented by a set of M particles, and the formula for the set expression is as follows:

[0016]

[0017] X t The confidence level is given by M, and the total number of particles is given by M.

[0018] The initial confidence level is obtained by randomly generating M particles and sampling them using a motion model. Starting from the current confidence level, the particles are used, and the importance weight of the particles is determined using a measurement model. The positioning accuracy of the temperature measurement point is then determined by increasing the total number of particles M. After the temperature measurement point is determined, the gimbal autonomously rotates to the preset point to perform infrared temperature measurement. After the temperature measurement is completed, the autonomous navigation software guides the robot to the next point.

[0019] Furthermore, the 16-line lidar is used to collect multi-dimensional 3D image information of the wind vent scene to obtain a multi-dimensional 3D model of the wind vent scene, and the depth camera is used to collect spatial distance information of the wind vent scene.

[0020] Furthermore, the IMU (Inertial Measurement Unit) is used to collect the rotation and horizontal angle information of the robot in real time, and the gimbal is used to send control commands to the temperature and humidity sensor, gas sensor and sound intensity sensor.

[0021] Furthermore, the temperature and humidity sensor, gas sensor, and sound intensity sensor are used to collect temperature and humidity data, gas concentration data, and on-site sound wave data in the air vent scenario.

[0022] Furthermore, the 16-line lidar, IMU inertial measurement unit, depth camera, gimbal, temperature and humidity sensor, gas sensor, sound intensity sensor, and database server are all electrically connected to the industrial control electromechanical system via wireless transmission technology.

[0023] An autonomous navigation method for windy scenes based on point cloud fusion includes the following steps:

[0024] Step 1: Connect each sensor to the industrial control computer via a wireless communication network;

[0025] The various sensors are integrated into the industrial control computer, and topics are published to release the real-time frequency of sensor data.

[0026] Step 2: Integrate and calculate to release new odometer ODOM information;

[0027] By integrating the motion data from the robot chassis and the data from the IMU, the odometry information is corrected, where odom_trans.transform.translation.x=x+(vx*cos(th)-vy*sin(th))*dt;

[0028] odom_trans.transform.translation.y=y+(vx*sin(th)+vy*cos(th))*dt;

[0029] odom_trans.transform.rotation=tf::createQuaternionMsgFromYaw(th).

[0030] Step 3: Integrate the point cloud data from the depth camera and the point cloud data from the 16-line LiDAR;

[0031] Based on the positions of the 16-line LiDAR and depth camera, the TF transformation relationship between the two sensors is added to the coordinate system of base_link, and the point cloud data of the two parts are fused to reduce some overlapping point cloud data in front.

[0032] Step 4: Optimize and integrate the point cloud data;

[0033] The integrated point cloud data was further optimized by deleting point cloud data on the ground, point cloud data that was too far away, useless point cloud data that intersected at 90° in the back, and point cloud data that overlapped too much. The simplified point cloud was then compressed back into 2D data.

[0034] Step 5: Compare with the existing map;

[0035] The integrated point cloud data and 2D data obtained in step 4 are compared with the pre-built map. Based on the positional weight information, the possible locations of the robot are provided, and locations with poor matching are deleted. Then, various existing positioning information are integrated to obtain a more accurate positioning location. Based on this location, a local path planning algorithm is performed, and the autonomous navigation software controls the robot to move in real time along the path. If there are obstacles or mismatched point cloud data, the autonomous navigation software will control the robot to perform obstacle avoidance maneuvers.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] This invention collects multi-dimensional 3D imagery and spatial distance information from the ventilator site, fuses these two data into point clouds to obtain multi-dimensional point cloud data of the ventilator site, enabling better acquisition of information about the ventilator and obstacles. It employs an odometry fusion algorithm to obtain more accurate odometry, providing the robot vehicle with more precise coarse positioning. Furthermore, it uses an IMU (Inertial Measurement Unit) for point cloud data fusion, acquiring the robot vehicle's rotation and horizontal angle information, which improves point cloud data processing and achieves higher-precision positioning, ensuring accurate temperature measurement at the designated temperature points. Finally, it utilizes a positioning-autonomous navigation scheme to ensure the robot vehicle accurately reaches the temperature measurement points even in high-temperature and high-pressure environments, enabling high-frequency daily inspections of the ventilator site. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the initial fusion of point cloud information and its matching with a map according to the present invention.

[0039] Figure 2 This is a schematic diagram illustrating the matching of the processed point cloud information of the present invention with a map.

[0040] Figure 3 This is a schematic diagram illustrating the working principle of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] To address the technical challenges of traditional methods for inspecting air vents, which involve manually holding a temperature gun and posing significant health risks due to the hazardous environment, thus reducing the feasibility of current on-site air vent temperature measurement methods, please refer to [link to relevant documentation]. Figures 1-3 This embodiment provides the following technical solution:

[0043] An autonomous navigation system for windy vent scenes based on point cloud fusion includes: a robotic vehicle, characterized in that: the robotic vehicle is used for autonomous navigation to the temperature measurement point in the windy vent scene; the robotic vehicle is equipped with a 16-line LiDAR, which is used to collect multi-dimensional 3D image information of the windy vent scene to obtain a multi-dimensional 3D model of the windy vent scene; an IMU (Inertial Measurement Unit), which is used to collect the rotation angle and horizontal angle information of the robotic vehicle in real time, which can improve the point cloud data processing effect and achieve higher precision positioning to ensure accurate temperature measurement of the temperature measurement point; and a depth camera, which is used to collect spatial distance information of the windy vent scene, accurately determining the distance of each point in the windy vent scene image from the camera, and then matching it with the x and y coordinates in the 2D image to obtain the three-dimensional spatial coordinates of each point in the windy vent scene; the three-dimensional reality of the windy vent scene can be reconstructed using the three-dimensional coordinates, so as to... Subsequently, the information acquired by the 16-line LiDAR is fused into a point cloud. A gimbal is used to send control commands to the temperature and humidity sensor, gas sensor, and sound intensity sensor. The temperature and humidity sensor, gas sensor, sound intensity sensor, and database server are used to collect temperature and humidity data, gas concentration data, and on-site sound wave data in the wind vent scene. The database server is loaded with programming software, screen editing software, and a database. The programming software includes point cloud fusion software and autonomous navigation software. The 16-line LiDAR, IMU inertial measurement unit, depth camera, gimbal, temperature and humidity sensor, gas sensor, sound intensity sensor, and database server are all electrically connected to the industrial control computer via wireless transmission technology, integrating the various sensors onto the industrial control computer. The real-time frequency of sensor data is published via a topic, so that the client can monitor the temperature and humidity information of the wind vent scene and the driving progress of the robot vehicle in real time through the industrial control computer.

[0044] Point cloud fusion software, including:

[0045] The point cloud fusion module is used to fuse the information acquired by the 16-line LiDAR and depth camera to obtain multi-dimensional fused point cloud data of the wind tunnel scene. This is mainly to facilitate better acquisition of information about the wind tunnel scene and obstacles, to make fine planning for the robot car's travel path, and to mark obstacles, so as to effectively control the robot car to perform autonomous obstacle avoidance.

[0046] The point cloud optimization module is used to optimize the multi-dimensional fused point cloud data of the windy scene into point cloud information suitable for the environment. Specifically, it deletes point cloud data on the ground, point cloud data that is too far away, useless point cloud data that intersects at 90° in the back, and point cloud data with too much overlap, and recompresses the simplified point cloud into 2D data. It is mainly used to optimize the realism of the image information of the windy scene, highlight important local paths, and provide more accurate path information and positioning information for the robot car.

[0047] The point cloud matching module is used to match the matched point cloud information with the established map point cloud information. The confidence level is represented by a set of M particles, and the formula for the set expression is as follows:

[0048]

[0049] X t The confidence level is given by M, and the total number of particles is given by M.

[0050] The initial confidence level is obtained by randomly generating M particles and sampling them using a motion model. Starting from the current confidence level, the particles are used, and the importance weight of the particles is determined using a measurement model. The positioning accuracy of the temperature measurement point is then determined by increasing the total number of particles M. After the temperature measurement point is determined, the gimbal autonomously rotates to the preset point to perform infrared temperature measurement. After the temperature measurement is completed, the autonomous navigation software guides the robot to the next point.

[0051] The path planning module is used to plan a local path for the robot car based on the established temperature measurement points and the 3D real-time situation of the air vent scene. The autonomous navigation software performs real-time movement control of the robot car based on the local path and controls the robot car to perform obstacle avoidance actions based on the point cloud data of mismatched obstacle protrusions.

[0052] Specifically, by collecting multi-dimensional 3D imagery and spatial distance information from the ventilator site, and fusing these two data into point clouds, multi-dimensional point cloud data of the ventilator site is obtained. Using an odometry fusion algorithm, combined with rotation and horizontal angle information of the robot vehicle collected by an IMU inertial measurement unit, the point cloud data processing achieves better ventilator scene reconstruction, providing the robot vehicle with more accurate coarse positioning. This enables the robot vehicle to perform more precise positioning work, ensuring accurate temperature measurement at the temperature measurement points. An autonomous navigation scheme ensures the robot vehicle accurately reaches the temperature measurement points in high-temperature and high-pressure environments. The fused information allows for autonomous navigation and obstacle avoidance, ensuring high-frequency daily inspections of the ventilator site environment. With 12 temperature measurement inspections per day, the system can monitor temperature data changes at all ventilators on the platform in real time, significantly exceeding the number of manual inspections and effectively replacing manual inspections.

[0053] In this embodiment, a method for autonomous navigation in a windy scene based on point cloud fusion is described, and the specific operation steps are as follows:

[0054] Step 1: Connect each sensor to the industrial control computer via a wireless communication network;

[0055] The various sensors are integrated into the industrial control computer, and topics are published to release the real-time frequency of sensor data.

[0056] Step 2: Integrate and calculate to release new odometer ODOM information;

[0057] By integrating the motion data from the robot chassis and the data from the IMU, the odometry information is corrected, where odom_trans.transform.translation.x=x+(vx*cos(th)-vy*sin(th))*dt;

[0058] odom_trans.transform.translation.y=y+(vx*sin(th)+vy*cos(th))*dt;

[0059] odom_trans.transform.rotation=tf::createQuaternionMsgFromYaw(th).

[0060] Step 3: Integrate the point cloud data from the depth camera and the point cloud data from the 16-line LiDAR;

[0061] Based on the positions of the 16-line LiDAR and depth camera, the TF transformation relationship between the two sensors is added to the coordinate system of base_link, and the point cloud data of the two parts are fused to reduce some overlapping point cloud data in front.

[0062] Step 4: Optimize and integrate the point cloud data;

[0063] The integrated point cloud data is further optimized by deleting point cloud data on the ground, point cloud data that is too far away, useless point cloud data that intersects at 90° in the back, and point cloud data that overlaps too much. The simplified point cloud is then compressed back into 2D data.

[0064] Step 5: Compare with the existing map;

[0065] The integrated point cloud data and 2D data obtained in step 4 are compared with the map established at the beginning. The possible locations of the robot are provided based on the position weight information, and locations with very poor matching are deleted. Then, various existing positioning information are integrated to obtain a more accurate positioning location. Based on this location, a local path planning algorithm is performed, and the autonomous navigation software controls the robot to move in real time according to the path. If there are obstacles or mismatched point cloud data, the autonomous navigation software will control the robot to perform obstacle avoidance actions, thereby realizing the autonomous navigation and obstacle avoidance functions of the robot in windy scenes.

[0066] The beneficial effects achieved by the above are as follows: By adopting point cloud fusion technology, better information on air vents and obstacles can be obtained; by using odometry fusion algorithms, more accurate coarse positioning of the robot can be provided; by using an IMU inertial measurement unit to collect the rotation angle and horizontal angle information of the robot, high-precision positioning can be completed, ensuring accurate temperature measurement of the temperature measurement point; and by adopting a positioning autonomous navigation scheme, the robot can accurately reach the temperature measurement point in a high-temperature and high-pressure environment, ensuring high-frequency operation every day.

[0067] Working principle: By collecting multi-dimensional 3D image information and spatial distance information from the ventilator site, the two are fused into point cloud data to obtain multi-dimensional point cloud data of the ventilator site. Using the odometry fusion algorithm, combined with the rotation and horizontal angle information of the robot vehicle collected by the IMU inertial measurement unit, the robot vehicle can perform more precise positioning, ensuring accurate temperature measurement of the temperature measurement points. An autonomous navigation system ensures that the robot vehicle can autonomously navigate and avoid obstacles in high-temperature and high-pressure environments. Through 12 temperature measurement inspections per day, the system can monitor the temperature data changes of all ventilators on the ventilator platform in real time, replacing manual inspections.

[0068] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0069] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An autonomous navigation system for windy scenes based on point cloud fusion, comprising: The robot car is characterized in that: the robot car is used for autonomous navigation to the temperature measurement point in the windy scene; the robot car is equipped with a 16-line LiDAR, an IMU inertial measurement unit, a depth camera, a gimbal, a temperature and humidity sensor, a gas sensor, a sound intensity sensor, and a database server in sequence; the 16-line LiDAR, IMU inertial measurement unit, depth camera, temperature and humidity sensor, gas sensor, and sound intensity sensor are all connected to the gimbal via wireless transmission technology; the gimbal is used to control the operating status of the above-mentioned sensing devices; the database server is loaded with programming software, image editing software, and a database; the programming software includes point cloud fusion software and autonomous navigation software. The point cloud fusion software includes: The point cloud fusion module is used to fuse the information acquired by the 16-line LiDAR and the depth camera to obtain multi-dimensional fused point cloud data of the windy scene. The point cloud optimization module is used to optimize multi-dimensional fused point cloud data from windy scenarios into point cloud information suitable for the environment. The point cloud matching module is used to match the matched point cloud information with the established map point cloud information to determine the correct temperature measurement point location. The path planning module is used to plan the local path of the robot car based on the established temperature measurement points and the three-dimensional real-time situation of the air vent scene. The autonomous navigation software performs real-time movement control of the robot car based on the local path and controls the robot car to perform obstacle avoidance actions based on the point cloud data of mismatched obstacle protrusions. After matching the matched point cloud information with the recorded map point cloud information, the confidence level is represented by a set of M particles, and the formula for the set expression is as follows: ; For confidence level, The total number of particles; The initial confidence level is obtained by randomly generating M particles and sampling them using a motion model. Starting from the current confidence level, the particles are used, and the importance weight of the particles is determined using a measurement model. The positioning accuracy of the temperature measurement point is then determined by increasing the total number of particles M. After the temperature measurement point is determined, the gimbal autonomously rotates to the preset point to perform infrared temperature measurement. After the temperature measurement is completed, the autonomous navigation software guides the robot to the next point.

2. The autonomous navigation system for windy scenes based on point cloud fusion according to claim 1, characterized in that: Optimize multi-dimensional fusion point cloud data of trending scenarios, specifically: Remove point cloud data from the ground, point cloud data that is too far away, useless point cloud data that overlaps at 90° in the back, and point cloud data that overlaps too much. Then compress the simplified point cloud back into 2D data.

3. The autonomous navigation system for windy scenes based on point cloud fusion according to claim 1, characterized in that: The 16-line lidar is used to collect multi-dimensional 3D image information of the wind vent scene to obtain a multi-dimensional 3D model of the wind vent scene, and the depth camera is used to collect spatial distance information of the wind vent scene.

4. The autonomous navigation system for windy scenes based on point cloud fusion according to claim 1, characterized in that: The IMU (Inertial Measurement Unit) is used to collect the rotation and horizontal angle information of the robot in real time, and the gimbal is used to send control commands to the temperature and humidity sensor, gas sensor and sound intensity sensor.

5. The autonomous navigation system for windy scenes based on point cloud fusion according to claim 1, characterized in that: The temperature and humidity sensor, gas sensor, and sound intensity sensor are used to collect temperature and humidity data, gas concentration data, and on-site sound wave data in the air vent scenario.

6. The autonomous navigation system for windy scenes based on point cloud fusion according to claim 1, characterized in that: The 16-line lidar, IMU inertial measurement unit, depth camera, gimbal, temperature and humidity sensor, gas sensor, sound intensity sensor, and database server are all electrically connected to the industrial control electromechanical system via wireless transmission technology.

7. A method for autonomous navigation in windy scenes based on point cloud fusion, implemented based on the autonomous navigation system for windy scenes based on point cloud fusion as described in any one of claims 1-6, characterized in that: Includes the following steps: Step 1: Connect each sensor to the industrial control computer via a wireless communication network; The various sensors are integrated into the industrial control computer, and the real-time frequency of sensor data is published in a topic. Step 2: Integrate and calculate to release new odometer ODOM information; By integrating the motion data from the robot chassis and the data from the IMU, the odometry information is corrected, where odom_trans.transform.translation.x = x + (vx * cos(th) - vy * sin(th)) * dt; odom_trans.transform.translation.y = y + (vx * sin(th) + vy * cos(th)) *dt; odom_trans.transform.rotation= tf::createQuaternionMsgFromYaw(th); Step 3: Integrate the point cloud data from the depth camera and the point cloud data from the 16-line LiDAR; Based on the positions of the 16-line LiDAR and depth camera, the TF transformation relationship between the two sensors is added to the coordinate system of base_link to fuse the point cloud data of the two parts and reduce some overlapping point cloud data in front. Step 4: Optimize and integrate the point cloud data; The integrated point cloud data is optimized by deleting point cloud data on the ground, point cloud data that is too far away, useless point cloud data that intersects at 90° in the back, and point cloud data that overlaps too much. The simplified point cloud is then compressed back into 2D data. Step 5: Compare with the existing map; The integrated point cloud data and 2D data obtained in step 4 are matched and compared with the pre-established map. The location of the robot is provided according to the position weight information, and the poorly matched locations are deleted. Then, the various existing positioning information are integrated to obtain a more accurate positioning location. Based on this location, a local path planning algorithm is performed, and the autonomous navigation software controls the robot to move in real time according to the path. If there are obstacles or mismatched point cloud data, the autonomous navigation software will control the robot to perform obstacle avoidance actions.

Images