Coal mine track robot inspection method based on multi-sensor equipment fusion

Abstract

The invention relates to a coal mine rail robot inspection method based on multi-sensor equipment fusion, a rail robot carries an obstacle avoidance camera, a pan-tilt camera, an obstacle avoidance sensor and an encoder, and the method comprises the following steps: S1, the rail robot runs along a rail and obtains position information; s2, the track robot obtains a track condition image in the moving direction in the moving process, and meanwhile the distance of an obstacle in the moving direction is judged; s3, if it is judged that an obstacle exists in the track condition image and the object distance of the obstacle is smaller than a braking threshold value, the track robot starts to brake, and otherwise, the track robot continues to move; s4, after the rail robot moves to the target position, the rail robot stops, the target is detected, and after detection is completed, the rail robot continues to run along the rail; and S5, repeating the steps S1 to S4 until the inspection is completed. Through control and data acquisition of a plurality of sensors, precise positioning of the inspection target and the track robot is realized, the inspection efficiency is improved, and the situation of false alarm touch is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of coal mine track robots, and in particular to a coal mine track robot inspection method based on multi-sensor device fusion. Background Art

[0002] As an important energy industry, coal mines have always been highly valued for their safe production. With the increase in the depth of coal mining and the difficulty of mining, traditional manual mining methods are facing more and more safety risks and efficiency problems. With the rapid development of modern technologies such as the Internet of Things, cloud computing, big data, and artificial intelligence, the concept of smart mines has emerged, emphasizing the use of technical means to improve the intelligence and automation level of mines. Therefore, the use of rail robot technology to replace manual production and improve production efficiency and safety has become an important trend in the development of the coal mining industry. The development of coal mine rail robots is precisely to meet this demand and improve the safety and efficiency of coal mining through automation and intelligence.

[0003] In recent years, track robots in coal mines have been widely used in underground inspections. The various sensor technologies carried by track robots can autonomously collect and analyze equipment inspection data by simulating human senses, realize the linkage function of active alarm and emergency braking, and timely control risks to achieve inspection results. At present, most track robots process data independently after each sensory information is collected. After each unit collects data, it processes and uploads it at the same time. Although the efficiency is very high, there is a high probability of false alarms, which often leads to misjudgment of the decision-making of the on-duty personnel and the wrong shutdown of the linkage equipment, which brings trouble to production. Summary of the invention

[0004] In order to solve the problems existing in the above-mentioned technology, the present invention provides a coal mine track robot inspection method based on multi-sensor device fusion, wherein the track robot is equipped with an obstacle avoidance camera, a pan-tilt camera, an obstacle avoidance sensor and an encoder, and the method comprises the following steps:

[0005] S1 The track robot runs along the track and obtains position information through an encoder;

[0006] S2 The track robot obtains the track condition image in the moving direction through the obstacle avoidance camera during the movement, and determines the distance of obstacles in the moving direction through the obstacle avoidance sensor;

[0007] S3: If it is determined in the track condition image that there is an obstacle and the distance of the obstacle is less than the braking threshold, the track robot starts braking, otherwise it continues to move;

[0008] S4: When the track robot moves to the target position, it stops and detects the target through the pan-tilt camera, and continues to move along the track after the detection is completed;

[0009] S5 repeats steps S1 to S4 until the inspection is completed.

[0010] In step S2, the host processes the track condition image through the trained YOLO algorithm model and determines whether there are obstacles in the moving direction.

[0011] Wherein, the obstacle avoidance sensor in step S2 is an ultrasonic sensor, and the obstacle avoidance sensor sends the obtained obstacle distance information to the host, and the host processes the obstacle distance information through median filtering.

[0012] Among them, the median filtering formula is:

[0013] y(i)=Med{x(iN)Λx(i)Λx(i+N)};

[0014] Wherein, Med represents the signal sample window; x(iN),…,x(i),…,x(i+N) are sample acquisition two-dimensional data; x(i) is the signal sample value located at the center of the window; N is the length of the defined acquisition sequence; y(i) is the output value of the final filter.

[0015] Wherein, an RFID tag is arranged on the track at the target position, and the track robot is equipped with an RFID card reader;

[0016] In step S4, when the rail robot reaches the target position, the information of the RFID tag is read by the RFID reader, and when the information of the RFID tag matches the position information of the rail robot, the target is inspected by the pan-tilt camera.

[0017] Among them, in step S4, before performing target detection, the target is first locked, and then the pan-tilt camera is controlled to rotate so that the target is located at the image center of the pan-tilt camera.

[0018] The beneficial effects of the present invention are as follows: through the control and data collection of multiple sensors such as encoders, obstacle avoidance sensors, obstacle avoidance cameras, RFID tags, pan-tilt cameras, etc., the data are integrated to achieve accurate positioning of the inspection target and the rail robot itself. At the same time, during the positioning process, the target object is locked through an image recognition algorithm, thereby improving the inspection efficiency, improving the accuracy of target detection, and reducing the possibility of false alarms. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention are described clearly and completely below in conjunction with the embodiments of the present invention. In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0020] The present application provides a coal mine track robot inspection method based on multi-sensor device fusion;

[0021] The track robot is equipped with an obstacle avoidance camera, a pan-tilt camera, an obstacle avoidance sensor, an encoder, a host, an RFID card reader, an illuminator and a walking motor, and an RFID tag is arranged at the position of a target to be detected on the track.

[0022] The inspection method specifically includes the following steps:

[0023] S1 The track robot runs along the track and obtains position information through an encoder;

[0024] Furthermore, the encoder data is transmitted to the acquisition module of the host, and the value is calculated by combining the ratio of the rail robot reducer, wheel diameter and other parameters stored in the host. After the rail robot runs, the value of each interval is calculated with the previous value, and the accumulated value of this difference is the current position value of the rail robot. The current position value of the rail robot is a signed value. The running direction of the rail robot can be determined by the increase or decrease of the current position value. Combined with the running direction and position information of the rail robot, the inspection task can be performed at the continuous inspection target position or the specified inspection target position.

[0025] S2 The track robot obtains the track condition image in the moving direction through the obstacle avoidance camera during the movement, and determines the distance of obstacles in the moving direction through the obstacle avoidance sensor;

[0026] Specifically, the host processes the track condition image through the trained YOLO algorithm model and determines whether there are obstacles in the moving direction;

[0027] Obstacles such as scaffolding or cables temporarily built on the track, etc., are obtained by obtaining images of scaffolding and cables on the track to form training sets and test sets, and the training sets and test sets are input into the TOLO model for training and detection to obtain the trained YOLO model;

[0028] Further, the obstacle avoidance sensor is an ultrasonic sensor, and the obstacle avoidance sensor sends the obtained obstacle distance information to the host, and the host processes the obstacle distance information through median filtering;

[0029] The median filter formula:

[0030] y(i)=Med{x(iN)Λx(i)Λx(i+N)};

[0031] Where Med represents the signal sample window; x(iN), ..., x(i), ..., x(i+N) are sample acquisition two-dimensional data; x(i) is the signal sample value located at the center of the window; N is the length of the defined acquisition sequence; y(i) is the output value of the final filter;

[0032] In one embodiment, N is 2, the sample window is 0.1 seconds, the detection interval of the ultrasonic sensor is 20 milliseconds, and a total of 5 signal sample values ​​x(0.02)...x(0.10) are obtained, and the signal sample value is the distance value between the track robot and the obstacle in the moving direction;

[0033] The five signal sample values ​​are sorted by size, and the median is taken as the output value of the final filtering of the sample window.

[0034] By filtering, data with large jitter is filtered out, and the filtered data becomes smoother, which has a good filtering effect on pulse noise, making the alarm data more accurate and improving the reliability of the inspection process.

[0035] S3: If it is determined in the track condition image that there is an obstacle and the distance of the obstacle is less than the braking threshold, the track robot starts braking and issues an alarm, otherwise it continues to move;

[0036] Furthermore, the braking threshold includes a deceleration threshold and a parking threshold. During deceleration, the distance between the obstacle and the robot is calculated in linear proportion to the running speed of the track robot. As the distance decreases, the speed gradually decreases until the distance is reduced to the parking threshold, triggering the track robot to stop, sound an alarm, and clean the track.

[0037] S4 When the track robot reaches the target position, the position information of the RFID tag is read through the RFID card reader. When the position information of the RFID tag matches the position information of the track robot, the robot stops and the target is inspected through the pan-tilt camera. After the inspection is completed, the robot continues to run along the track;

[0038] The RFID tag number is composed of 3 bytes of hexadecimal code, where the first byte is the function code, which enables the track robot to perform the corresponding function, the second byte is the number code, which can preset the rotation angle of the pan / tilt by the number, and the third byte is the data code, which is the specific execution content of the first byte function code;

[0039] Furthermore, the position of the inspection target of the rail robot is determined by the target position and the RFID tag number. The target position is preset, and the robot's travel distance is calculated by the encoder to determine whether it has moved to the preset target position. When the encoder determines that the robot has moved to the preset position, and reads the RFID tag information, and the position information of the encoder matches the position information of the RFID tag, the pan-tilt camera is adjusted to a preset angle for target inspection.

[0040] Before target detection, the target is first locked, and then the pan-tilt camera is controlled to rotate so that the target is located in the image center of the pan-tilt camera.

[0041] Specifically, the target position information is obtained through the visible light camera of the gimbal. If the target is not in the center of the image, the gimbal angle is continuously adjusted until the target is moved to the center of the image.

[0042] Furthermore, the host's analysis and processing module locks the target through the YOLO target detection algorithm, and then calculates the coordinate value of the target's center point, and then sends instructions to the host's control module to control the gimbal to rotate and slowly move the target's center coordinate value to the center position of the entire image.

[0043] After the target is adjusted to the center position, the control module adjusts the zoom or aperture function of the pan-tilt camera to enlarge or focus the image. The analysis and processing module captures a picture of the current target. The target may be a device to be detected. The state of the device to be detected can be determined by the picture of the target. The YOLO target detection algorithm detects the specific content of the target. The YOLO target detection algorithm model used for target detection is a trained model.

[0044] If the target is not detected, adjust the zoom or aperture, and then detect again. The detection time is limited for a single target detection. If the set detection time is exceeded, the detection will be exited and the next target detection will be carried out.

[0045] During detection, if the acquisition module detects that the light on site is dim, the control module will turn on the lighting to supplement the equipment.

[0046] S5 repeats steps S1 to S4 until the inspection is completed.

[0047] Furthermore, the walking motor is divided into two left and right parts, which are fixedly connected to the top of the host. As the main action unit of the entire track robot, it receives control instructions from the host control module and also transmits the pulse signal fed back by the encoder to the acquisition module of the host.

[0048] The obstacle avoidance sensors and obstacle avoidance cameras are installed in two directions, front and back, respectively. The maximum detection distance of the obstacle avoidance sensor is 2m, and each detection angle is 30°, which can completely cover the detection surface in the running direction. The obstacle avoidance sensor uploads the data to the acquisition module of the host every 20ms, and then feeds back the accurate data to the control module through the median filtering method. The control module issues control instructions to adjust the speed of the walking motor.

[0049] The RFID reader is installed on the upper left side of the track robot in order to keep it parallel to several RFID tags installed on the steel track. Whenever the RFID reader reads the RFID tag signal, the acquisition module sends the parsed data to the analysis and processing module, which then issues and executes the task.

[0050] The PTZ camera is installed upside down under the host machine. It has the function of 360 degrees horizontally and ±90 degrees vertically rotating. Therefore, the inverted installation of the PTZ camera can scan all viewing angles below the track robot. It is also equipped with a high-definition zoom camera, and the device can clearly capture images within a 20m range.

[0051] There are lights installed on the front side of the track robot. Each light has an illumination angle of 60° and an illumination distance of 20m. In dim environments, it can provide additional light for the target and improve the accuracy of target detection.

[0052] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Claims

1. A coal mine track robot inspection method based on multi-sensor equipment fusion, characterized in that: The rail robot is equipped with an obstacle avoidance camera, a pan-tilt camera, an obstacle avoidance sensor and an encoder, and the method comprises the following steps: S1 The track robot runs along the track and obtains position information through an encoder; S2 The track robot obtains the track condition image in the moving direction through the obstacle avoidance camera during the movement, and determines the distance of obstacles in the moving direction through the obstacle avoidance sensor; S3: If it is determined in the track condition image that there is an obstacle and the distance of the obstacle is less than the braking threshold, the track robot starts braking, otherwise it continues to move; S4: When the track robot moves to the target position, it stops and detects the target through the pan-tilt camera, and continues to move along the track after the detection is completed; S5 repeats steps S1 to S4 until the inspection is completed.

2. According to claim 1, a coal mine track robot inspection method based on multi-sensor equipment fusion is characterized in that: In step S2, the host processes the track condition image through the trained YOLO algorithm model and determines whether there are obstacles in the moving direction.

3. The coal mine track robot inspection method based on multi-sensor equipment fusion according to claim 1 is characterized in that: In step S2, the obstacle avoidance sensor is an ultrasonic sensor, and the obstacle avoidance sensor sends the obtained obstacle distance information to the host, and the host processes the obstacle distance information through median filtering.

4. The coal mine track robot inspection method based on multi-sensor equipment fusion according to claim 3 is characterized in that: The median filter formula: y(i)=Med{x(iN)Λx(i)Λx(i+N)}; Wherein, Med represents the signal sample window; x(iN),…,x(i),…,x(i+N) are sample acquisition two-dimensional data; x(i) is the signal sample value located at the center of the window; N is the length of the defined acquisition sequence; y(i) is the output value of the final filter.

5. The coal mine track robot inspection method based on multi-sensor equipment fusion according to claim 1 is characterized in that: An RFID tag is arranged on the track at the target position, and the track robot is equipped with an RFID card reader; In step S4, when the rail robot reaches the target position, the information of the RFID tag is read by the RFID reader, and when the information of the RFID tag matches the position information of the rail robot, the target is inspected by the pan-tilt camera.

6. The coal mine track robot inspection method based on multi-sensor equipment fusion according to claim 1 is characterized in that: In step S4, before performing target detection, the target is first locked, and then the pan-tilt camera is controlled to rotate so that the target is located at the image center of the pan-tilt camera.