Belt derailment inspection system and method based on indoor intelligent unmanned aerial vehicle

Abstract

The invention relates to the technical field of intelligent inspection of belt conveyors, in particular to a belt derailment inspection system and method based on an indoor intelligent unmanned aerial vehicle. The system comprises an indoor unmanned aerial vehicle, an intelligent machine library and a server, the indoor unmanned aerial vehicle and the intelligent machine library are respectively connected with the server through a router, the server is internally provided with an inspection algorithm module, a belt working state module and an inspection identification module, and an inspection route of the unmanned aerial vehicle is planned through an A * search algorithm of the inspection algorithm module. And the belt falling condition is monitored through a canny edge detection algorithm of the belt working state module. According to the invention, the analysis and processing of the inspection data of the belt conveyor can be realized, and the analysis and processing mainly comprise the functions of infrared temperature measurement, carrier roller falling, bearing heating, foreign matter falling, belt deviation, illegal person monitoring, safety helmet monitoring and the like, such as calculation, analysis, alarm and report, so that the purpose of protecting the safety of equipment is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent inspection of belt conveyors, and in particular to a belt derailment inspection system and method based on an indoor intelligent unmanned aerial vehicle. Background Art

[0002] In today's industrial production field, belt conveyors, as a key material conveying equipment, are widely used in many industries such as coal, mining, electricity, ports, etc. Their stable operation plays a vital role in the continuity and efficiency of the entire production process. However, traditional belt inspection methods mainly rely on manual inspection and ground robot inspection, which have many limitations.

[0003] Manual inspection is not only labor-intensive and inefficient, but also greatly affected by human subjective factors, prone to missed inspections and misjudgments. At the same time, some complex and dangerous working environments (such as high altitude, narrow space, areas with risk of harmful gas leakage, etc.) pose a threat to the personal safety of inspectors, making it difficult for manual inspections to conduct comprehensive and in-depth inspections of belt conveyors.

[0004] In the inspection and maintenance scenarios of indoor coal belt conveyors, robots are gradually becoming powerful assistants for inspection and maintenance operations, usually mainly ground robots (wheeled robots, multi-legged robots, etc.) and track robots. Traditional ground robots and track robots are affected by obstacles such as gullies and pipelines on the ground, and cannot achieve full coverage of the inspection area. In particular, for some belt parts that are located at high altitudes or difficult to reach, ground robots are often helpless. Moreover, the deployment cost of track robots is high, and special tracks need to be laid on site, which not only consumes a lot of time and money, but also limits the flexibility and scalability of robots, making it difficult to adapt to complex and changeable industrial production environments. In particular, for equipment to be inspected at high altitudes, there is a lack of effective means for data collection. Therefore, robot inspection has developed slowly in the field of belt conveyor inspection. Summary of the invention

[0005] In order to solve the above-mentioned problems, the present invention provides a belt derailment inspection system and method based on an indoor intelligent drone.

[0006] In the first aspect, the present invention provides a belt derailment inspection system based on an indoor intelligent drone, which adopts the following technical solutions:

[0007] A belt inspection system based on an indoor intelligent drone, comprising:

[0008] Indoor UAV, smart hangar and server, wherein the indoor UAV and smart hangar are respectively connected to the server through a router, and the server is provided with an inspection algorithm module, a belt working status module and an inspection identification module. The inspection route of the UAV is planned by the A* search algorithm of the inspection algorithm module, and the belt shedding situation is monitored by the canny edge detection algorithm of the belt working status module; the wearing situation of the safety device of the personnel is monitored by the target detection algorithm recognition of the inspection identification module.

[0009] Furthermore, the indoor drone is equipped with a laser radar, an infrared sensor, a gas sensor and a camera, wherein indoor positioning and obstacle avoidance are performed based on the multi-echo detection method and SLAM algorithm of the laser radar, the safety wearing status of the workers is judged based on the camera, the cable damage is judged based on the infrared radiation characteristics of the infrared sensor, and the gas concentration is converted into an electrical signal based on the semiconductor adsorption and electrochemical reaction of the gas sensor to judge the gas leakage.

[0010] Furthermore, a host computer module is arranged between the router and the intelligent drone. The router and the host computer module are connected via WiFi. The host computer module has a built-in wireless network card and a high-gain external antenna. It receives command data from the router based on the TCP / IP protocol, converts it into command format data recognizable by the drone, and stores the data acquired by the drone in a high-speed cache area.

[0011] Furthermore, the smart hangar and the router are connected via a wired network, the router and the server are connected via a WiFi network, and the router is also connected to a mobile terminal via a WiFi network, wherein the smart hangar and the router are connected via a Category 6 Ethernet cable, the smart hangar is equipped with a standard RJ45 Ethernet interface, and a fixed static IP address is configured for the smart hangar on the router for precise network access and management control of the smart hangar, and the network access rights of the smart hangar are set using the router's access control list ACL.

[0012] Furthermore, the intelligent drone is also equipped with a manual remote control for manually controlling the drone to return after the drone is disconnected from the server. When the heartbeat packet sent by the drone is not received within the threshold time set by the server and the signal strength is lower than the set threshold based on the automatic disconnection detection method, the manual remote control automatically takes over the control of the drone and controls the drone to return through the staff.

[0013] Furthermore, the smart hangar and the smart drone are respectively configured with ZigBee communication modules, and information is exchanged through the ZigBee wireless communication protocol to achieve automatic return and charging. A star network topology is used to construct a ZigBee communication network between the smart hangar and the smart drone, with the smart hangar serving as the central node of the network, and communicating with multiple drones as terminal nodes.

[0014] In the second aspect, a belt inspection method based on an indoor intelligent drone includes:

[0015] Acquire lidar data, infrared sensor data, gas sensor data and image data based on indoor intelligent drones;

[0016] According to the acquired LiDAR data, the inspection algorithm module based on the server plans the inspection route of the intelligent drone;

[0017] According to the acquired infrared sensor data, the inspection and recognition module based on the server obtains the image of the worker's working status;

[0018] According to the acquired image data, the belt detection module based on the server identifies the working status of the belt.

[0019] Furthermore, the server-based inspection algorithm module plans the inspection route of the intelligent drone, including using the A* search algorithm to perform training route planning for the intelligent drone, wherein after the inspection area is divided into grids, a preliminary path from the starting point to the target point is found, and the next node is selected by calculating the actual cost from the starting point to the current node and the estimated cost from the current node to the target point. For adjacent grids, the moving cost from the current node to the adjacent node is calculated, and the estimated cost from the adjacent node to the target point is added, and the node with the smallest cost is selected as the next node to be expanded until the target point is reached or no feasible path can be found.

[0020] Furthermore, the server-based inspection and recognition module obtains images of workers' working status, including detecting and locating workers using a target detection algorithm based on the acquired infrared sensor images, and identifying the wearing of workers' safety devices based on bounding boxes, wherein a large number of labeled worker status images containing different working scenes are used to train the YOLOv5 model, and the trained YOLOv5 model is used to detect the workers' safety helmets and safety belts, and a region proposal network is used to generate candidate regions that may contain objects, and then these candidate regions are classified and regressed to determine the category and position of the objects.

[0021] Furthermore, based on the acquired image data, a server-based belt detection module identifies the working status of the belt, including Gaussian smoothing based on the acquired belt image using the Canny edge detection algorithm, detecting the edge of the belt by calculating the gradient amplitude and direction of the image, and determining the belt detachment condition by calculating the grayscale difference between the belt and the background.

[0022] In summary, the present invention has the following beneficial technical effects:

[0023] The present invention can realize the analysis and processing of belt conveyor inspection data, mainly including the calculation, analysis, alarm, reporting and other functions of infrared temperature measurement, roller shedding, bearing heating, foreign matter falling, belt deviation, illegal personnel monitoring, safety helmet monitoring and so on, so as to achieve the purpose of protecting equipment safety.

[0024] The A* algorithm can quickly find an effective path from the starting point to the target point, reducing the calculation time. It has the ability to avoid obstacles autonomously and can adjust the path in time when encountering obstacles to ensure the safety of the inspection equipment. The path can be dynamically adjusted according to environmental changes to adapt to different inspection scenarios. The inspection route algorithm can effectively plan a safe and efficient inspection route, and autonomously adjust the path when encountering obstacles or environmental changes to ensure the smooth progress of the inspection task. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of a belt inspection system based on an indoor intelligent drone according to an embodiment of the present invention.

[0026] Figure 2 Schematic diagram of a path planning algorithm according to an embodiment of the present invention.

[0027] Figure 3 Schematic diagram of belt fall-off detection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0029] Example 1

[0030] Reference Figure 1 In this embodiment, a belt inspection system based on an indoor intelligent drone includes an indoor drone, an intelligent hangar and a server, wherein the indoor drone and the intelligent hangar are respectively connected to the server through a router, and the server is provided with an inspection algorithm module, a belt working status module and an inspection identification module. The inspection route of the drone is planned by the A* search algorithm of the inspection algorithm module, and the belt shedding situation is monitored by the canny edge detection algorithm of the belt working status module; the wearing situation of the personnel safety device is monitored by the target detection algorithm of the inspection identification module.

[0031] Specifically, this embodiment selects a quadcopter drone designed for an industrial environment, and its fuselage is made of special materials that are waterproof, dustproof, and explosion-proof to adapt to the harsh environment of coal mines. The drone is equipped with a high-performance flight control system that can hover stably and fly accurately, with a flight accuracy of up to centimeters. The laser radar on board uses multi-echo detection technology, which can accurately obtain distance information of the surrounding environment even in dusty environments. A high-precision map is constructed through the SLAM algorithm to achieve accurate indoor positioning and obstacle avoidance functions, ensuring that the drone can fly safely in complex tunnels and avoid collisions with obstacles such as walls and belt conveyor brackets.

[0032] The infrared sensor uses a high-sensitivity model that can keenly detect temperature changes caused by cable overload or poor contact. By analyzing the infrared radiation characteristics of the cable, it can promptly discover hidden dangers of cable damage and prevent serious consequences such as fire or equipment shutdown caused by cable failure.

[0033] The gas sensor uses a composite semiconductor sensor that can detect multiple harmful gases (such as carbon monoxide, methane, etc.) at the same time. When a gas leak occurs underground, it can quickly convert the gas concentration into an electrical signal and send it back to the server through the drone's communication system so that ventilation, evacuation and other measures can be taken in time to ensure the safety of personnel and the normal operation of equipment.

[0034] The camera uses an industrial-grade camera with low illumination, high resolution and wide dynamic range. It can clearly capture the operating status of the belt conveyor and the working conditions of the surrounding workers in a dimly lit environment, providing a high-quality data source for subsequent image analysis.

[0035] Smart hangar: Install the smart hangar in a relatively open and well-ventilated location in the lane near the starting end of the belt conveyor to facilitate the take-off and landing operations of the drone. The hangar adopts a sturdy metal frame structure with waterproof, dustproof and rust-proof functions. It is equipped with high-precision positioning devices and smart charging equipment. It is connected to the router via a Category 6 Ethernet cable to ensure stable and reliable data transmission. The standard RJ45 Ethernet interface equipped in the hangar is closely connected to the network cable. A fixed static IP address (for example, 192.168.1.100) is configured for it on the router, and its network access rights are restricted by the access control list ACL, which only allows data interaction with specific ports of the server to ensure the network security of the hangar and prevent external illegal intrusion.

[0036] Server: Industrial-grade servers are selected and placed in the coal mine's monitoring center. Equipped with multi-core processors, large-capacity memory and high-speed hard disk arrays, the servers can quickly process large amounts of data collected by drones and run complex inspection algorithms and models. The server's operating system uses a stable and reliable Windows Server or Linux system, and is installed with a specially developed belt inspection software platform, which integrates core functions such as inspection algorithm modules, belt working status modules and inspection identification modules. Through communication connections with drones and smart hangars, it realizes centralized management and control of the entire inspection system.

[0037] Router: An enterprise-level wireless router was selected and installed in a suitable location in the lane to ensure that its signal can cover the flight area of ​​the drone and the connection range of the smart hangar and the server. The router is connected to the server through a wired network, and is connected to the smart hangar through a Category 6 cable, and is connected to the drone through a WiFi network, providing a stable network communication environment for the entire system. The router was configured in detail, and a high-strength password and encryption method were set to ensure network security. At the same time, according to the requirements of drone inspection data transmission, the wireless signal strength, frequency band and channel of the router were optimized to ensure stable and efficient data transmission between the drone and the router, and reduce data packet loss and delay.

[0038] Working principle: On the inspection platform, operators use the inspection algorithm module to plan the inspection route of the drone based on the map information of the lanes and the layout of the belt conveyor. For example, the key parts of the belt conveyor (such as the nose, tail, drive roller, redirection roller, roller, etc.) and areas prone to failure are set as inspection points. The A* search algorithm is used to generate an optimal inspection route that takes off from the smart hangar, passes through each inspection point in turn, and finally returns to the smart hangar, and the route information is sent to the drone.

[0039] At the same time, the flight parameters of the drone are set on the server, such as flight altitude, speed, sensor acquisition frequency, etc., as well as relevant parameters of the inspection and identification module, such as the threshold for identifying the safety wearing status of workers and the standard for judging abnormal belt operation status, to ensure that the drone can accurately collect data and perform effective status identification during the inspection process.

[0040] The drone automatically takes off from the smart hangar and flies along the pre-planned inspection route. During the flight, the drone perceives the surrounding environment in real time through the laser radar it carries, and uses the multi-echo detection method and SLAM algorithm to continuously update its position and avoid obstacles. At the same time, it maintains a stable WiFi connection with the router through the host computer module, receives control commands sent by the server, and transmits the collected laser radar data, infrared sensor data, gas sensor data, and camera image data back to the server in real time.

[0041] When the drone passes over the conveyor belt, the camera captures images of the belt's running status, including whether the belt surface has defects such as tearing, scratching, and wear, whether the belt is deviating, and whether the rollers are stuck or falling off. The infrared sensor detects the temperature of the cable to determine whether the cable is damaged. The gas sensor monitors the gas concentration in the surrounding environment and immediately sends an alarm message to the server once a harmful gas leak is detected.

[0042] The built-in wireless network card and high-gain external antenna of the host computer module ensure stable communication with the router. Based on the TCP / IP protocol, it receives the server command data forwarded by the router, converts it into command format data that the drone can recognize, and sends it to the flight control system of the drone to control the flight attitude and movement of the drone. At the same time, the data obtained by the drone is stored in the high-speed cache area to prevent data loss, and the cached data is uploaded to the server at a certain frequency.

[0043] Drone return and charging: After the drone completes an inspection mission, it will automatically return to the smart hangar according to the pre-set return program. During the return process, if the communication between the drone and the server is abnormal, such as the server does not receive the heartbeat packet sent by the drone within the set threshold time (such as 5 seconds) or the signal strength is lower than the set threshold, the drone's manual remote control will automatically take over the control of the drone. The operator can manually control the drone to return through the joystick and buttons on the manual remote control according to the positioning indicator and camera image on the drone to ensure that the drone can return to the smart hangar safely.

[0044] After the drone returns to the smart hangar, the smart hangar exchanges information with the drone through the ZigBee wireless communication protocol and automatically detects the drone's battery level and flight status. If the battery level is lower than the set value, the smart hangar will automatically start the charging program to quickly charge the drone's battery and prepare for the next inspection mission. At the same time, the smart hangar will also perform simple maintenance checks on the drone, such as checking whether the fuselage is damaged and whether the sensors are working properly, to ensure that the drone is always in good operating condition.

[0045] Example 2

[0046] The difference between this embodiment and Embodiment 1 is that this embodiment provides a belt inspection method based on an indoor intelligent drone, including obtaining lidar data, infrared sensor data, gas sensor data and image data based on the indoor intelligent drone; planning the inspection route of the intelligent drone based on a server-based inspection algorithm module according to the acquired lidar data; obtaining images of the working status of workers based on a server-based inspection recognition module according to the acquired infrared sensor data; and identifying the working status of the belt based on a server-based belt detection module according to the acquired image data.

[0047] Specifically, after receiving the data transmitted back by the drone, the server of this embodiment first pre-processes and analyzes the data, and classifies and stores different types of sensor data and image data and performs preliminary analysis. Then, the inspection and recognition module is used to further process the image data, and through the target detection and behavior recognition algorithm based on deep learning, it is judged whether the workers are wearing safety helmets and safety belts correctly, and whether there are any illegal operations (such as crossing the running belt, staying in the dangerous area, etc.); at the same time, the belt working status module is used to comprehensively analyze the image and sensor data of the belt to judge whether the running state of the belt is normal, such as whether there are hidden faults such as deviation, surface defects, and bearing heating. If the inspection and recognition module detects that the worker has illegal operation behavior or the belt has abnormal operation, the server immediately issues an alarm message, reminds the operator in the monitoring center through the sound and light alarm device, and sends detailed fault information (including fault type, location, severity, etc.) to relevant maintenance personnel and management personnel in the form of text messages or system pop-up windows, so that they can take timely measures to deal with it.

[0048] The inspection route algorithm of this embodiment is intended to plan a safe route for the inspection drone that can avoid obstacles, automatically adjust the path, and ensure that all inspection points are covered, including:

[0049] First, the environment modeling is performed and the inspection area is divided into grids, each grid representing a small area. Figure 2 As shown in the figure, the yellow squares in the figure are regarded as divided grids. The grid positions of the starting point, target point, and obstacles are marked. At the same time, the key positions that may cause deviation (near the deviation point) and the redundant nodes for the backup path are identified.

[0050] When performing path planning, first initialize the A* algorithm:

[0051] Use the A* (A-Star) algorithm to find a preliminary path from the starting point to the target point. The A* algorithm selects the next node by comprehensively considering the actual cost from the starting point to the current node (g value) and the estimated cost from the current node to the target point (h value).

[0052] During initialization, the g value of the starting point is set to 0, and the h value is set to the straight-line distance from the starting point to the target point (Euclidean distance or Manhattan distance can be used).

[0053] Path search:

[0054] Starting from the starting point, search for neighboring grids that are passable (grids that are not marked as obstacles). For each neighboring grid, calculate its g value (the g value of the current node plus the movement cost from the current node to the neighboring node, usually 1) and h value (the estimated cost of the neighboring node to the target point). Select the node with the smallest f value (g+h) as the next node to be expanded. Repeat this process until the target point is reached or no feasible path can be found.

[0055] Autonomous obstacle avoidance: During the path search, if an obstacle is encountered: the algorithm will find a path around the obstacle. This can be achieved by checking the traversable grid around the obstacle.

[0056] Utilize redundant nodes to build alternate paths. For example, when encountering an obstacle in the graph, the algorithm can improve the path by connecting redundant nodes to avoid the obstacle.

[0057] If deviation occurs when approaching the target point (such as encountering a near deviation point):

[0058] The algorithm will dynamically adjust the path based on the relative position of the current position and the target point.

[0059] Recalculate portions of the route to ensure that the destination can be reached safely and accurately.

[0060] Path optimization: Once the preliminary path from the starting point to the target point is found, the path is optimized. Check whether there are unnecessary detours or redundant nodes in the path, try to simplify the path, reduce the moving distance and time of the inspection equipment, and use the path smoothing algorithm to eliminate sharp turns in the path to make the path smoother.

[0061] Dynamic adjustment: During the inspection process, the environment changes are monitored in real time. If a new obstacle is found or the position of an existing obstacle changes: the path replanning is triggered immediately. Based on the new environmental information, the optimal path from the current position to the target point is recalculated.

[0062] Fully connect the server, storage, drones, and smart hangars, and set inspection data parameters, including the temperature collection range and accuracy of the infrared sensor, the image resolution and frame rate of the camera, the gas type and detection threshold of the gas identification sensor, and charging parameters, such as the drone's charging current, voltage, and charging time.

[0063] Start the flight control software and inspection algorithm software to conduct system joint debugging test. During the test, simulate various belt conveyor failure conditions, such as placing simulated foreign objects on the belt, adjusting the roller position to simulate roller shedding, etc., to check whether the drone can accurately collect data and correctly analyze and process it through the inspection algorithm software, to achieve effective detection of infrared temperature measurement, roller shedding, bearing heating, foreign objects falling, belt deviation, material blockage identification, illegal personnel monitoring, safety helmet monitoring and other functions, and timely generate alarm information and inspection reports.

[0064] After the system debugging is completed and confirmed to be correct, the indoor intelligent drone belt inspection system is officially started. According to the preset inspection time and route, the drone automatically takes off from the intelligent hangar and inspects the belt conveyor along the set path. During the inspection process, the drone automatically avoids obstacles and adjusts the path. At the same time, it collects temperature, gas, video and other data through various sensors on board, and transmits the data to the server system through the host computer communication module.

[0065] The inspection algorithm software processes and analyzes the data in real time. Once an abnormal situation is detected, such as excessive temperature in a certain part of the belt conveyor or the roller falling off, an alarm message is immediately generated and displayed on the server side and related monitoring terminals. At the same time, an inspection report is automatically generated based on the data processing results. The report content includes the inspection time, inspection path, abnormal conditions found and related data, and processing suggestions.

[0066] Establish a special monitoring room or use a remote monitoring terminal to arrange professionals to monitor the drone inspection system in real time. The monitoring personnel can check the flight status of the drone, data collection and the operation status of the belt conveyor at any time.

[0067] Regularly perform maintenance on drones, smart hangars, servers, storage and other equipment. For drones, check whether the fuselage structure is damaged, whether the sensors are working properly, whether the battery performance is degraded, etc., and repair or replace them in time; for smart hangars, check whether the charging equipment is normal and whether the network connection is stable, etc. For servers and storage, perform maintenance work such as data backup, system update, hardware testing, etc. to ensure the stable operation of the entire inspection system.

[0068] The inspection algorithm software is continuously optimized based on the problems and needs found. For example, the accuracy and sensitivity of fault identification are improved, and the data processing speed and report generation format are optimized. At the same time, according to new inspection scenarios or needs, the types or parameters of sensors carried by drones are adjusted, such as adding sensors for detecting certain specific gases, or improving the low-light shooting capabilities of cameras, etc., to meet the inspection requirements of various indoor and outdoor scenarios such as factories, warehouses, and power distribution rooms.

[0069] In this embodiment, according to the acquired infrared sensor data, the inspection and recognition module based on the server obtains the image of the worker's working status. Due to the certain temperature difference between the human body and the surrounding environment, the temperature distribution of the helmet and the head worn by the worker will show a specific thermal characteristic pattern in the infrared image. The sensor collects data at a certain frame rate (for example, 10 frames per second), and transmits the collected raw infrared data to the server in real time through the communication link between the drone and the server. Using the target detection algorithm based on deep learning, the YOLO model is used to locate the approximate area of ​​the worker's head in the preprocessed infrared image. By training on a large number of infrared image data sets containing workers' heads, the characteristic pattern of the head is learned, and the head position in the image can be accurately detected and marked in the form of a rectangular frame.

[0070] After determining the head area, the features of the helmet are further searched in the head area. The helmet usually has a relatively regular shape and a specific temperature distribution pattern. The top and sides of the helmet can show a relatively uniform low temperature area in the infrared image (due to the heat insulation of its material), while the hat band part may show a relatively high temperature due to contact with the human body. By analyzing and extracting these features, the position and range of the helmet in the head area can be preliminarily determined.

[0071] In this embodiment, according to the acquired image data, the belt detection module based on the server identifies the working state of the belt, wherein the grayscale image is Gaussian smoothed to reduce the influence of noise in the image on edge detection. Gaussian smoothing is achieved by weighted averaging each pixel in the image and the pixels in its neighborhood, and the weight is determined by the Gaussian function. The commonly used Gaussian kernel size is 5x5 or 7x7, and the standard deviation is selected according to the noise situation. For each pixel (x, y) in the image, the pixel value G(x, y) after Gaussian smoothing is calculated as follows:

[0072]

[0073] Where I(x,y) is the pixel value of the original image at (x,y), k is the radius of the Gaussian kernel, and σ is the standard deviation. Gaussian smoothing can effectively remove noise from the image and make the edges of the image smoother, which is convenient for subsequent gradient calculation and edge detection.

[0074] The Sobel operator is used to calculate the gradient magnitude and direction of the image. The Sobel operator performs convolution operations on the image in the horizontal and vertical directions to approximately calculate the first-order derivative of the image. The Sobel operator Gx in the horizontal direction and the Sobel operator Gy in the vertical direction are as follows:

[0075] For each pixel (x, y) in the image, the gradient magnitude G(x, y) and direction θ(x, y) are calculated as follows:

[0076]

[0077] By calculating the gradient magnitude and direction, we can determine the direction in which the rate of change of pixels in the image is the largest, that is, the direction of the edge. At the same time, the magnitude of the gradient magnitude reflects the strength of the edge.

[0078] According to the approximate position and shape information of the belt (which can be roughly estimated by the pre-set belt area coordinate range or based on the edge detection result), the belt region of interest (ROI) is extracted from the original grayscale image. This can reduce the workload of subsequent calculations and avoid the interference of the background area on the judgment of belt shedding.

[0079] Calculate the average gray value Gbelt of the pixels in the belt ROI and the average gray value Gb of the pixels in the background area (the area within a certain range around the belt ROI can be selected as the background area). Then calculate the gray difference ΔG=G belt ―G background .

[0080] According to the preset grayscale difference threshold Tgap, which can be obtained by statistically analyzing a large number of belt images in normal and falling states, it is determined whether the belt is falling off. gap , it is considered that the belt may be in the normal position; if ΔG≤T gap , it indicates that the belt may have fallen off. At the same time, in order to improve the accuracy and reliability of the judgment, a comprehensive judgment can be made in combination with the integrity and continuity of the belt edge. If a large break or discontinuous area is found on the belt edge in the edge detection result, and the grayscale difference also meets the falling condition, it is judged that the belt has fallen off.

[0081] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A belt inspection system based on an indoor intelligent drone, characterized in that: include: Indoor drones, smart hangars and servers, wherein the indoor drones and smart hangars are respectively connected to the server through a router, and the server is provided with an inspection algorithm module, a belt working status module and an inspection identification module, and the inspection route of the drone is planned by the A* search algorithm of the inspection algorithm module, and the belt shedding condition is monitored by the canny edge detection algorithm of the belt working status module; The target detection algorithm of the inspection and recognition module is used to monitor the wearing of personnel safety devices.

2. The belt inspection system based on indoor intelligent drone according to claim 1 is characterized in that: The indoor drone is equipped with a laser radar, an infrared sensor, a gas sensor and a camera. Indoor positioning and obstacle avoidance are performed based on the multi-echo detection method and SLAM algorithm of the laser radar, the safety wearing status of the workers is judged based on the camera, the cable damage is judged based on the infrared radiation characteristics of the infrared sensor, and the gas concentration is converted into an electrical signal based on the semiconductor adsorption and electrochemical reaction of the gas sensor to judge the gas leakage.

3. The belt inspection system based on indoor intelligent drone according to claim 2 is characterized in that: A host computer module is also arranged between the router and the intelligent drone. The router and the host computer module are connected via WiFi. The host computer module has a built-in wireless network card and a high-gain external antenna. It receives command data from the router based on the TCP / IP protocol, converts it into command format data recognizable by the drone, and stores the data acquired by the drone in a high-speed cache area.

4. The belt inspection system based on indoor intelligent drone according to claim 3 is characterized in that: The smart hangar and the router are connected via a wired network, the router and the server are connected via a WiFi network, and the router is also connected to a mobile terminal via a WiFi network. The smart hangar and the router are connected via a Category 6 Ethernet cable. The smart hangar is equipped with a standard RJ45 Ethernet interface. A fixed static IP address is configured for the smart hangar on the router for precise network access and management control of the smart hangar. At the same time, the network access rights of the smart hangar are set using the router's access control list ACL.

5. The belt inspection system based on indoor intelligent drone according to claim 4 is characterized in that: The intelligent drone is also equipped with a manual remote control for manually controlling the drone to return after the drone is disconnected from the server. When the heartbeat packet sent by the drone is not received within the threshold time set by the server and the signal strength is lower than the set threshold based on the automatic disconnection detection method, the manual remote control automatically takes over the control of the drone and controls the drone to return through the staff.

6. The belt inspection system based on indoor intelligent drone according to claim 5 is characterized in that: The smart hangar and the smart drone are respectively equipped with ZigBee communication modules, and information is exchanged through the ZigBee wireless communication protocol to achieve automatic return and charging. A star network topology is used to build a ZigBee communication network between the smart hangar and the smart drone, with the smart hangar as the central node of the network, and communication connections are established between multiple drones as terminal nodes.

7. A belt inspection method based on an indoor intelligent drone, characterized in that: include: Acquire lidar data, infrared sensor data, gas sensor data and image data based on indoor intelligent drones; According to the acquired LiDAR data, the inspection algorithm module based on the server plans the inspection route of the intelligent drone; According to the acquired infrared sensor data, the inspection and recognition module based on the server acquires images of the workers' working status; According to the acquired image data, the belt detection module based on the server identifies the working status of the belt.

8. The belt inspection method based on indoor intelligent drone according to claim 7 is characterized in that: The server-based inspection algorithm module plans the inspection route of the intelligent drone, including using the A* search algorithm to train the intelligent drone for route planning, wherein after the inspection area is gridded, a preliminary path from the starting point to the target point is found, and the next node is selected by calculating the actual cost value from the starting point to the current node and the estimated cost value from the current node to the target point. For adjacent grids, the moving cost value from the current node to the adjacent node is calculated, and the estimated cost value from the adjacent node to the target point is calculated, and the node with the smallest cost value is selected as the next node to be expanded until the target point is reached or no feasible path can be found.

9. A belt inspection method based on indoor intelligent drone according to claim 8, characterized in that: The server-based inspection and recognition module obtains images of workers' working status, including detecting and locating workers using a target detection algorithm based on the acquired infrared sensor images, and identifying the wearing of workers' safety devices based on a bounding box form, wherein a YOLOv5 model is trained using a large number of labeled worker status images containing different working scenes, and the trained YOLOv5 model is used to detect workers' safety helmets and safety belts, and candidate regions that may contain objects are generated through a region proposal network, and then these candidate regions are classified and regressed to determine the category and position of the objects.

10. A belt inspection method based on indoor intelligent drone according to claim 9, characterized in that: According to the acquired image data, the server-based belt detection module identifies the working status of the belt, including Gaussian smoothing based on the acquired belt image using the Canny edge detection algorithm, detecting the edge of the belt by calculating the gradient amplitude and direction of the image, and determining the belt detachment condition by calculating the grayscale difference between the belt and the background.

Images