Unmanned Aerial Vehicle (UAV) Inspection System and Automated Inspection Methods
By combining the drone inspection system with machine learning models, the problem of manual intervention in drone inspections has been solved, achieving automated task management and data processing, and improving the completeness and accuracy of inspections.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WISTRON CORP
- Filing Date
- 2024-12-04
- Publication Date
- 2026-05-26
AI Technical Summary
Existing drone inspection technology requires operators to follow closely to control the flight path and mission execution. It lacks a unified system to manage multiple inspection tasks, and data collection and processing require manual intervention, making it difficult to guarantee completeness and accuracy.
Design a drone inspection system that includes a server and drones. The system receives inspection tasks through an interface, acquires environmental images using an image capture module, identifies anomalies using a machine learning model, and manages task scheduling and data processing through a unified interface.
It enables drones to automatically perform inspection tasks, reduces human intervention, improves data integrity and accuracy, and provides a unified task management system that is suitable for collaborative work among multiple units.
Smart Images
Figure CN122090660A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to a drone inspection system and automatic inspection method capable of automatically performing inspection tasks. Background Technology
[0002] In existing technologies, drones are widely used in various inspection tasks, especially in hazardous or inaccessible environments, such as high-voltage power towers, petrochemical equipment, and structural inspections after natural disasters. Drones can effectively inspect equipment in these environments, assisting in identifying structural anomalies or damage. However, current drone inspection technology still has several limitations. One is the need for operators to closely follow the drone to continuously monitor its flight path and mission execution. Furthermore, manual intervention is still required in many stages of data collection and processing to ensure data integrity and accuracy. Additionally, there is a lack of a unified system to manage inspection tasks when multiple related units wish to conduct them. Summary of the Invention
[0003] This disclosure proposes a drone inspection system and an automatic inspection method, which can overcome the shortcomings of conventional drone inspection tasks operated by humans.
[0004] This disclosure proposes a drone inspection system, comprising a server and a drone. The server provides an operating interface and receives inspection tasks, including inspection routes, through the interface. The drone is communicatively connected to the server. The drone includes an image acquisition module. The server performs scheduling to configure the drone to perform inspection tasks. The image acquisition module continuously acquires multiple environmental images, and the drone uses these images to determine its position and move along the inspection route. While moving along the inspection route, the drone transmits the environmental images to the server, which then uses these images to determine if any anomalies have occurred and generates inspection results corresponding to the inspection task.
[0005] In one embodiment of this disclosure, the drone also transmits its speed, altitude, and position to a server. The server determines whether the drone's travel path conforms to the inspection route; if the travel path does not conform to the inspection route, the server terminates the inspection task.
[0006] In one embodiment of this disclosure, the inspection route is a tunnel. The drone is used to detect the track and sidewalls in environmental imagery to calculate a first distance between the drone and the track and a second distance between the drone and the sidewalls. The drone is then positioned at the center of the tunnel based on the first and second distances.
[0007] In one embodiment of this disclosure, the server is used to predict the signal strength of the inspection route based on a machine learning model. When the signal strength in a region of the inspection route falls below a threshold, the server reduces the speed of the drone in that region.
[0008] In one embodiment of this disclosure, the inspection results include the types and locations of abnormal phenomena, such as cracks, water seepage, or rail distortion.
[0009] In one embodiment of this disclosure, when the server determines that an anomaly has occurred, the server controls the drone to hover, issues a warning message, and receives a response message from an external device. The server uses the response message to determine whether to control the drone to continue the inspection task.
[0010] From another perspective, embodiments of the present invention propose an automatic inspection method applicable to servers and drones. The drone includes an image acquisition module. The automatic inspection method includes: receiving an inspection task, which includes an inspection route, through an interface provided by the server; executing a schedule to set the drone to perform the inspection task, wherein the image acquisition module continuously acquires multiple environmental images; the drone identifying its position based on the environmental images and moving along the inspection route; while moving along the inspection route, the drone transmitting the environmental images to the server; and the server determining whether any abnormal phenomena have occurred based on the environmental images to generate inspection results corresponding to the inspection task.
[0011] In one embodiment of this disclosure, the automatic inspection method further includes: transmitting the speed, altitude, and position of the drone to a server, which uses the data to determine whether the drone's travel route conforms to the inspection route; and terminating the inspection task if the travel route does not conform to the inspection route.
[0012] In one embodiment of this disclosure, the inspection route is a tunnel, and the inspection method further includes: using a drone to detect the track and sidewalls in environmental images to calculate a first distance between the drone and the track and a second distance between the drone and the sidewalls; and using the drone to control the drone to be located at the center of the tunnel based on the first distance and the second distance.
[0013] In one embodiment of this disclosure, the above-described automatic inspection method further includes: the server predicting the signal strength of the inspection route based on a machine learning model; and when the signal strength of a region in the inspection route is lower than a critical value, reducing the speed of the drone in that region.
[0014] To make the above features and advantages of the present invention more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings.
[0015] In one embodiment of this disclosure, the above-mentioned automatic inspection method further includes: when the server determines that an abnormal phenomenon has occurred, the server controls the drone to hover, issues a warning message, and receives a reply message from an external device; and determines whether to control the drone to continue the inspection task based on the reply message. Attached Figure Description
[0016] Figure 1 This is a schematic diagram illustrating an unmanned aerial vehicle (UAV) inspection system according to one embodiment.
[0017] Figure 2 This is a schematic diagram illustrating the location identification of a drone according to one embodiment.
[0018] Figure 3 This is a schematic diagram illustrating the signal strength in a certain area according to an embodiment.
[0019] Figure 4 This is a flowchart illustrating the response when an anomaly is detected, based on one embodiment.
[0020] Figure 5 This is a flowchart illustrating an automatic inspection method according to one embodiment.
[0021] The reference numerals in the attached figures are explained as follows:
[0022] 110: Server
[0023] 120: Database
[0024] 131, 132: Drones
[0025] 131_C: Image Capture Module
[0026] 141: Drone Front-End Operation Interface
[0027] 142: Unmanned Aerial Vehicle (UAV) Mission Scheduling Management System
[0028] 143: Unmanned Aerial Vehicle Service System
[0029] 144: Artificial Intelligence Inspection Management System
[0030] 145: Unmanned Aerial Vehicle (UAV) Equipment Management System
[0031] 146: Drone Event Notification Management System
[0032] 210, 220: Sidewall
[0033] 230: Track
[0034] D1~D3: Distance
[0035] 310: Region
[0036] 401-411, 501-505: Steps
[0037] 420: External device Detailed Implementation
[0038] Some embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Component symbols used in the following description, when appearing in different drawings, are considered to be the same or similar components. These embodiments are only a part of the present invention and do not disclose all possible implementations of the invention. More precisely, these embodiments are merely examples of the systems and methods described in the claims of the present invention.
[0039] The terms "first," "second," etc., used in this article do not specifically refer to order or sequence; they are merely used to distinguish elements or operations described using the same technical terms.
[0040] Figure 1 This is a schematic diagram illustrating a drone inspection system based on one embodiment. Please refer to it. Figure 1 The drone inspection system includes a server 110 and multiple drones 131 and 132. Each drone 131 and 132 is equipped with at least one image acquisition module (e.g., image acquisition module 131_C), a communication module, or other sensors (e.g., an accelerometer, a positioning system, or an altitude sensor) to obtain its own speed, altitude, and other information.
[0041] Server 110 is electrically connected to database 120 and executes multiple interfaces and systems, which may include software or hardware. The interfaces and systems mentioned below are merely examples; in other embodiments, a system may be broken down into multiple subsystems, or multiple systems may be integrated together. In this embodiment, server 110 provides a drone front-end operation interface 141 (also simply referred to as the operation interface) and executes a drone task scheduling management system 142, a drone service system 143, an artificial intelligence inspection management system 144, a drone equipment management system 145, and a drone event notification management system 146. The drone inspection system provides a unified operation interface, allowing personnel from different departments or organizations to input inspection tasks. The drone inspection system automatically schedules and dispatches drones to perform inspection tasks, detects any abnormalities, and finally generates inspection results. These inspection results can be stored in database 120, and relevant personnel can also browse or analyze these inspection results through the operation interface. The functions of this interface and system will be described below.
[0042] The drone front-end operation interface 141 is used to provide relevant personnel with input and management of inspection tasks, and the server 110 receives inspection tasks through this interface. Relevant personnel can use this drone front-end operation interface 141 via a browser, mobile application, or computer application on a suitable device. Relevant personnel can set up regular inspection tasks (e.g., daily or weekly) or irregular inspection tasks (e.g., scheduled for a specific event). Inspection tasks include the date, inspection route, and designation of a particular drone. In some embodiments, this system is used in a subway system, where the inspection route includes a tunnel containing tracks. However, in other embodiments, the drone inspection system can also be used at sea, construction sites, or any other suitable location; this invention is not limited to these applications. In some embodiments, relevant personnel can also view inspection results through this interface.
[0043] The drone equipment management system 145 establishes communication links with drones 131 and 132 to transmit and receive data. In some embodiments, the drone equipment management system 145 exchanges data through an Application Programming Interface (API), which can be structured or unstructured files. Structured files include JavaScript Object Notation (JSON), which stores data from a relational database, such as flight information like altitude, speed, and position. Unstructured files include video streams and photos. The API allows drones 131 and 132 to synchronize with server 110 and obtain system information from server 110, including Message Queuing Telemetry Transport (MQTT) paths (for transmitting aircraft information) and Real Time Messaging Protocol (RTMP) paths (for transmitting video streams). Data received by the drone equipment management system 145 can be stored in database 120 for future retrieval and verification. In some embodiments, the unmanned aerial vehicle (UAV) equipment management system 145 transmits data with UAVs 131 and 132 using Transport Layer Security (TLS) / Secure Sockets Layer (SSL) to prevent data from being eavesdropped on or tampered with during transmission.
[0044] The UAV mission scheduling management system 142 has at least two main functions: mission assignment and mission path setting. Mission assignment is used to schedule inspection tasks according to the pre-set tasks, thereby setting which UAV will perform which inspection task at what time. Mission path setting is used to pre-set inspection routes for UAVs 131 and 132 based on the path data provided by server 110, avoiding errors in UAV inspection tasks caused by human scheduling mistakes.
[0045] The drone service system 143 is used to monitor the status, flight path, and video streaming status of drones 131 and 132 in real time. The status of drones 131 and 132 includes position, altitude, speed, and battery level. Taking drone 131 as an example, during the inspection mission, drone 131 continuously acquires multiple environmental images. Based on these images, the drone identifies its own position and moves along the inspection route. For example... Figure 2 This is a schematic diagram illustrating the location identification of a drone according to one embodiment. The inspection route described above is a tunnel with sidewalls 210 and 220 and a track 230. In some embodiments, the track 230 has markings with numbers indicating the distance between the markings and the starting point (or ending point), which the drone 131 can identify to confirm its position. In some embodiments, the drone 131 can also identify the track 230 and the sleepers below it, thereby continuing along the track and calculating the sleeper comparison to its current position until it reaches the end of the track 230 and then returns to the starting point. In addition, the drone 131 also detects the sidewalls 210 and 220 and the track 230 in the environmental imagery, thereby calculating the distance D3 between the drone 131 and the track 230, and the distances D1 and D2 between the drone 131 and the sidewalls 210 and 220. The drone 131 will control itself to be in the center of the tunnel based on these distances D1 to D3. For example, it will make the distance D1 the same as the distance D2, and also set the distance D3 within a certain range to avoid colliding with the side walls 210 and 220 or flying too low and colliding with other obstacles.
[0046] In some embodiments, the drone 131 transmits its speed, altitude, and position to the drone service system 143, which then determines whether the drone 131's path conforms to the pre-set inspection route. If the drone 131's path does not conform to the inspection route, the drone service system 143 can terminate the inspection task, reset the inspection route, or replan the drone 131's path. For example, if there are obstacles on the inspection route, and the drone 131 deviates from the route to avoid them, the drone service system 143 can reset the inspection route based on the current environmental conditions and the inspection task, ensuring that the drone 131 can avoid obstacles and complete the task according to the new inspection route.
[0047] In some embodiments, the server 110 and the drone 131 communicate via a 5G mobile network (or cellular network) or other high-speed wireless communication technologies. The drone service system 143 also monitors the strength of the communication signal, latency, and data integrity (whether there are lost packets or unrecoverable errors) in real time. If video streaming is interrupted or the latency is too high, the drone service system 143 can also interrupt the inspection task.
[0048] In some embodiments, the drone 131 flies back and forth in the tunnel, thus passing through the same area multiple times, and can collect the signal strength of the area each time it passes. The drone service system 143 can predict the signal strength along the inspection route based on a machine learning model, and can reduce the speed of the drone 131 in the area when the predicted signal strength is below a threshold. Figure 3 This is a schematic diagram illustrating the signal strength in a certain area, based on an embodiment. Please refer to it. Figure 3 The drone service system 143 can collect historical signal strength within area 310 and then input this historical signal strength into a machine learning model, such as a Long Short-Term Memory (LSTM) model, to predict future signal strength. When the predicted signal strength within area 310 is lower than a critical value, the speed of the drone 131 passing through area 310 will be reduced to avoid damage to the drone 131 due to insufficient reaction in the event of an abnormal situation.
[0049] Please refer to Figure 2The AI-powered inspection and management system 144 executes a machine learning model to determine if any anomalies exist. Specifically, environmental images captured by the drone 131 are transmitted to the AI-powered inspection and management system 144. The executed machine learning model can be a decision tree, random forest, multi-layered neural network, convolutional neural network, support vector machine, etc., which are not limited to this invention. In some embodiments, the types of anomalies include cracks, water seepage, or rail distortion, but these are not limited to this invention. When an anomaly occurs, the AI-powered inspection and management system 144 also records the location of the anomaly and writes both the location and type into the database 120. After the drone 131 completes its inspection task, the server 110 generates inspection results. These results include whether any anomalies occurred, and if so, the results also include information such as the type and location of the anomaly. The inspection results also include information such as the inspection route, time, the drone that performed the task, and relevant environmental images. Relevant personnel can view or analyze these inspection results through the drone's front-end operation interface 141.
[0050] Figure 4 This is a flowchart illustrating the response to an anomaly detected, based on one embodiment. Please refer to it. Figure 1 and Figure 4 In step 401, UAV 131 performs an inspection task. In step 402, UAV 131 starts streaming and sends the stream to server 110. Simultaneously, in step 403, UAV 131 flies along the inspection route. In step 404, server 110 converts the stream into images (referred to as environmental images). In step 405, server 110 identifies anomalies in these environmental images. Next, in step 406, it checks if there are any anomalies. If not, it returns to step 404 to continue processing the subsequent streams. If there are anomalies, server 110 controls UAV 131 to hover, land, or directly terminate the inspection task and return to base. For example, in step 407, the UAV checks if it has received an instruction from server 110 indicating whether there are any anomalies. If there are no anomalies, UAV 131 returns to step 403 to continue flying. If there are anomalies, UAV 131 executes step 408 to hover (or land).
[0051] On the other hand, server 110 sends a warning message to external device 420 through drone event notification management system 146. This warning message can be provided to external device 420 in various forms such as email, information in applications or web pages, and SMS. In step 409, external device 420 notifies relevant personnel to check this warning message. In step 410, relevant personnel or units intervene to determine whether drone 131 should continue to perform the inspection task. Next, external device 420 provides a response message to server 110, and server 110 determines whether to control drone 131 to continue the inspection task based on this response message (for simplicity, ...). Figure 4 The diagram shows the arrow from step 410 to drone 131, because at this point, it is actually the external device 420 that decides whether to continue the inspection task. In step 411, drone 131 determines whether to continue the inspection task based on the response information. If yes, it returns to step 403; otherwise, the process ends.
[0052] Figure 5 This is a flowchart illustrating an automated inspection method according to one embodiment. In step 501, an inspection task, including an inspection route, is received through an interface provided by the server. In step 502, scheduling is performed to set a drone to perform the inspection task, wherein the drone's image acquisition module continuously acquires multiple environmental images. In step 503, the drone identifies its position based on the environmental images and moves along the inspection route. In step 504, while the drone is moving along the inspection route, it transmits the environmental images to the server. In step 505, the server determines whether an anomaly has occurred based on the environmental images and generates an inspection result corresponding to the inspection task. Figure 5 The steps involved have been explained in detail above, and will not be repeated here. It is worth noting that... Figure 5 Each step can be implemented as multiple program codes or circuits, but this invention is not limited thereto. Furthermore, Figure 5 The method can be used in conjunction with the above embodiments or alone; in other words, Figure 5 Other steps can also be added between the various steps.
[0053] The aforementioned system and method provide a unified user interface that allows personnel from various units to set inspection tasks, solving the problem of manually setting tasks and scheduling. Furthermore, since GPS cannot be used in locations such as tunnels, the above embodiment uses imagery captured by drones for positioning. After completing the inspection task, the drone automatically returns to its home base, avoiding the need for operators to move with the drone. Finally, the system and method utilize artificial intelligence models to automatically identify anomalies. All judgment results and relevant drone data are stored in a database, which can be viewed by relevant personnel through the user interface.
[0054] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make some modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.
Claims
1. A drone inspection system, comprising: A server is used to provide a user interface and receive an inspection task through that interface, the inspection task including an inspection route; and A drone, which is connected to the server, contains an image capture module. The server is used to schedule the drone to perform the inspection task, and the image acquisition module is used to continuously acquire multiple environmental images. The drone identifies its position based on these environmental images and moves along the inspection route. When the drone travels along the inspection route, it transmits multiple environmental images to the server. The server uses these images to determine if an anomaly has occurred and generates an inspection result corresponding to the inspection task.
2. The drone inspection system as described in claim 1, wherein the drone also transmits its speed, altitude, and position to the server, which uses this information to determine whether a single travel path of the drone conforms to the inspection route. If the travel route does not conform to the inspection route, the server will terminate the inspection task.
3. The drone inspection system as described in claim 1, wherein the inspection route is a tunnel, and the drone is used to detect the track and sidewalls in the multiple environmental images to calculate a first distance between the drone and the track and a second distance between the drone and the sidewalls. The drone is positioned at the center of the tunnel based on the first distance and the second distance.
4. The drone inspection system as described in claim 1, wherein the server is used to predict a signal strength along the inspection route based on a machine learning model. When the signal strength in a region along the inspection route falls below a critical value, the server reduces the speed of the drone in that region.
5. The drone inspection system as described in claim 1, wherein the inspection result includes the type and location of the anomaly, the type including cracks, water seepage, or rail distortion.
6. The drone inspection system as described in claim 1, wherein when the server determines that the abnormal phenomenon has occurred, the server controls the drone to hover, issues a warning message, and receives a response message from an external device. The server is used to determine whether to control the drone to continue the inspection task based on the response information.
7. An automatic inspection method applicable to a server and a drone, wherein the drone includes an image acquisition module, the automatic inspection method comprising: A patrol task is received through an operation interface provided by the server, wherein the patrol task includes a patrol route. A schedule is executed to set the drone to perform the inspection task, wherein the image acquisition module is used to continuously acquire multiple environmental images; The drone identifies its location based on multiple environmental images to travel along the inspection route; As the drone travels along the inspection route, it transmits multiple environmental images to the server. as well as The server determines whether an anomaly has occurred based on the multiple environmental images and generates an inspection result corresponding to the inspection task.
8. The automatic inspection method as described in claim 7, further comprising: The speed, altitude, and position of the drone are transmitted to the server, which then uses them to determine whether the drone's route conforms to the inspection route. as well as If the route does not conform to the inspection route, the inspection task shall be terminated.
9. The automatic inspection method as described in claim 7, wherein the inspection route is a tunnel, and the inspection method further includes: The drone detects the track and sidewall in multiple environmental images to calculate the first distance between the drone and the track and the second distance between the drone and the sidewall; as well as The drone is positioned at the center of the tunnel based on the first distance and the second distance.
10. The automatic inspection method as described in claim 7, further comprising: The server predicts a signal strength along the inspection route based on a machine learning model. as well as When the signal strength in a region of the inspection route falls below a critical value, the speed of the drone in that region is reduced.
11. The automatic inspection method as described in claim 7, wherein the inspection result includes the type and location of the abnormal phenomenon, the type including cracks, water seepage, or rail distortion.
12. The automatic inspection method as described in claim 7, further comprising: When the server determines that the abnormal phenomenon has occurred, the server controls the drone to hover, issues a warning message, and receives a response message from an external device; as well as Based on the response information, determine whether to control the drone to continue the inspection mission.