A method, system and computer-readable storage medium for intelligent river inspection using a fleet of unmanned aerial vehicles based on a predetermined route and autonomous control

A UAV fleet with autonomous control and real-time image analysis addresses the limitations of independent drone operations, enabling comprehensive and efficient river inspections by utilizing GPS data for precise routing and uninterrupted operations.

CN118732709BActive Publication Date: 2024-12-06SHANDONG ZHIYANG ELECTRIC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411231558.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-12-06
Estimated Expiration
2044-09-04

AI Technical Summary

Technical Problem

Existing river inspection systems using drone swarms are limited by independent drone operations, failing to fully utilize cluster collaboration for efficient and detailed inspections, leading to incomplete information acquisition and missed detection of river conditions.

Method used

A method and system employing a fleet of unmanned aerial vehicles (UAVs) with autonomous control, utilizing GPS coordinate point information to plan precise flight routes, real-time image analysis with Yolov8-based deep learning, and automatic battery replacement to ensure uninterrupted inspections, enabling detailed and comprehensive river monitoring.

Benefits of technology

The system achieves efficient, uninterrupted, and detailed river inspections, capturing fine-grained information and identifying anomalies, ensuring all-weather continuous monitoring and effective river management.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118732709B_ABST
    Figure CN118732709B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of intelligent control and image analysis, and more specifically, relates to a method, system and computer-readable storage medium for intelligent river inspection of drone fleets based on predetermined routes and autonomous control. The method includes obtaining accurate geographic information of the river; planning the flight routes of the drone fleet; performing real-time identification and inspection; performing uninterrupted inspections; abnormality detection and evidence collection; after the inspection is completed, the inspection data of the drone fleet will be integrated into the system and an inspection record report will be generated. The present invention solves the problem that the advantages of the cluster cannot be fully utilized for efficient collaboration, resulting in the inability to obtain fine-grained information on the river during drone inspections and the inability to perform fine inspections.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent control and image analysis, and more specifically, relates to a method, system and computer-readable storage medium for intelligent river inspection of a fleet of unmanned aerial vehicles based on a predetermined route and autonomous control. Background Art

[0002] As society's awareness of environmental protection increases, the management and protection of water resources has become increasingly important. As an important carrier of water resources, the environmental conditions of rivers directly affect the ecological balance and human life. Traditional river inspection methods mainly rely on manual labor, which has problems such as low efficiency, narrow coverage, and poor real-time performance, making it difficult to meet the needs of modern river management. In recent years, the rapid development of drone technology has provided new solutions for river inspections, especially the application of drone swarm technology, which has greatly improved the efficiency and accuracy of inspections.

[0003] In the existing river inspection solutions, usually one drone is used for inspection, flying along the center of the river, photographing and inspecting the entire river. If the length of the river and the inspection time are relatively long, multiple drone nests are deployed, and the drones land in the nests in turn to change points and take off again for inspection.

[0004] Chinese patent document CN105929845A discloses a river cruise system and cruise method based on a drone network, including a drone, a hangar and a ground control center. The hangar serves as a platform for placing the drone. Multiple hangars are arranged at intervals along one side of the river. The pan-tilt mechanism of the drone is provided with a camera and a loudspeaker. The ground control center communicates and transmits data with the hangar and the drone respectively. Each drone takes off and flies to the center of the river where its own hangar is located. When it flies along the river and shoots the center of the river where the next hangar is located, it turns around and returns to shoot the other side of the river. The ground control center performs image processing based on the video information transmitted back.

[0005] In summary, the application of drone swarms in river inspections can achieve large-scale, high-frequency monitoring, but the existing system is often limited to the independent operation of a single drone, and cannot fully utilize the advantages of the cluster for efficient collaboration, resulting in the inability of drones to obtain fine-grained information on the river during inspections, and unable to perform fine inspections. It may not be possible to inspect scattered floating garbage, swimming, and unable to distinguish between pedestrians and fishing behaviors on both sides of the river. If a single drone performs a fine inspection, it cannot obtain the full picture at once, and multiple inspections are required, with loopholes and blind spots, such as missing moving floating garbage. Summary of the invention

[0006] The present invention aims to overcome at least one defect of the above-mentioned prior art and provide a method and system for intelligent river inspection using a swarm of drones based on a predetermined route and autonomous control, so as to solve the problems that the existing systems are often limited to the independent operation of a single drone, and cannot fully utilize the advantages of the cluster for efficient collaboration, resulting in the inability to obtain fine-grained information on the river during drone inspections. The method is suitable for the inspection of rivers and channels in cities, and rivers with river banks and roads on both sides of the river.

[0007] The detailed technical scheme of the present invention is as follows:

[0008] A method for intelligent river inspection using a fleet of unmanned aerial vehicles based on a predetermined route and autonomous control, the method comprising:

[0009] In another aspect of the present invention, a method for intelligent river inspection of a drone fleet based on a predetermined route and autonomous control is provided, the method comprising:

[0010] S1. Obtain accurate geographic information of the river, the data format is the GPS coordinate point information of the river bank line;

[0011] Furthermore, the precise geographic information includes but is not limited to the GPS coordinates of the river bank, river width and shape.

[0012] S2. Obtain the river bank line according to the GPS coordinate point information of the river bank line, and plan the flight route of the drone fleet;

[0013] S3. When executing the route mission, each drone runs a Yolov8-based deep learning model to perform target detection and segmentation on the captured images, quickly identify key elements in the river channel, and perform real-time recognition.

[0014] S4, perform uninterrupted inspections. When the drone runs out of power during a mission and returns to the vehicle-mounted nest for battery replacement, the drone takes off again and continues the inspection work seamlessly.

[0015] S5. If any abnormality is detected during the inspection, the current route will be suspended, the aircraft will hover above the target, zoom in on the target, enlarge the target and make detailed records for evidence collection. After the evidence collection is completed, the mission will continue;

[0016] S6. After the inspection is completed, the inspection data of the drone fleet will be integrated into the system and an inspection record report will be generated.

[0017] The inspection data of all drones in the drone fleet will be integrated into the system, including the images, videos, abnormal event records, etc., and a complete and detailed inspection record report will be generated based on the inspection information of the drone fleet. The staff at the control end will conduct a detailed analysis of these data and generate an inspection report, including the overall condition of the river, a summary of abnormal events, the handling status, and areas that may need attention. These reports can serve as decision support to help managers understand changes in the river environment and formulate or adjust river management strategies. Through continuous and uninterrupted inspections and data analysis, the drone fleet river inspection system can ensure continuous monitoring of the river environment, timely detect and deal with problems, and thus achieve efficient management and protection of the river.

[0018] Furthermore, the river bank line obtained according to the GPS coordinate point information of the river bank line is specifically:

[0019] If the GPS coordinate point information of the river bank obtained is vector data, it is discretized into equally spaced discrete points, and then the line formed by connecting these points with straight lines is approximated as the river bank;

[0020] If the GPS coordinate point information of the river bank obtained is already an approximate curve formed by connecting discrete points, the approximate curve is used as the river bank. At the same time, the existing discrete points are equally spaced to obtain equally spaced discrete points.

[0021] Furthermore, the planning of the flight route of the drone fleet includes:

[0022] S21. There are N drones in the drone fleet. The width of the riverbank of two rivers is L. The drone fleet moves forward in a row. What is the width of each drone inspection? Should be:

[0023] (1);

[0024] Drone flight path width The midline position of the kth UAV is the waypoint Should be:

[0025] (2);

[0026] In formula (2), k=1, 2, ..., N;

[0027] S22, starting from one side of the river bank, the waypoint ID of the drone is compiled: drone α, drone β, ..., drone ε;

[0028] Then, the waypoints are calibrated starting from one end of the equally spaced discrete points. The equally spaced waypoints are subscripted from 1 to n for the drone’s waypoint ID, where n is the total number of discrete points. The drone waypoint set is obtained: {drone α1, drone α2, …, drone α n}, ..., {UAV ε1, UAV ε2, ..., UAV ε n};

[0029] S23. Fit the waypoints of each UAV to obtain the flight route of the UAV fleet.

[0030] The flight routes of drone fleets are planned based on discrete points and river banks, and the advantages of clusters are used for efficient collaboration. During drone inspections, fine-grained information about the river can be obtained, allowing for detailed inspections without multiple inspections. This not only enables all-round inspections, but also makes route planning more convenient and efficient.

[0031] Furthermore, the S4 specifically includes:

[0032] The starting point of the route is set at the departure point of the vehicle-mounted aircraft nest;

[0033] When the drone swarm is performing a mission, the vehicle-mounted nest vehicle drives around the drone swarm along the road routes on both sides of the river. When the drone route is completed or the drone battery is lower than the threshold, the drone flies above the vehicle-mounted nest, identifies the vehicle-mounted nest and lands quickly;

[0034] When the drone swarm lands in the nest, the route is suspended, the drone route is suspended, and the nest replaces the batteries of all the drones in the swarm. After the battery replacement is completed, they can take off again and continue to execute the route.

[0035] Furthermore, when the drone swarm is executing the route mission, it will autonomously control and move according to the order and position of the waypoint ID subscripts, and guide the drone to the next waypoint through the drone positioning information; at the bend of the river, each drone in the drone swarm controls its speed to offset the difference in moving distance between the outer circle and the inner circle, so that the order of the waypoint IDs at the same time is the same, thereby ensuring that the drone swarm has position synchronization when executing the route. The execution of the route mission also includes autonomously controlling the speed at the bend of the river:

[0036] The inspection flight speed of the drone group is pre-set. When the waypoint ID subscript number of a drone in the drone group is lower than the average waypoint ID subscript number of all current drones, its own speed will be increased; similarly, when the waypoint ID subscript number of a drone in the drone group is higher than the average waypoint ID subscript number of all current drones, its own speed will be reduced;

[0037] The speed is controlled every Z waypoints, increasing or decreasing the speed by Om / s each time.

[0038] Furthermore, when the river bank widens or narrows during the drone fleet inspection, the drone's gimbal camera is uniformly zoomed to ensure that all locations in the river are within the drone's inspection range. When the drone is flying, it will calculate and adjust the camera's zoom multiple according to the preset overlap rate (such as 20% or 10%). When the overlap rate calculated in real time is outside the set range, the drone's camera zoom multiple is adjusted to make the current overlap rate within the preset value. Generally, the overlap rate range can be set to (overlap rate-overlap rate X50%) to (overlap rate + overlap rate X50%). The execution of the route mission also includes automatic zooming when the river bank widens or narrows:

[0039] The pixel width of the river bank line on both sides of the drone’s edge from the nearest edge of the drone image , which is the pixel width of the area outside the river bank line;

[0040] The pixel width of the riverbank of the drone channel on both sides of the edge from the geometric center of the drone image is ;

[0041] Drone gimbal camera image pixel width ;

[0042] but , and The relationship is:

[0043] (3);

[0044] Overlap rate of images from two adjacent drones The calculation formula is:

[0045] (4);

[0046] If the width of the river bank changes, The overlap rate threshold is preset. When the overlap rate is greater than the threshold, the drone automatically zooms, and the new magnification F is:

[0047] (5);

[0048] In formula (5), F' is the magnification factor of the change in river bank width, is the pixel width before the river bank width changes The value of The pixel width after the width of the river bank changes value.

[0049] The present invention also includes an intelligent river inspection system based on a predetermined route and autonomously controlled drone fleet, including

[0050] Drone swarm: consists of multiple drones flying in coordination; performing tasks together.

[0051] PTZ camera: installed on the drone, with zoom function, used to shoot the river bank scene;

[0052] Artificial intelligence module: Integrates artificial intelligence image segmentation algorithms to achieve real-time detection and identification of river channels and riverbanks;

[0053] Video processing hardware: designed to process drone video streams and support real-time segmentation model inference operations;

[0054] Vehicle-mounted drone nest: equipped with battery replacement facilities, allowing the drone to land automatically and replace the battery;

[0055] Control and communication system: responsible for route planning, drone control, video data transmission, and receiving artificial intelligence recognition results.

[0056] In another aspect of the present invention, a machine-readable storage medium is provided, which stores executable instructions. When the instructions are executed, the machine executes the above-mentioned method for intelligent river inspection using a swarm of drones based on a predetermined route and autonomous control.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] (1) The present invention provides a method, system and computer-readable storage medium for intelligent river inspection based on a predetermined route and autonomous control of a drone fleet, which can achieve detailed river inspection in a short time, solve the problem that conventional river inspection by a single drone is not detailed enough, solve the problem that drones cannot simultaneously obtain the overall picture and overall information of the river during detailed inspection, and fill the loopholes and blind spots of multiple drone inspections. Only one inspection is needed to complete both the overall picture information of conventional river inspection and the detailed river inspection.

[0059] (2) The present invention provides a method, system and computer-readable storage medium for intelligent river inspection based on a predetermined route and autonomously controlled drone fleet. The method, system and computer-readable storage medium can automatically identify the river bank through an artificial intelligence algorithm. When encountering a target with hidden dangers in the river, the method can automatically hover, zoom in and take photos for evidence, and then continue the inspection. The method can automatically adjust the zoom of the drone by identifying the river bank, so that the entire river channel and river bank are within the field of view of the drone camera. While performing a detailed inspection, it can also ensure a regular inspection of the entire river channel and river bank. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1It is a flow chart of the method for intelligent river inspection by a fleet of unmanned aerial vehicles based on a predetermined route and autonomous control according to the present invention.

[0061] Figure 2 This is a schematic diagram of route planning and allocation in Example 1 of the present invention.

[0062] Figure 3 This is a schematic diagram of the overlapping fields of view during inspection by a fleet of drones in Example 1 of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0064] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0065] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0066] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0067] Example 1

[0068] Ginseng Figure 1 This embodiment provides a method for intelligent river inspection using a fleet of drones based on a predetermined route and autonomous control, the method comprising:

[0069] S1. Obtaining river channel geographic information: Obtaining accurate river channel geographic information in the form of GPS coordinate point information of the river channel bank line. The accurate geographic information includes but is not limited to the GPS coordinates of the river channel bank line, the width and shape of the river channel;

[0070] Preferably, the precise geographic information is obtained through satellite images, GIS system, three-dimensional point cloud information mapping and field measurement, and the final data format is the GPS coordinate point information of the river channel and riverbank.

[0071] S2. Obtain river bank and plan flight route: Obtain river bank according to GPS coordinate point information of river bank, and plan flight route of drone fleet;

[0072] Preferably, in this embodiment, the river bank line is obtained according to the GPS coordinate point information of the river bank line:

[0073] If the GPS coordinate point information of the river bank obtained is vector data, it is discretized at equal intervals to form dense points. Then, these points are connected by straight lines to form a line approximating the river bank. Generally, according to the accuracy requirements of the inspection, the discretization can be carried out at intervals of 1 meter, 5 meters, 10 meters, etc.

[0074] If the GPS coordinate point information of the river channel and river bank obtained is already an approximate curve formed by connecting discrete points, there is no need to discretize it, and finally the dense discrete point information and approximate curve of the discretized river channel and river bank are obtained.

[0075] Route design ensures that the drone can conduct effective and comprehensive inspections over the river to avoid missing any area. This can be achieved through fitting algorithms, equal division planning, etc. For example, if there are two drones, the system will divide the width of the river into four equal parts and assign them to the two drones, forming two non-intersecting routes to inspect both sides of the river at the same time. Figure 2 As shown in the figure, line1 and line2 are two solid lines representing the routes of the two UAVs respectively, and the dotted lines on both sides represent the location information of the river bank.

[0076] Preferably, planning the flight route of the drone fleet includes:

[0077] S21. According to the river bank and the number of drone groups, an algorithm is used to automatically generate an optimized flight route. The drone group has a total of N drones. The number of drones can be set independently based on the specific conditions of the river to be inspected. The width of the river bank of the two rivers is L, and the width of each drone inspection is Should be:

[0078] (1);

[0079] Then according to and the river bank positions on both sides, and obtain the route information of each drone in the drone fleet. The inspection width of each drone is , then from the river bank line on one side to the length of The area is the inspection range of the first drone in the drone fleet, so the route of this drone is the distance from the river bank line. location.

[0080] Then, the inspection width of each drone is , the drone route should be within the width The midline position of , then the route position of the kth UAV should be:

[0081] (2);

[0082] In formula (2), k=1, 2, …, N.

[0083] Calculate the drone’s route position After that, the locations of the drone route waypoints are recorded respectively. Since the width of the river bank L may change, the drone route location It will also change with the width L of the river bank, so the drone route is composed of multiple drone waypoints.

[0084] S22, starting from one side of the river bank, the waypoint ID of the drone is compiled: drone α, drone β, ..., drone ε;

[0085] Then, the waypoints are calibrated starting from one end of the equally spaced discrete points. The equally spaced waypoints are subscripted from 1 to n for the drone’s waypoint ID, where n is the total number of discrete points. The drone waypoint set is obtained: {drone α1, drone α2, …, drone α n}, ..., {UAV ε1, UAV ε2, ..., UAV ε n};

[0086] S23. Fit the waypoints of each UAV to obtain the flight route of the UAV fleet.

[0087] The route design ensures that the drone can conduct effective and comprehensive inspections over the river to avoid missing any area.

[0088] For example: For multi-UAV coordinated routes, if there are multiple UAVs, the system will calculate how to effectively distribute the UAVs to cover the entire river. For example, if there are two UAVs and the width of the river bank is 8 meters, according to the formula, the routes of the two UAVs are respectively at the positions of 2 meters and 6 meters of the river bank width, forming two non-intersecting routes, which can inspect both sides of the river at the same time and improve efficiency. Since the width of the river bank will change continuously, the flight route of the UAV is a curve.

[0089] The starting point of the route is usually set at the take-off and landing platform of the UAV. Preferably, in this embodiment, it is set at the starting point of the vehicle-mounted nest;

[0090] When the drone swarm is performing a mission, the vehicle-mounted nest vehicle drives around the drone swarm along the road routes on both sides of the river. When the drone route is completed or the drone battery is lower than the threshold, the drone flies above the vehicle-mounted nest, identifies the vehicle-mounted nest and lands quickly;

[0091] When the drone swarm lands in the nest, the route is suspended, the drone route is suspended, and the nest replaces the batteries of all the drones in the swarm. After the battery replacement is completed, they can take off again and continue to execute the route.

[0092] S3. Real-time recognition of captured images: During the route mission execution phase, each drone runs a Yolov8-based deep learning model to perform target detection and segmentation on the captured images to quickly identify key elements in the river channel.

[0093] After training, the model can perform target detection and segmentation on images captured by the camera in real time, enabling the drone to quickly identify key elements in the river during flight, such as garbage, aquatic plants, boats, swimmers, anglers, potential sewage outlets and other potential targets in river inspections, as well as identify river banks.

[0094] During the flight, the artificial intelligence recognition technology equipped on each drone will analyze the images captured by the camera in real time to complete normal river inspections, such as using detection algorithms to identify floating objects such as garbage on the water surface, aquatic plants, boats, swimming, fishing, sewage outlets, etc. At the same time, the drones on both sides of the edge will automatically identify the river bank.

[0095] Furthermore, when the drone fleet is performing a route mission, it will be autonomously controlled and moved according to the waypoint ID subscript order and position, and the drone positioning information, such as GPS data, will be used to guide the drone to the next waypoint;

[0096] When the drones in the swarm move at the bends of the river, there will be a difference in the moving distance between the outer circle and the inner circle. Each drone in the swarm controls its own speed so that the sequence of waypoint IDs at the same time is the same, thereby ensuring that the drone swarm has position synchronization when executing the route.

[0097] The inspection flight speed of the drone group is pre-set. When the waypoint ID subscript number of a drone in the drone group is lower than the average waypoint ID subscript number of all current drones, its own speed will be increased; similarly, when the waypoint ID subscript number of a drone in the drone group is higher than the average waypoint ID subscript number of all current drones, its own speed will be reduced;

[0098] Preferably, the speed is controlled automatically every 10 waypoints, increasing or decreasing the speed by 1 m / s each time.

[0099] Furthermore, the execution of the route mission also includes automatically zooming when the river bank widens or narrows:

[0100] When the drone fleet is inspecting, the drones on the two sides of the edge must not only identify hidden dangers, but also identify the river bank. Once the river bank line in the field of view is detected to change (the river bank becomes wider or narrower), the drone will calculate the zoom multiple of the drone gimbal camera based on the change of the river bank line, and then the drone fleet will automatically adjust the zoom multiple of the gimbal camera to keep the river bank always within the camera's field of view. When the drone is flying, it will calculate and adjust the camera's zoom multiple according to the preset overlap rate (such as 20% or 10%) so that there is enough overlap between two adjacent images, such as Figure 3 The drone fleet shown has overlapping fields of view during inspection.

[0101] This ensures continuous image coverage during the drone's flight mission, avoiding missing any possible problem areas. Since the route planning and allocation is divided equally according to the width of the river bank, the overlap rate of the images of two adjacent drone gimbal cameras is twice the image ratio outside the river bank. Image overlap allows continuous river bank sections to be recorded in different images, ensuring no-blind-angle monitoring of the river conditions.

[0102] Furthermore, the execution of the route mission also includes automatically zooming when the river bank widens or narrows:

[0103] The pixel width of the river bank line on both sides of the drone’s edge from the nearest edge of the drone image , which is the pixel width of the area outside the river bank line;

[0104] The pixel width of the riverbank of the drone channel on both sides of the edge from the geometric center of the drone image is ;

[0105] Drone gimbal camera image pixel width ;

[0106] but , and The relationship is:

[0107] (3);

[0108] Overlap rate of images from two adjacent drones The calculation formula is:

[0109] (4);

[0110] If the width of the river bank changes, The overlap rate threshold is preset. When the overlap rate is greater than the threshold, the drone automatically zooms, and the new magnification F is:

[0111] (5);

[0112] In formula (5), F' is the magnification factor of the change in river bank width, is the pixel width before the river bank width changes The value of The pixel width after the width of the river bank changes value.

[0113] S4. Perform uninterrupted inspections: After the drone runs out of power during a mission and returns to the vehicle-mounted nest for battery replacement, the drone can take off again and continue the inspection work seamlessly;

[0114] Furthermore, the starting point of the route is set at the departure point of the vehicle-mounted machine nest;

[0115] When the drone swarm is performing a mission, the vehicle-mounted nest vehicle drives around the drone swarm along the road routes on both sides of the river. When the drone route is completed or the drone battery is lower than the threshold, the drone flies above the vehicle-mounted nest, identifies the vehicle-mounted nest and lands quickly;

[0116] When the drone swarm lands in the nest, the route is suspended, the drone route is suspended, and the nest replaces the batteries of all the drones in the swarm. After the battery replacement is completed, they can take off again and continue to execute the route.

[0117] At the same time, the fast charging technology and drone nest replacement technology equipped in the vehicle can shorten the drone's downtime and improve the overall inspection efficiency. In this way, the drone fleet river inspection system can achieve all-weather, uninterrupted river environment monitoring, providing strong technical support for river management and environmental protection.

[0118] S5. Handle inspection anomalies. During the inspection process, if the drone detects an abnormal situation on the river, such as river pollution, illegal boat activities, swimming and fishing, etc., the drone fleet will pause the current route, hover above the target, zoom in on the target, enlarge the target and record it in detail, mark the location and take photos for evidence. After the evidence is collected, the drone resumes the suspended route and continues to perform the mission.

[0119] These abnormal information will be transmitted to the control end in real time for analysis and processing by professionals. According to the severity and urgency of the problem, the control end can quickly notify the relevant departments to take corresponding measures, such as dispatching personnel to verify on-site, initiating emergency response or conducting follow-up investigations. At the same time, the system will record all abnormal events for data analysis and inspection effect evaluation, optimize inspection strategies and improve response efficiency. In this way, the drone fleet river inspection system can efficiently and comprehensively monitor the river conditions, promptly handle and feedback abnormal problems, and ensure the safety and health of the river environment.

[0120] S6. Integrate inspection data and generate inspection record report: After the inspection is completed, the inspection data of all drones in the drone fleet will be integrated into the system, and a complete and detailed inspection record report will be generated based on the inspection information of the drone fleet.

[0121] The inspection data of all drones in the drone fleet will be integrated into the system, including the images, videos, abnormal event records, etc., and a complete and detailed inspection record report will be generated based on the inspection information of the drone fleet. The staff at the control end will conduct a detailed analysis of these data and generate an inspection report, including the overall condition of the river, a summary of abnormal events, the handling status, and areas that may need attention. These reports can serve as decision support to help managers understand changes in the river environment and formulate or adjust river management strategies. Through continuous and uninterrupted inspections and data analysis, the drone fleet river inspection system can ensure continuous monitoring of the river environment, timely detect and deal with problems, and thus achieve efficient management and protection of the river.

[0122] Example 2

[0123] This embodiment provides a river intelligent inspection system based on a predetermined route and autonomous control of a drone fleet, the system comprising:

[0124] Drone swarm: consists of multiple drones flying in coordination to perform tasks together;

[0125] PTZ camera: installed on the drone, with zoom function, used to shoot the river bank scene;

[0126] Artificial Intelligence Module: Integrates artificial intelligence image segmentation algorithms to achieve real-time detection and identification of river channels and riverbanks;

[0127] Video processing hardware: designed to process drone video streams and support real-time segmentation model inference operations;

[0128] Vehicle-mounted drone nest: equipped with battery replacement facilities, allowing the drone to land automatically and replace the battery;

[0129] Control and communication system: responsible for route planning, drone control, video data transmission, and receiving artificial intelligence recognition results.

[0130] Example 3

[0131] This embodiment also provides a computer-readable storage medium storing executable instructions, which, when executed, enable the machine to execute the isolation switch one-key sequential control video double confirmation method as described above.

[0132] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.

[0133] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute part of this specification.

[0134] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code may be downloaded from a server computer or a cloud via a communication network.

[0135] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0139] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A method for intelligent river inspection using a fleet of drones based on a predetermined route and autonomous control, characterized in that: The method comprises: S1. Obtain accurate geographic information of the river, the data format is the GPS coordinate point information of the river bank line; S2. Obtain the river bank line according to the GPS coordinate point information of the river bank line, and plan the flight route of the drone fleet; The river bank line obtained according to the GPS coordinate point information of the river bank line is specifically: If the GPS coordinate point information of the river bank obtained is vector data, it is discretized into equally spaced discrete points, and then the line formed by connecting these points with straight lines is approximated as the river bank; If the GPS coordinate point information of the river bank obtained is already an approximate curve formed by connecting discrete points, the approximate curve is used as the river bank, and at the same time, the existing discrete points are equally spaced to obtain equally spaced discrete points; The planning of the flight route of the drone fleet includes: S21. There are N drones in the drone fleet. The width of the riverbank of two rivers is L. The drone fleet moves forward in a row. What is the width of each drone inspection? Should be: (1); Drone flight path width The midline position of the kth UAV is the waypoint Should be: (2); In formula (2), k = 1, 2, ..., N; S22, starting from one side of the river bank, the waypoint ID of the drone is compiled: drone α, drone β, ..., drone ε; Then, the waypoints are calibrated starting from one end of the equally spaced discrete points. The equally spaced waypoints are subscripted from 1 to n for the drone’s waypoint ID, where n is the total number of discrete points. The drone waypoint set is obtained: {drone α1, drone α2, …, drone α n }, ..., {UAV ε1, UAV ε2, ..., UAV ε n }; S23, fitting the waypoints of each UAV to obtain the flight route of the UAV fleet; S3. When executing the flight mission, each drone runs a Yolov8-based deep learning model to detect and segment the captured images for real-time recognition; The execution of the route mission also includes automatic zooming when the river bank widens or narrows: The pixel width of the river bank line on both sides of the drone’s edge from the nearest edge of the drone image , which is the pixel width of the area outside the river bank line; The pixel width of the riverbank of the drone channel on both sides of the edge from the geometric center of the drone image is ; Drone gimbal camera image pixel width ; but , and The relationship is: (3); Overlap rate of images from two adjacent drones The calculation formula is: (4); If the width of the river bank changes, The overlap rate threshold is preset. When the overlap rate is greater than the threshold, the drone automatically zooms, and the new magnification F is: (5); In formula (5), F' is the magnification factor of the change in river bank width, The pixel width of the river bank line on both sides of the UAV from the geometric center of the UAV screen before the river bank line width changes; S4, perform uninterrupted inspections. When the drone runs out of power during a mission and returns to the vehicle-mounted nest for battery replacement, the drone takes off again and continues the inspection work seamlessly. S5. If any abnormality is detected during the inspection, the current route will be suspended, the aircraft will hover above the target, zoom in on the target, enlarge the target and make detailed records for evidence collection. After the evidence collection is completed, the mission will continue; S6. After the inspection is completed, the inspection data of the drone fleet will be integrated into the system and an inspection record report will be generated.

2. According to claim 1, a method for intelligent river inspection using a fleet of drones based on a predetermined route and autonomous control is characterized in that: The execution of the route mission also includes self-control of speed at river bends: The inspection flight speed of the drone group is pre-set. When the waypoint ID subscript number of a drone in the drone group is lower than the average waypoint ID subscript number of all current drones, its own speed will be increased; similarly, when the waypoint ID subscript number of a drone in the drone group is higher than the average waypoint ID subscript number of all current drones, its own speed will be reduced; The speed is controlled every Z waypoints, increasing or decreasing the speed by 1m / s each time.

3. The method for intelligent river inspection based on a predetermined route and autonomous control of a drone fleet according to claim 1 is characterized in that: The S4 specifically includes: The starting point of the route is set at the departure point of the vehicle-mounted aircraft nest; When the drone swarm is performing a mission, the vehicle-mounted nest vehicle drives around the drone swarm along the road routes on both sides of the river. When the drone route is completed or the drone battery is lower than the threshold, the drone flies above the vehicle-mounted nest, identifies the vehicle-mounted nest and lands quickly; When the drone swarm lands in the nest, the route is suspended, the drone route is suspended, and the nest replaces the batteries of all the drones in the swarm. After the battery replacement is completed, they can take off again and continue to execute the route.

4. The method for intelligent river inspection based on a predetermined route and autonomous control of a drone fleet according to claim 1 is characterized in that: The precise geographic information includes but is not limited to the GPS coordinates of the river bank, river width and shape.

5. A system for implementing the method for intelligent river inspection based on a predetermined route and autonomous control of a drone fleet as claimed in claim 1, characterized in that: The system comprises: Drone swarm: consists of multiple drones flying in coordination; performing tasks together; PTZ camera: installed on the drone, with zoom function, used to shoot the river bank scene; Artificial intelligence module: Integrates artificial intelligence image segmentation algorithms to achieve real-time detection and identification of river channels and riverbanks; Video processing hardware: designed to process drone video streams and support real-time segmentation model inference operations; Vehicle-mounted drone nest: equipped with battery replacement facilities, allowing the drone to land automatically and replace the battery; Control and communication system: responsible for route planning, drone control, video data transmission, and receiving artificial intelligence recognition results.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.