Method for automatically updating water area three-dimensional map by using inspection unmanned aerial vehicle

Abstract

The method for automatically updating the water area three-dimensional map by using the inspection unmanned aerial vehicle comprises the following steps: S1, marking a flight boundary, an unmanned aerial vehicle flight airspace and an unmanned aerial vehicle flight sub-airspace in an existing three-dimensional map when the unmanned aerial vehicle inspects, dividing into cube-shaped three-dimensional grids, setting an inspection point in each three-dimensional grid, all the three-dimensional grids are marked as uninspected three-dimensional grids, an inspection unmanned aerial vehicle release point and an inspection unmanned aerial vehicle recovery point are arranged on the outer side of each unmanned aerial vehicle inspection sub-airspace, an inspection unmanned aerial vehicle is released at the inspection unmanned aerial vehicle release point, and the inspection unmanned aerial vehicle flies through a satellite positioning device; and the inspection unmanned aerial vehicle flies to the inspection point in the three-dimensional grid and hovers, and inspection is started. According to the method, the combination of unmanned aerial vehicle inspection and map updating can be realized, and the updating efficiency of the three-dimensional map is greatly improved while the high efficiency and no dead angle in the inspection process are kept.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) control, and in particular to a method for automatically updating a three-dimensional map of a water area using an inspection UAV. Background Art

[0002] In areas with inland waterways or coastlines, waterway inspections are a key task for relevant management departments. These tasks include monitoring incoming and outgoing ships, monitoring the surrounding landforms, monitoring illegal activities on the water surface, and inspecting navigation marks. Traditional inspections involve personnel operating boats within the waters. This method is slow, covers a limited area, and is labor-intensive. With the rapid development of drone and wireless communication technologies, these methods are gradually being replaced by drone inspections.

[0003] Drone water inspections allow workers to wirelessly control existing 2D or 3D drone navigation maps indoors, flying over the relevant waters as indicated. Alternatively, the system can automatically control drones along pre-set routes. Using the drone's onboard observation equipment, the drone can monitor, observe, and collect evidence on targets within the waters, achieving far greater efficiency than traditional inspection methods. The importance of the 3D map used by the navigation system is self-evident during drone inspections, especially during autonomous flight.

[0004] However, in existing technologies, the production of 3D maps requires a map production company, and subsequent map updates require timely on-site data collection and mapping by the 3D map company, significantly limiting the frequency of updates. Currently, to mitigate the limited field of view caused by foggy weather, some drone inspections are equipped with lightweight synthetic aperture radars (SARs). These radar waves generate the images needed for inspections. Immediately utilizing these images for 3D image updates significantly improves the speed and frequency of map updates. Therefore, the present invention aims to combine the 3D image generation process of SAR on water area inspection drones with the updating of 3D maps for navigation. This provides a method and system for automatically updating 3D maps during drone inspections, improving the efficiency of traditional drone inspections and 3D map updates. Summary of the Invention

[0005] In order to solve the above-mentioned deficiencies in the prior art, the present invention provides a method for automatically updating a three-dimensional map of a water area using an inspection drone.

[0006] A method for automatically updating a three-dimensional map of a water area using an inspection drone, characterized in that the method is performed in sequence according to the following steps: Step S1: Marking the flight boundary, drone flight airspace and drone flight sub-airspace of the drone during inspection in the existing three-dimensional map, dividing the three-dimensional grid into cube-shaped grids, setting inspection points in each three-dimensional grid, marking all three-dimensional grids as uninspected three-dimensional grids, setting inspection drone release points and inspection drone recovery points at the outer positions of each drone inspection sub-airspace, releasing the inspection drone at the inspection drone release point, and flying the inspection drone through a satellite positioning device. The inspection drone flies to the inspection point in the three-dimensional grid and hovers, and starts the inspection, and executing step S2; Step S2: The inspection drone uses a synthetic aperture radar to scan the surrounding environment of the three-dimensional grid to obtain a three-dimensional inspection map. The inspection drone uses the communication equipment to upload the inspection stereo image to the server and executes step S3; step S3: the inspection stereo image is compared with the three-dimensional environment around the stereo grid in the existing three-dimensional map in the server. When the inspection stereo image matches the existing three-dimensional map, the stereo grid where the inspection drone is located is marked as an inspected stereo grid, and the server sends a continue inspection message to the inspection drone and executes step S4; step S4: after receiving the continue inspection message, the inspection drone stops hovering, flies to the inspection point in the uninspected stereo grid that is closest to the inspected stereo grid, hovers, and executes the steps in sequence starting from step S2; when the inspection drone calculates that the airborne time is lower than the time threshold for the inspection drone to fly to the inspection drone recovery point, the inspection drone stops the above steps and flies to the inspection drone recovery point.

[0007] Furthermore, step S1 specifically includes the following steps that are executed in sequence: step S11: marking the flight boundary of the drone during inspection in the existing three-dimensional map, the flight boundary including the outer boundary and the inner obstacle boundary, and the space inside the boundary is marked as the drone flight airspace; step S12: dividing the drone flight airspace into a cube-shaped three-dimensional grid, the side length of the three-dimensional grid is less than or equal to 40% of the maximum working distance of the synthetic aperture radar of the inspection drone, the center point of the three-dimensional grid is set as the inspection point, and the satellite positioning information of each inspection point is measured and marked in the three-dimensional map;

[0008] Step S13: According to the priority of daily inspections, the drone flight airspace is divided into drone flight sub-airspaces consisting of multiple adjacent three-dimensional grids. Each inspection drone performs a flight inspection only in one drone inspection sub-airspace, and all three-dimensional grids in each drone flight sub-airspace are marked as uninspected three-dimensional grids; Step S14: Each inspection drone sets a three-dimensional grid as the flight starting point in its corresponding drone inspection sub-airspace, and all equipment carried by the inspection drone is uniformly energy-managed, and the length of time the inspection drone stays in the air is monitored in real time according to the energy consumption situation; Step S15: At least one inspection drone release point and inspection drone recovery point are set at the outer position of each drone inspection sub-airspace, and at least one inspection drone corresponding to it is released at the inspection drone release point. The inspection drone flies through a satellite positioning device. After entering the flight boundary, the inspection drone only flies in the drone flight airspace. The inspection drone flies to the satellite positioning coordinates of the inspection point in the three-dimensional grid set as the flight starting point, hovers at the inspection point, and starts the inspection.

[0009] Furthermore, step S3 also includes: step S31: when there is a partial mismatch between the inspection stereo image and the existing three-dimensional map, the mismatched position is marked in the inspection stereo image, the inspection drone remains hovering, and after a period of time, the synthetic aperture radar is used again to scan the surrounding environment of the stereo grid to obtain a re-inspection stereo image, and the inspection drone uses a communication device to upload the re-inspection stereo image to the server, and the inspection stereo image is compared with the re-inspection stereo image in the server to determine whether the inspection stereo image and the re-inspection stereo image match, and to determine whether the position where the inspection stereo image does not match the existing three-dimensional map is located in the drone's flight airspace and whether it is stationary.

[0010] Furthermore, in the step S3, after step S31, it also includes: step S32a: when it is determined that the local mismatch position between the inspection stereo image and the existing three-dimensional map is located outside the drone's flight airspace and is in a stationary state, the camera equipment carried by the inspection drone is turned on, and the camera equipment is manually remotely controlled to turn to the mismatch position to shoot image data and manually judge the nature of the mismatch position. The inspection drone uploads the re-inspected stereo image and the shot image data to the server, and the re-inspected stereo image is embedded in the existing three-dimensional map based on the matching part in the server, and the existing three-dimensional map is updated. The three-dimensional grid where the inspection drone is located is marked as an inspected three-dimensional grid, and the server sends a continue inspection information to the inspection drone, and executes step S4.

[0011] Furthermore, in the step S3, after step S31, it also includes: step S32b: when it is determined that the local mismatch position between the inspection stereo image and the existing three-dimensional map is located within the drone's flight airspace and is in a stationary state, the camera equipment carried by the inspection drone is turned on, and the camera equipment is manually remotely controlled to turn to the mismatch position to shoot image data and manually determine the nature of the mismatch position; when it is manually determined that the nature of the mismatch position belongs to a fixed object, the inspection drone uploads the re-inspection stereo image and the shot image data to the server, marks the mismatch position of the re-inspection stereo image as an obstacle in the server, constructs a digital surface model after adding the minimum avoidance distance outside the obstacle, and then embeds the re-inspection stereo image into the server based on the matching part. In the existing three-dimensional map, the existing three-dimensional map is updated, wherein the digital surface model boundary of the obstacle is set as the internal obstacle boundary, the three-dimensional grid where the inspection drone is located is marked as the inspected three-dimensional grid, and the server sends a continue inspection message to the inspection drone, and executes step S4; when it is manually determined that the nature of the unmatched position is a movable object, the inspection drone uploads the re-inspected three-dimensional image and the captured image data to the server, deletes the three-dimensional image of the unmatched position in the re-inspected three-dimensional image on the server, and then embeds the re-inspected three-dimensional image into the existing three-dimensional map based on the matching part, updates the existing three-dimensional map, marks the three-dimensional grid where the inspection drone is located as the inspected three-dimensional grid, and sends a continue inspection message to the inspection drone, and executes step S4.

[0012] Furthermore, in the step S3, after step S31, it also includes: step S32c: when it is determined that the position where the inspection stereo image and the existing three-dimensional map are partially mismatched is located within the drone's flight airspace and is in a moving state, the camera equipment carried by the inspection drone is turned on, and the camera equipment is manually remotely controlled to turn to the mismatched position to shoot image data and manually determine the nature of the mismatched position. The inspection drone uploads the re-inspection stereo image and the shot image data to the server, deletes the three-dimensional image of the mismatched position in the re-inspection stereo image on the server, and then performs the re-inspection stereo image based on the matching part. Embed it into the existing three-dimensional map, update the existing three-dimensional map, and mark the three-dimensional grid where the inspection drone is located as the inspected three-dimensional grid; manually judge whether the unmatched position needs to be tracked. When it is judged that the unmatched position does not need to be tracked, the server sends a continue inspection message to the inspection drone and executes step S4; when it is judged that the unmatched position needs to be tracked, the inspection drone stops hovering, keeps the camera equipment turned on, and the captured image data is uploaded to the server in real time. The console manually controls the shooting direction of the camera equipment and the flight of the inspection drone. When the inspection drone flies to the outer boundary and the internal obstacle boundary, the console issues an alarm.

[0013] Furthermore, the camera device is a low-light camera.

[0014] Furthermore, in step S1: at least one inspection drone is released at a time in each drone inspection sub-airspace, and the flight starting point of each inspection drone is set to a different three-dimensional grid in its corresponding drone inspection sub-airspace.

[0015] Furthermore, in step S4, it also includes: when all the three-dimensional grids in the drone flight sub-airspace are marked as inspected three-dimensional grids, the server sends a stop inspection message to the inspection drone, and all the inspection drones in the drone flight sub-airspace fly back to the inspection drone recovery point to obtain an updated three-dimensional map, and execute in sequence starting from step S13.

[0016] Compared with the existing technology, the present invention has the following beneficial effects: the present invention divides the inspection area into grids of reasonable size. During the automatic inspection process, on the one hand, the entire inspection area can be fully covered without blind spots by inspecting each grid one by one. At the same time, the stereoscopic image of the surrounding environment generated by the synthetic aperture radar during the inspection process can be used. Through a series of comparison methods, it can be determined whether the change in the image content belongs to an environmental change, and the three-dimensional map can be automatically updated, realizing the combination of drone inspection and map update. While maintaining the high efficiency and no blind spots in the inspection process, the update efficiency of the three-dimensional map is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a method for automatically updating a three-dimensional map of a water area using an inspection drone according to the present invention;

[0018] Figure 2 Schematic diagram of the UAV flight process in an embodiment of the method of the present invention. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0020] like Figure 1 、 Figure 2 As shown, an embodiment of the present invention provides a method for automatically updating a three-dimensional map of a water area using a patrol drone. In this embodiment, the scheme is carried out according to the following steps: Step 1: Preprocess the three-dimensional map and launch the drone.

[0021] In step 1, first, mark the flight boundary of the drone during inspection on the existing 3D map, that is, the outermost boundary of the drone during flight and the outer surface of obstacles in the flight space. Generally speaking, it corresponds to the outer boundary and the inner obstacle boundary. The outer boundary includes the embankment, dam, road, and water surface of the water area. Figure 2 The cross-section of the river channel is shown in the figure. Its outer boundaries include the riverbed and floodplain at the bottom of the river channel, the valley slopes on both sides of the river channel, the terraces on the right side of the river channel cross-section, and the valley shoulders at the highest points on both sides of the river channel, the plane at the highest flight altitude required for inspection, etc. Internal obstacles include islands, bridges, fixed buildings, signal base stations, etc. in the water area. The internal obstacles are not shown in the figure. The space inside the boundary is marked as the drone flight airspace. In order to facilitate the display of subsequent processes, the marked flight boundary is not shown in the figure.

[0022] Then, the drone's flight airspace is divided into cube-shaped grids. Each grid is completely free of obstacles for drone flight. The side length of the grid is less than or equal to 40% of the maximum operating range of the inspection drone's synthetic aperture radar. The center of the grid is set as an inspection point, and the satellite positioning information of each inspection point is measured and annotated on the three-dimensional map. As a result, when a drone is within the operating range of the drone's synthetic aperture radar, it can completely cover the grid in which the drone is located and the 26 adjacent grids. The maximum operating range of small and medium-sized synthetic aperture radars is generally within 3 kilometers. In this embodiment, the grid side length is set to 300 meters.

[0023] Next, based on the priority of daily inspections, the drone flight airspace is divided into multiple drone flight sub-airspaces consisting of adjacent 3D grids. Sub-airspaces with higher inspection priorities, such as those with frequent ship traffic, frequent illegal sand and gravel mining, and illegal sewage discharge, will be used to launch larger numbers of drones. Lower-priority areas will be used for smaller inspections. Each inspection drone conducts a single flight within a single drone inspection sub-airspace. All 3D grids within each drone flight sub-airspace are marked as uninspected. At this point, the preprocessing of the 3D map is complete, and the drones are ready for flight.

[0024] Each inspection drone uses a grid as its flight starting point within its corresponding drone inspection sub-airspace. Starting from the starting point on the left side of the diagram, the inspection drone inspects each grid one by one. Energy consumption is uniformly managed for all equipment carried by the inspection drone, and the duration of the inspection drone's airborne time is monitored in real time based on energy consumption.

[0025] Then, at least one inspection drone release point and inspection drone recovery point (not shown) are set outside each drone inspection sub-airspace for the release and recovery of drones. At least one inspection drone is released from the inspection drone release point. Each inspection drone's flight starting point is set to a different three-dimensional grid within its corresponding drone inspection sub-airspace. After being released, the inspection drone uses a satellite positioning device to fly. After entering the flight boundary, the inspection drone flies only within the drone flight airspace, flying to the satellite positioning coordinates of the inspection point within the three-dimensional grid set as the flight starting point. It then hovers at the inspection point, ready to begin the inspection. At this point, all preparations for the drone inspection are complete, and step two begins.

[0026] Step 2 involves drone photography: The inspection drone uses synthetic aperture radar to scan the surroundings of its 3D grid, generating a 3D image. The drone then uses communication equipment to upload this image to a server. The resulting 3D image covers a larger area than the grid where the drone is located and its 26 adjacent grids (i.e., a 3x3x3 grid). This allows the drone to capture the gaps between the flight boundary and the drone's flight sub-airspace on the 3D map, created by the grid divisions. Once the 3D image is uploaded, proceed to step 3.

[0027] In step 3, the server compares the inspection 3D image with the 3D environment around the 3D grid in the existing 3D map, and determines the surrounding environment based on the comparison results. The specific steps include:

[0028] Case 1: For example, in the first grid that the drone enters, when the inspection stereo image matches the existing three-dimensional map, it means that the water environment and surrounding terrain of the inspection area have not changed, the three-dimensional map does not need to be updated, and there is no target that needs to be inspected. At this time, the drone no longer needs to stay in this grid for inspection. Therefore, the three-dimensional grid where the inspection drone is located is marked as an inspected three-dimensional grid in the server, and the server sends a continue inspection message to the inspection drone, and step four can be executed.

[0029] Case 2: For example, Figure 2In the example, the drone detects a local mismatch between the inspection stereo image and the existing 3D map at the grid where the three black dots are located. New mismatching objects A, B, and C appear. This indicates that the surrounding water environment and topography have changed, and the changed area represents the mismatching local location. Therefore, the server marks the mismatching location in the inspection stereo image and issues a hover command to the inspection drone. The drone maintains hovering at the black dot. After a period of time, it uses synthetic aperture radar to scan the surrounding environment of the grid again for re-inspection and confirmation, generating a re-inspected stereo image. The inspection drone uses its communication equipment to upload the re-inspected stereo image to the server, which compares the inspection stereo image with the re-inspected stereo image to determine whether the inspection stereo image matches the re-inspected stereo image. This determines whether the location where the inspection stereo image mismatches with the existing 3D map is within the drone's flight airspace or whether it is stationary. In case 2, there are multiple scenarios: the mismatching location can be outside or within the drone's flight airspace, or it can be stationary or moving. The server will now conduct further analysis based on the two scenarios identified.

[0030] Case 2a (Mismatched Location Outside Airspace and Stationary, Object A): After the drone hovers and surveys the grid near Object A, it is determined that the local mismatch between the inspection stereo image and the existing 3D map is outside the drone's flight airspace and stationary. For example, if Object A appears in the image, manual intervention is required to determine the specific cause of the mismatch. The server sends an alert to the control console, and the inspection drone's camera is activated. Staff remotely control the camera through the control console to take photos or videos of the mismatched location. The nature of the mismatched location is manually determined. After determining the location's nature (e.g., a legally constructed building), staff decide whether to preserve evidence for further processing. To facilitate nighttime operations, the drone's camera can be a low-light camera. The inspection drone uploads the re-inspected stereo image and captured video footage to the server for evidence preservation. The server then overlays the re-inspected stereo image with the existing 3D map based on the matching portions, embedding the mismatched portion into the existing 3D map and updating the existing 3D map. At this time, the inspection work of the grid is completed, and the server marks the three-dimensional grid where the inspection drone is located as an inspected three-dimensional grid. The server sends a continue inspection message to the inspection drone and executes step four.

[0031] Case 2b (the mismatched location is within the flight airspace and stationary): After the drone flies to the grid near object B and hovers to survey, it is determined that the local mismatch between the inspection stereo image and the existing three-dimensional map is located within the drone's flight airspace and is stationary. At this time, the mismatched area is within the flight area and within the inspection waters, posing an obstacle to the drone inspection and requiring a determination of its nature. At this point, the camera equipment carried by the inspection drone is turned on, and the camera equipment is manually remotely controlled to turn to the mismatched location to capture image data and manually determine the nature of the mismatched location. There are two other cases in Case 2b:

[0032] 1) When the unmatched location is manually determined to be a fixed object (e.g., object B, which could be a newly constructed building, island, or other immovable object), the inspection drone uploads the re-inspected stereo image and captured video footage to the server for evidence. Since the object has now constituted a change in the environment and topography, requiring an update to the 3D map, the server marks the unmatched location in the re-inspected stereo image as an obstacle. A digital surface model is constructed by adding the minimum avoidance distance for the drone to the obstacle. The re-inspected stereo image is then embedded into the existing 3D map based on the matching portion (using the same principle as above). The existing 3D map is updated, with the obstacle's digital surface model boundary set as the internal obstacle boundary. Because the drone's detection range is significantly greater than the grid length, the newly discovered unmatched location will not affect the drone's flight during this process. After the 3D map is updated, the drone can fly outside the internal obstacle boundary according to the map, successfully avoiding the obstacle and continuing its inspection. The 3D grid where the inspection drone is located is then marked as inspected, and the server sends a message to the inspection drone to continue the inspection, proceeding to step 4.

[0033] 2) When the unmatched location is manually determined to be a movable object (such as a temporarily moored vessel, a sunken ship, an illegally operated vessel, etc.), the object can be moved. The water environment and topography at the object's location have not changed, so only the target object needs to be monitored without updating the 3D map. At this time, the inspection drone uploads the re-inspected stereo image and the captured video data to the server to preserve relevant evidence. The 3D image of the unmatched location in the re-inspected stereo image is deleted from the server to avoid adding the 3D graphics of that location to the 3D map when the 3D map is updated. The re-inspected stereo image is then embedded into the existing 3D map based on the matching parts, and the existing 3D map is updated so that only the parts of the map where the environment and topography have actually changed are updated. The 3D grid where the inspection drone is located is marked as the inspected 3D grid, and the server sends a continue inspection message to the inspection drone, executing step 4.

[0034] Case 3c (Mismatched Location Within Flight Airspace and Moving, Object C): If the location where the inspection stereo image and the existing 3D map partially mismatches is determined to be within the drone's flight airspace and in motion, the object at the mismatched location may be a ship or other floating object, requiring manual identification. The server can then alert the drone control console, activate the inspection drone's onboard camera, and manually remotely control the camera to capture footage of the mismatched location. The nature of the mismatched location can then be manually determined. The inspection drone uploads the re-inspection stereo image and the captured footage to the server for evidence. The server deletes the 3D image of the mismatched location from the re-inspection stereo image to prevent it from being added to the 3D map when the 3D map is updated. The re-inspection stereo image is then embedded into the existing 3D map based on the matching portion. The existing 3D map is updated, and the stereo grid where the inspection drone resides is marked as inspected.

[0035] Then, a manual determination is made as to whether the mismatched location requires tracking. If it is determined that the mismatched location does not require tracking (e.g., object C is a standard, legally compliant vessel, or operating on water), the server sends a message to the inspection drone to continue the inspection, and step four is executed. If the mismatched location requires tracking (e.g., object C is an illegal vessel, an illegal fishing vessel, or an overturned vessel), the inspection drone stops hovering, keeps its camera on, and uploads the captured footage to the server in real time for evidence preservation. Staff then manually intervene in the drone's flight through the control console, manually controlling the camera's direction and the inspection drone's flight. When the inspection drone reaches the outer boundary or the inner obstacle boundary, the console issues an alarm, notifying staff that the drone has exceeded the boundary and needs to hover or return.

[0036] Step 4: Upon receiving the "continue inspection" message, the inspection drone stops hovering and flies to the inspection point in the nearest uninspected grid cell to the inspected grid cell. There, it hovers and repeats the above steps starting from Step 2, continuing to inspect each grid cell. If the inspection drone calculates that its airborne time is less than the time threshold required to reach the inspection drone's recovery point, the drone's remaining energy is insufficient and it needs to return to refuel. The inspection drone stops all the above steps and flies to the inspection drone's recovery point (not shown in the figure).

[0037] When all the 3D grids in the drone's flight sub-airspace are marked as inspected, the entire flight sub-airspace has been inspected and can be stopped. The server sends a stop message to the inspection drones, and all inspection drones in the drone flight sub-airspace return to the inspection drone recovery point, ultimately obtaining an updated 3D map of the entire area. At this point, staff can restart a new round of inspections as needed. This new round of inspections does not require further pre-processing of the 3D map, so it can be directly executed from the flight sub-airspace division step in step 1.

[0038] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0039] The present invention is described with reference to flowcharts and / or block diagrams of method products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0040] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0041] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0042] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A method for automatically updating a three-dimensional map of a water area using an inspection drone, characterized in that: The method is performed in the following steps: Step S1: Mark the flight boundary, drone flight airspace, and drone flight sub-airspace of the drone inspection on the existing three-dimensional map, divide the three-dimensional grid into a cube shape, set an inspection point in each three-dimensional grid, mark all three-dimensional grids as uninspected three-dimensional grids, set an inspection drone release point and an inspection drone recovery point at the outer position of each drone inspection sub-airspace, release the inspection drone at the inspection drone release point, and fly the inspection drone through the satellite positioning device. The inspection drone flies to the inspection point in the three-dimensional grid and hovers to start the inspection, and then execute step S2; Step S2: The inspection drone uses a synthetic aperture radar to scan the surrounding environment of the three-dimensional grid to obtain a three-dimensional inspection image. The inspection drone uses a communication device to upload the three-dimensional inspection image to the server, and then executes step S3; Step S3: The server compares the inspection stereo image with the three-dimensional environment surrounding the three-dimensional grid in the existing three-dimensional map. When the inspection stereo image matches the existing three-dimensional map, the three-dimensional grid where the inspection drone is located is marked as an inspected three-dimensional grid. The server sends a continue inspection message to the inspection drone, and step S4 is executed. Step S4: After receiving the inspection continuation information, the inspection drone stops hovering and flies to the inspection point in the uninspected three-dimensional grid closest to the inspected three-dimensional grid, hovers, and executes the steps starting from step S2 in sequence; When the inspection drone calculates that the time it stays in the air is less than the time threshold for the inspection drone to fly to the inspection drone recovery point, the inspection drone stops the above steps and flies to the inspection drone recovery point.

2. The method of automatically updating a three-dimensional map of a water area using an inspection drone according to claim 1, characterized in that: Step S1 specifically includes the following steps performed in sequence: Step S11: Marking the flight boundary of the drone during inspection on the existing three-dimensional map. The flight boundary includes the outer boundary and the inner obstacle boundary. The space inside the boundary is marked as the drone flight airspace; Step S12: Divide the UAV's flight airspace into a cube-shaped three-dimensional grid, with the side length of the three-dimensional grid being less than or equal to 40% of the maximum operating range of the synthetic aperture radar of the inspection UAV. The center point of the three-dimensional grid is set as the inspection point, and the satellite positioning information of each inspection point is measured and marked on the three-dimensional map; Step S13: Based on the priority of daily inspections, the drone flight airspace is divided into drone flight sub-airspaces consisting of multiple adjacent three-dimensional grids. Each inspection drone performs a flight inspection only within one drone inspection sub-airspace. All three-dimensional grids in each drone flight sub-airspace are marked as uninspected three-dimensional grids. Step S14: Each inspection drone sets a three-dimensional grid as its flight starting point within its corresponding drone inspection sub-airspace. All equipment carried by the inspection drone is uniformly energy-consumption-managed, and the time the inspection drone remains in the air is monitored in real time based on the energy consumption. Step S15: At least one inspection drone release point and inspection drone recovery point are set outside each drone inspection sub-airspace, and at least one inspection drone corresponding to the drone inspection sub-airspace is released at the inspection drone release point. The inspection drone flies through a satellite positioning device. After entering the flight boundary, the inspection drone only flies in the drone flight airspace. The inspection drone flies to the satellite positioning coordinates of the inspection point in the three-dimensional grid set as the flight starting point, hovers at the inspection point, and starts inspection.

3. The method of automatically updating a three-dimensional map of a water area using an inspection drone according to claim 1, characterized in that: In step S3, it also includes: Step S31: When there is a partial mismatch between the inspection stereo image and the existing three-dimensional map, the mismatched position is marked in the inspection stereo image, the inspection drone remains in hover, and after a period of time, uses the synthetic aperture radar to scan the surrounding environment of the stereo grid again to obtain a re-inspection stereo image. The inspection drone uses a communication device to upload the re-inspection stereo image to the server, and the inspection stereo image is compared with the re-inspection stereo image on the server to determine whether the inspection stereo image and the re-inspection stereo image match, and to determine whether the position where the inspection stereo image does not match the existing three-dimensional map is located in the drone's flight airspace and whether it is stationary.

4. The method of automatically updating a three-dimensional map of a water area using an inspection drone according to claim 3, characterized in that: In the step S3, after step S31, the following steps are further included: Step S32a: When it is determined that the local mismatch between the inspection stereo image and the existing three-dimensional map is located outside the drone's flight airspace and is in a stationary state, the camera equipment carried by the inspection drone is turned on, and the camera equipment is manually remotely controlled to turn to the mismatched position to shoot image data and manually determine the nature of the mismatched position. The inspection drone uploads the re-inspected stereo image and the shot image data to the server, and the re-inspected stereo image is embedded in the existing three-dimensional map based on the matching part in the server. The existing three-dimensional map is updated, and the three-dimensional grid where the inspection drone is located is marked as an inspected three-dimensional grid. The server sends a continue inspection information to the inspection drone and executes step S4.

5. The method of automatically updating a three-dimensional map of a water area using an inspection drone according to claim 3, characterized in that: In the step S3, after step S31, the following steps are further included: Step S32b: When it is determined that the local mismatch between the inspection stereo image and the existing three-dimensional map is located within the drone's flight airspace and is stationary, the camera carried by the inspection drone is turned on, and the camera is manually remotely controlled to rotate to the mismatching location to capture image data and manually determine the nature of the mismatching location; When it is manually determined that the nature of the mismatched position belongs to a fixed object, the inspection drone uploads the re-inspected stereo image and the captured image data to the server, and the mismatched position of the re-inspected stereo image is marked as an obstacle in the server. A digital surface model is constructed after adding a minimum avoidance distance outside the obstacle. The re-inspected stereo image is then embedded into the existing three-dimensional map based on the matched part, and the existing three-dimensional map is updated, wherein the digital surface model boundary of the obstacle is set as the internal obstacle boundary, and the three-dimensional grid where the inspection drone is located is marked as the inspected three-dimensional grid. The server sends a continue inspection message to the inspection drone, and executes step S4; When it is manually determined that the nature of the unmatched position is a movable object, the inspection drone uploads the re-inspection stereo image and the captured image data to the server, deletes the three-dimensional image of the unmatched position in the re-inspection stereo image on the server, and then embeds the re-inspection stereo image into the existing three-dimensional map based on the matching part, updates the existing three-dimensional map, marks the three-dimensional grid where the inspection drone is located as an inspected three-dimensional grid, and sends a continue inspection information to the inspection drone to execute step S4.

6. The method of automatically updating a three-dimensional map of a water area using an inspection drone according to claim 3, characterized in that: In the step S3, after step S31, the following steps are further included: Step S32c: When it is determined that the location where the inspection stereo image and the existing three-dimensional map partially mismatch exists is located within the drone's flight airspace and is in a moving state, the camera device carried by the inspection drone is turned on, and the camera device is manually remotely controlled to capture image data at the mismatched location and the nature of the mismatched location is manually determined. The inspection drone uploads the re-inspection stereo image and the captured image data to the server, deletes the three-dimensional image at the mismatched location in the re-inspection stereo image from the server, and then embeds the re-inspection stereo image into the existing three-dimensional map based on the matched portion, updates the existing three-dimensional map, and marks the three-dimensional grid where the inspection drone is located as an inspected three-dimensional grid; Manually determine whether the unmatched position needs to be tracked. If it is determined that the unmatched position does not need to be tracked, the server sends a continue inspection message to the inspection drone and executes step S4; When it is determined that the mismatched position needs to be tracked, the inspection drone stops hovering, keeps the camera device turned on, and uploads the captured image data to the server in real time. The console manually controls the shooting direction of the camera device and the flight of the inspection drone. When the inspection drone flies to the outer boundary and the internal obstacle boundary, the console issues an alarm.

7. A method for automatically updating a three-dimensional map of a water area using an inspection drone according to any one of claims 4 to 6, characterized in that: The camera device is a low-light camera.

8. The method of automatically updating a three-dimensional map of a water area using an inspection drone according to claim 2, characterized in that: In step S1: at least one inspection drone is released at a time in each drone inspection sub-airspace, and the flight starting point of each inspection drone is set to a different three-dimensional grid in its corresponding drone inspection sub-airspace.

9. The method of automatically updating a three-dimensional map of a water area using an inspection drone according to claim 2, characterized in that: In step S4, it also includes: When all the three-dimensional grids in the drone flight sub-airspace are marked as inspected three-dimensional grids, the server sends a stop inspection message to the inspection drone, and all the inspection drones in the drone flight sub-airspace fly back to the inspection drone recovery point to obtain an updated three-dimensional map, and execute in sequence starting from step S13.

Images