Inspection target positioning method, flying device, and electronic device

Abstract

The application discloses a positioning method of an inspection target, a flight device, an electronic device, and a computer readable storage medium. The positioning method of the inspection target comprises the following steps: acquiring a remote sensing map; acquiring an image frame in which the inspection target exists; matching the image frame with the remote sensing map; and acquiring the position of the inspection target in the remote sensing map. Through the above method, the positioning accuracy of the inspection target can be improved.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) inspection technology, and in particular to methods for locating inspection targets, flight equipment, electronic equipment, and computer-readable storage media. Background Technology

[0002] With the increasing prevalence of drone technology, drones are being used more and more in ground target inspection scenarios, such as hot spot detection of photovoltaic panels in photovoltaic power plants, detection of illegal construction in urban management, and safety inspection of embankments. By combining deep learning-based target detection methods, when a target of interest, i.e., the inspection target, is detected, its location can be calculated based on the drone's pose during image capture. Inspection personnel can then directly proceed to that location to carry out appropriate actions or verification.

[0003] In the above scheme, accurate positioning of the inspection target is a key technical point. Ordinary civilian drones have relatively low positioning accuracy; the error of a typical civilian Global Positioning System (GPS) signal has a 50% probability of being within 2.5 meters and a 50% probability of being within 10 meters. Based on this, the calculated positioning accuracy of the inspection target is low. Adding other errors in the calculation process, the actual error range of the inspection target's location will be further amplified. Inspection personnel can only obtain a general location of the inspection target, and after arriving at the site, further investigation and searching are required, greatly increasing the workload of the inspection personnel. Summary of the Invention

[0004] This application provides a method for locating an inspection target, a flight device and electronic device, and a computer-readable storage medium to improve the positioning accuracy of the inspection target.

[0005] To solve the above-mentioned technical problems, the technical solution adopted in this application is: to propose a method for locating inspection targets, which includes: acquiring a remote sensing map; acquiring image frames containing inspection targets; matching the image frames with the remote sensing map; and obtaining the location of the inspection target in the remote sensing map based on the matching result.

[0006] The process of matching the image frame with the remote sensing map includes: obtaining the geographical area covered by the image frame based on the image information of the image frame; extracting the map area corresponding to the image frame from the remote sensing map based on the geographical area; and performing image matching between the image frame and the map area.

[0007] The step of extracting the map region corresponding to the image frame from the remote sensing map based on the geographic range includes: obtaining the redundant geographic range; wherein the redundant geographic range covers the geographic range and is larger than the geographic range; and extracting the map region corresponding to the image frame from the remote sensing map based on the redundant geographic range.

[0008] The step of obtaining the location of the inspection target in the remote sensing map based on the matching result includes: if the matching is successful, projecting the image frame onto the map area to obtain the location of the inspection target in the remote sensing map.

[0009] The process of determining the location of the inspection target in the remote sensing map based on the matching result also includes: if the matching fails, calculating the location of the inspection target in the remote sensing map based on the GPS information, camera attitude, viewing angle, and target pixel coordinates corresponding to the image frame.

[0010] The process of obtaining the location of the inspection target in the remote sensing map based on the matching result also includes: if the matching fails, obtaining the redundancy of the redundant geographical range relative to the geographical range; if the redundancy is greater than the redundancy threshold, calculating the location of the inspection target in the remote sensing map based on the GPS information, camera attitude, viewing angle, and target pixel coordinates corresponding to the image frame.

[0011] The process of projecting an image frame onto a map area includes: obtaining the homography matrix of the image frame; and transforming the pixel coordinates of the image frame into the reference coordinate system of the remote sensing map based on the homography matrix.

[0012] To solve the above-mentioned technical problems, the technical solution adopted in this application is as follows: a flight device is proposed, which includes: an image module for acquiring image frames of an inspection area; a cruise module for carrying the image module for cruise so that the image module can acquire image frames of the inspection area; and a positioning module for acquiring a remote sensing map and acquiring image frames containing inspection targets from the image frames of the inspection area, matching the image frames of the inspection area with the remote sensing map, and obtaining the position of the inspection target in the remote sensing map based on the matching result.

[0013] To solve the above-mentioned technical problems, the technical solution adopted in this application is: to propose an electronic device, which includes a processor and a memory, wherein the memory stores program data, and the processor executes the program data to implement the above-mentioned positioning method.

[0014] To solve the above-mentioned technical problems, the technical solution adopted in this application is: to propose a computer-readable storage medium that stores program instructions, and the program data can be executed by a processor to implement the above-mentioned positioning method.

[0015] The beneficial effects of the embodiments of this application are as follows: The method for locating inspection targets in this application matches image frames containing inspection targets with high-precision remote sensing maps, and obtains the position of inspection targets in the remote sensing maps based on the matching results. It can locate inspection targets in remote sensing maps based on image information of image frames containing inspection targets, and then obtain the position of inspection targets from the remote sensing maps. Because the positioning accuracy of remote sensing maps is high, the precise position of inspection targets can be obtained in this way, thereby improving the positioning accuracy of inspection targets. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the first embodiment of the method for locating the inspection target in this application;

[0017] Figure 2 This is a schematic diagram illustrating the matching of image frames and remote sensing maps in this application;

[0018] Figure 3 yes Figure 1 A detailed flowchart of step S300 in the embodiment;

[0019] Figure 4 yes Figure 3 A detailed flowchart of step S320 in the embodiment;

[0020] Figure 5 yes Figure 1 A detailed flowchart of step S400 in the embodiment;

[0021] Figure 6 yes Figure 5 A detailed flowchart of step S420 in the embodiment;

[0022] Figure 7 This is a flowchart illustrating the second embodiment of the method for locating the inspection target in this application;

[0023] Figure 8 This is a schematic diagram of the structure of an embodiment of the flight equipment of this application.

[0024] Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application;

[0025] Figure 10 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise expressly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0028] This application provides a method for locating inspection targets. This method uses a flight device to search for the inspection target and obtain its precise location. The flight device can be a drone or other flight equipment with inspection search and data processing capabilities. This positioning method is applicable to hot spot detection of photovoltaic panels in photovoltaic power plants, detection of illegal construction in urban management, and safety inspection of embankments. In this positioning method, the flight device includes at least a positioning end and a camera end. The positioning end can be an electronic device with GPS positioning capabilities, computing power, and storage capacity, while the camera end is a camera device with video recording capabilities.

[0029] Specifically, such as Figure 1 and Figure 2 As shown, Figure 1 This is a flowchart illustrating the first embodiment of the method for locating the inspection target in this application; Figure 2 This is a schematic diagram illustrating the matching of image frames and remote sensing maps in this application. The localization method includes the following steps:

[0030] Step S100: Obtain the remote sensing map.

[0031] The main entity performing this step is the positioning terminal. The positioning terminal acquires and stores a remote sensing map 100 in advance via wireless communication to prepare for subsequent steps. The remote sensing map 100 is a commercial product provided by a satellite company or aerial photography service provider. After geometric correction, the positioning accuracy of the remote sensing map 100 can reach the sub-pixel level, and each pixel on the remote sensing map 100 has corresponding latitude and longitude coordinates.

[0032] Step S200: Acquire image frames containing inspection targets.

[0033] Specifically, the flight equipment begins its inspection according to the predetermined inspection route. The camera acquires ground orthophoto frames of the inspection area at a certain frequency and transmits these image frames to the positioning end. The positioning end then performs preprocessing such as geometric correction on the image frames to make the content within the image frames clearer.

[0034] The positioning unit pre-stores image information of the inspection target. It performs image matching between the target image information and image frames of the inspection area transmitted from the camera to determine if a target exists in the image frame. If a target is found, the image frame is saved and compared with a pre-acquired remote sensing map 100 to determine the target's location on the map. If no target is found in the image frame, the positioning unit deletes it and continues checking the next image frame transmitted from the camera until a target is detected. Image frames that do not contain a target are promptly deleted to prevent image frame accumulation and reduce data storage requirements.

[0035] For example, in the application scenario of detecting illegal construction in buildings, staff use aerial equipment to obtain the specific location of illegal buildings. This aerial equipment is equipped with a positioning module (i.e., the positioning end) and a camera module (the camera end). Staff control the camera module on the aerial equipment to continuously capture images of the city's ground scene (i.e., image frames of the inspection area) at a certain frequency over the city. The positioning module on the aerial equipment determines whether there are illegal buildings, i.e., inspection targets, from the captured ground scene images. If an illegal building is found in the ground scene image, the image is saved and matched to a remote sensing map 100, thereby obtaining the location of the illegal building on the remote sensing map 100.

[0036] Step 300: Match the image frame with the remote sensing map.

[0037] After the positioning terminal acquires an image frame containing the inspection target in step S200, it matches this image frame with the remote sensing map 100 pre-acquired in step S100, so that the image frame containing the inspection target matches a specific area in the remote sensing map 100, thereby obtaining the specific location of the inspection target in the remote sensing map 100. Specifically, after the image frame is matched with the remote sensing map 100, the geographical information in the image frame can completely overlap with the geographical information in the remote sensing map 100, thus obtaining the location information of the inspection target on the remote sensing map 100.

[0038] Optional, such as Figure 3 As shown, Figure 3 yes Figure 1A schematic diagram of a specific process for step S300 in the embodiment. Matching image frames with remote sensing maps can be achieved using methods such as... Figure 3 The method shown is implemented by steps S310-S330:

[0039] Step S310: Obtain the geographical range covered by the image frame based on the image information of the image frame.

[0040] Because the remote sensing map 100 contains a vast amount of map information, directly matching the image frame with the remote sensing map would result in an enormous computational workload for the positioning end and be extremely inefficient. Therefore, in this step, the geographical area 300 covered by the image in the image frame is obtained in advance using the image information, and then the location of this geographical area 300 is found on the remote sensing map 100. This method can effectively reduce the computational workload of the positioning end, thereby significantly improving the positioning efficiency of the inspection target.

[0041] In addition to the specific image content, the image information of an image frame also includes corresponding GPS information, camera posture, and viewing angle parameters. Specifically, it includes the GPS information of the camera's geographical location when capturing and acquiring the image frame, the camera's posture information, and the camera's viewing angle information. The positioning unit uses the aforementioned GPS information, camera posture, and viewing angle parameters to calculate the geographical coverage area (300) of the image frame.

[0042] For example, in the application scenario of detecting illegal construction in buildings, staff use aerial equipment to obtain the specific location of illegal buildings. This aerial equipment is equipped with a positioning module (i.e., the positioning terminal) and a camera module (the camera terminal). Staff control the camera module on the aerial equipment to continuously capture images of the city's ground scene (i.e., image frames of the inspection area) at a certain frequency over the city. While capturing these images, the positioning module (i.e., the positioning terminal) records the location information (GPS information) at the time of image capture, and simultaneously records the specific pose and viewing angle of the camera on the aerial equipment. After determining that an illegal building exists in the image, the positioning module processes the GPS information, camera pose, and viewing angle of the ground scene image containing the illegal building using a corresponding algorithm, thereby obtaining the geographical area covered by the ground scene image.

[0043] Step S320: Extract the map region corresponding to the image frame from the remote sensing map based on the geographic range.

[0044] After obtaining the geographical area 300 covered by the image frame, the positioning terminal finds the corresponding map region 200 in the remote sensing map 100 based on the geographical area 300 corresponding to the image frame, and selects the map region 200 as the reference image frame. The map region 200 is used to match the image frame to the remote sensing map 100, ensuring that each pixel in the image frame coincides with the corresponding pixel in the map region 200, so that the coordinates of the pixels in the map region 200 are used as the specific coordinates of the corresponding pixels in the image frame.

[0045] Optional, such as Figure 4 As shown, Figure 4 yes Figure 3 A schematic diagram of a specific process for step S320 in the embodiment. Extracting the map region 200 corresponding to the image frame from the remote sensing map 100 based on the geographical range 300 can be achieved using methods such as... Figure 4 The method shown is implemented by steps S321-S322:

[0046] Step S321: Obtain the redundant geographic range of the geographic range. The redundant geographic range covers the geographic range and is larger than the geographic range.

[0047] Step S322: Extract the map region corresponding to the image frame from the remote sensing map based on the redundant geographic range.

[0048] The steps S321-S322 will be explained in a unified manner.

[0049] The positioning system calculates the geographical area 300 covered in the image frame using the aforementioned GPS information, camera attitude, and viewing angle parameters. However, the calculated geographical area 300 may differ from the actual geographical area due to inherent errors in the GPS information itself. Therefore, this error can be overcome when selecting map region 200, improving the matching success rate between the image frame and map region 200.

[0050] Specifically, such as Figure 2 As shown, after the positioning terminal obtains the geographic range 300 using the above method, it adds a certain redundancy e to the geographic range 300. That is, the area of ​​the geographic range 300 is uniformly expanded with a certain redundancy e around its perimeter to obtain a redundant geographic range. When the positioning terminal determines the map region 200 on the remote sensing map 100, it uses the redundant geographic range to determine the map region 200, so as to ensure that the extracted map region 200 includes all areas covered by the geographic range 300, while also having a certain redundancy e.

[0051] The redundancy e should not be too large, so as not to increase the amount of calculation required for the positioning end to match the image frame with the map region 200, thereby affecting the matching efficiency.

[0052] Step 330: Perform image matching between the image frame and the map region.

[0053] A specific algorithm is used to match the image in the image frame with the image in the map region 200 to determine whether the image in the image frame matches the image in the map region 200, thus preparing for subsequent determination of the inspection target location. The image matching between the positioning end and the image frame in the map region 200 typically includes feature point extraction algorithms and matching point search algorithms. Commonly used feature point extraction algorithms include SIFT, SURF, and Super Point, while commonly used matching point search algorithms include brute-force search, KNN, and Super Glue. This proposal does not specify which image matching method should be used.

[0054] For example, in the application scenario of detecting illegal construction in buildings, staff use aerial equipment to obtain the specific location of the illegal building. This aerial equipment is equipped with a positioning module (i.e., the positioning end) and a shooting module (the shooting end). The positioning module on the aerial equipment determines the specific map region 200 of the illegal building in the remote sensing map 100. Then, it extracts the pixel information of each building, street, tree, and river in the image using feature point extraction algorithms (such as SIFT, SURF, and SuperPoint). Finally, it uses matching point search algorithms (such as brute-force search, KNN, and Super Glue) to search for matching pixels (buildings, streets, trees, and rivers) in the map region 200 to determine if the obtained map region 200 matches the region in the image.

[0055] Step 400: Obtain the location of the inspection target in the remote sensing map based on the matching results.

[0056] The positioning terminal uses the above method to match the image frame with the map area 200 and obtain the matching result. Based on the matching result, it obtains the location of the inspection target in the remote sensing map.

[0057] Unlike existing technologies, the target location method of this application matches an image frame containing the target with a high-precision remote sensing map 100, and obtains the target's position in the remote sensing map 100 based on the matching result. This method can locate the target in the remote sensing map 100 based on the image information of the image frame containing the target, and then obtain the target's position from the remote sensing map 100. Because the remote sensing map 100 has high positioning accuracy, this method can obtain the precise position of the target, thereby improving the positioning accuracy of the target.

[0058] Optional, such as Figure 5 As shown, Figure 5 yes Figure 1 A detailed flowchart of step S400 in the embodiment is shown. Obtaining the location of the inspection target in the remote sensing map based on the matching result can be achieved using… Figure 5 The method implementation in [the document] includes the following steps:

[0059] Step S410: Determine whether the image frame and the map area are successfully matched.

[0060] Specifically, after acquiring the map region 200, the positioning terminal matches the image frame with the map region 200 to determine whether the image in the image frame overlaps with the image in the map region 200.

[0061] If the match is successful, the positioning end will execute step S420; if the match fails, it will execute step S430.

[0062] Step S420: Project the image frame onto the map area to obtain the location of the inspection target in the remote sensing map.

[0063] The image frame and map region 200 are successfully matched. This means the positioning system calculates using an algorithm that the geographical area 300 in the image frame belongs to the region in map region 200. The image frame is then projected one-to-one onto map region 200, and each pixel in the image frame (including the inspection target) is mapped one-to-one with its corresponding pixel in map region 200. This allows the location information of the inspection target to be obtained in map region 200. Specifically, the latitude and longitude coordinates of the pixels on the remote sensing map 100 corresponding to the pixels of the inspection target in the image frame are the corrected actual location of the inspection target.

[0064] For example, staff can obtain the specific location of illegal buildings using aerial equipment equipped with a positioning module (i.e., a positioning terminal) and a shooting module (shooting terminal). The positioning module on the aerial equipment uses algorithms to project the acquired image frame containing the illegal building onto a map region 200 of a remote sensing map 100 that matches the image frame. Each pixel in the image frame, such as a building, street, tree, or river, overlaps with a corresponding pixel in the map region 200, thus allowing the precise location information of the illegal building to be determined on the remote sensing map 100.

[0065] Optional, such as Figure 6 As shown, Figure 6 yes Figure 5 A schematic diagram of a specific process for step S420 in the embodiment. Projecting an image frame onto a map area can be achieved using methods such as... Figure 6The method shown is implemented, and the method specifically includes steps S421-S422:

[0066] Step S421: Obtain the homography matrix of the image frame;

[0067] Step S422: Transform the pixel coordinates of the image frame to the reference coordinate system of the remote sensing map based on the homography matrix.

[0068] The positioning end calculates the homography matrix based on the coordinates of the pixels of the matching point, and transforms the pixel coordinates of the image frame to the reference coordinate system of the remote sensing map based on the homography matrix, thereby projecting the image frame onto the map area 200, so that the pixels in the image frame are aligned one by one with the corresponding pixels in the remote sensing map. The latitude and longitude coordinates of the pixels on the remote sensing map 100 corresponding to the pixels of the inspection target on the image frame are the actual positions of the inspection target after correction.

[0069] Step S430: Calculate the position of the inspection target in the remote sensing map based on the GPS information, camera attitude, viewing angle, and target pixel coordinates corresponding to the image frame.

[0070] For detailed implementation methods, please refer to the above embodiments.

[0071] In summary, unlike existing technologies, the positioning method of this application matches an image frame containing the inspection target with a high-precision remote sensing map 100 to determine that the geographical area 300 covered in the image frame is located in a specific map region 200 of the remote sensing map 100. Then, the image frame is projected onto the corresponding map region 200 of the remote sensing map 100 at a one-to-one ratio, so that the pixels on the image frame, including the inspection target, correspond one-to-one with the pixels on the remote sensing map 100, thereby enabling the accurate acquisition of the location of the inspection target directly from the remote sensing map 100.

[0072] Optional, such as Figure 7 As shown, Figure 7 This is a schematic diagram of the second embodiment of the method for locating the inspection target in this application. Specifically, it includes the following steps:

[0073] Step S100: Obtain the remote sensing map.

[0074] The specific implementation method can adopt the above embodiments.

[0075] Step S200: Acquire image frames of the inspection target.

[0076] The specific implementation method can adopt the above embodiments.

[0077] Step S310: Obtain the geographical range covered by the image frame based on the image information of the image frame.

[0078] The specific implementation method can adopt the above embodiments.

[0079] Step S321: Obtain the redundant geographic range of the geographic range. The redundant geographic range covers the geographic range and is larger than the geographic range.

[0080] The specific implementation method can adopt the above embodiments.

[0081] Step S322: Extract the map region corresponding to the image frame from the remote sensing map based on the redundant geographic range.

[0082] The specific implementation method can adopt the above embodiments.

[0083] Step S330: Perform image matching between the image frame and the map region.

[0084] The specific implementation method can adopt the above embodiments.

[0085] Step S410: Determine whether the image frame and the map area are successfully matched.

[0086] If the match is successful, the positioning end executes step S420; if the match fails, it executes steps S440 and S450, and returns to step S321 to start the loop.

[0087] Step S420: Project the image frame onto the map area to obtain the location of the inspection target in the remote sensing map.

[0088] The specific implementation method can adopt the above embodiments.

[0089] Step S440: Obtain the redundancy of the redundant geographic range relative to the geographic range, and determine whether the redundancy is greater than the redundancy threshold.

[0090] Here, the redundancy threshold refers to the redundancy threshold of redundancy degree e, which is the size of the area that is larger than the geographical range 300. The redundancy threshold of redundancy degree e should not be too large. If it is too large, it will cause the positioning end to select an excessively large range when selecting map region 200, thereby increasing the workload of the positioning end when matching image frames with map region 200, thus affecting the efficiency of acquiring inspection targets.

[0091] In step S321, when obtaining the redundant geographic range 300, the redundancy e is generally taken from zero. If the image frame fails to match the map region 200 obtained based on the redundant geographic range 300, the positioning terminal executes step S440 to determine whether the redundancy is greater than the redundancy threshold. Further, if the redundancy e is greater than the redundancy threshold, the positioning terminal executes step S430; if the redundancy e is less than the redundancy threshold, it executes step S450 and returns to step S321 to start the loop; otherwise, it executes step S430.

[0092] In this process, after executing step S450, the positioning end returns to step S321 to begin a loop until a match is successful or a match fails and the redundancy e is determined to be greater than a threshold. If a match is successful, the location of the inspection target in the remote sensing map 100 is obtained using the method in step S420; or, until the redundancy e is greater than the redundancy threshold, the location of the inspection target in the remote sensing map 100 is obtained using the method in step S430. The redundancy threshold value of redundancy e is derived from long-term actual testing and will not be specifically described here. The implementation methods of steps S420 and S430 are consistent with those in the first embodiment and will not be repeated here.

[0093] Step S450: Increase the redundancy by a certain step size.

[0094] Increasing the redundancy e in steps means adding a certain amount of redundancy to the original redundancy e to obtain a new redundancy e. For example, in a certain instance, the redundancy threshold of redundancy e is set to 10. When the image frame fails to match the map region 200, if the value of redundancy e is 2, then step S450 is executed. After step S450, the value of redundancy e becomes 4. In other words, the value of redundancy e increases by 2 each time step S450 is executed, until the image frame successfully matches the map region 200 or until the value of redundancy e exceeds its redundancy threshold.

[0095] There are many reasons for matching failure. In general, it is because the geographic range 300 obtained by the method in step S321 is smaller than the actual geographic range 300, and the map area 200 after increasing the redundancy e cannot completely cover the actual geographic range 300. If the redundancy e is less than the threshold, the redundancy e is increased by a certain step size, and a redundant geographic range is obtained with the new redundancy e. Based on the new redundant geographic range, a new map area 200 is extracted from the remote sensing map 100 and matched with the image frame to obtain the location of the inspection target on the remote sensing map 100.

[0096] In summary, in this embodiment, step S440 pre-determines whether the redundancy e exceeds the threshold to prevent the positioning end from affecting the efficiency of acquiring the inspection target due to excessive computation. Furthermore, after a failure to match the image frame with the map region 200, the positioning end does not immediately execute the method of step S430 to acquire the location of the inspection target. Instead, it continues to increase the redundancy e in steps and restarts from step S321 to reacquire the redundant geographical range. Based on the new redundant geographical range, a new map region 200 is determined and matched with the image frame. These steps ensure that the positioning end preferentially uses the method of step S420 to acquire the location of the inspection target, thereby guaranteeing the accuracy of the final location obtained by the positioning end and improving the efficiency of staff in finding the inspection target. Simultaneously, when the redundancy e increases to exceed the redundancy threshold, step S430 is used as a backup method to acquire the location of the inspection target, preventing excessive computational load from affecting the efficiency of acquiring the location of the inspection target.

[0097] This application further proposes a flight device; please refer to [link / reference needed]. Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the flight device of this application. The flight device 40 includes an image module 42, a cruise module 41, and a positioning module 43, wherein the image module 42 and the positioning module 43 are mounted on the cruise module 41, and the image module 42 and the positioning module 43 are signal-connected. The cruise module 41 carries the image module 42 and the positioning module 43 for cruise, so that the image module 42 acquires image frames of the inspection area. The positioning module 43 is used to acquire the remote sensing map 100 and to acquire image frames containing the inspection target from the image frames of the inspection area, and to match the image frames of the inspection area with the remote sensing map 100, and to obtain the position of the inspection target in the remote sensing map 100 based on the matching result. The flight device 40 is also used to implement the above-mentioned positioning method for the inspection target.

[0098] This application further proposes an electronic device, please refer to... Figure 9 , Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 20 includes a processor 21 and a memory 22 connected to the processor 21. The memory 22 stores program data. The processor 21 executes the program data stored in the memory 22 to perform the steps in the above-described method embodiments.

[0099] Processor 21 can also be referred to as CPU (Central Processing Unit). Processor 21 may be an integrated circuit chip with signal processing capabilities. Processor 21 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor can be a microprocessor or any conventional processor.

[0100] The memory 22 is used to store the program data required for the processor 21 to run.

[0101] The processor 21 is also used to execute the program data stored in the memory 22 to implement the above-mentioned method for locating the inspection target.

[0102] Optionally, this application further proposes a computer-readable storage medium. See also... Figure 10 , Figure 10 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application.

[0103] The computer-readable storage medium 30 of this application embodiment stores program instructions 31, which are executed to implement the above-mentioned method for locating the inspection target.

[0104] Specifically, program instructions 31 can be formed into a program file and stored in the aforementioned storage medium in the form of a software product, so that an electronic device (which may be a personal computer, server, or network device, etc.) or processor can execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0105] In this embodiment, the computer-readable storage medium 30 may be, but is not limited to, a USB flash drive, SD card, PD optical drive, portable hard drive, large-capacity floppy drive, flash memory, multimedia memory card, server, etc.

[0106] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the electronic device to perform the steps in the above-described method embodiments.

[0107] Furthermore, if the aforementioned functions are implemented as software functions and sold or used as independent products, they can be stored in a mobile terminal-readable storage medium. That is, this application also provides a storage device storing program data, which can be executed to implement the methods of the above embodiments. This storage device can be, for example, a USB flash drive, an optical disc, or a server. In other words, this application can be embodied in the form of a software product, which includes several instructions to cause a smart terminal to execute all or part of the steps of the methods described in the various embodiments.

[0108] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for locating inspection targets, characterized in that, include: Obtain remote sensing maps; Acquire image frames containing the inspection target; Match the image frame with the remote sensing map; The location of the inspection target in the remote sensing map is obtained based on the matching results; The geographical range covered by the image frame is obtained based on the image information of the image frame; Obtain a redundant geographic range of the geographic range, wherein the redundant geographic range covers the geographic range and is larger than the geographic range; Based on the redundant geographic range, a map region corresponding to the image frame is extracted from the remote sensing map; The step of obtaining the location of the inspection target in the remote sensing map based on the matching result includes: If the matching fails, the redundancy of the redundant geographical range relative to the geographical range is obtained; if the redundancy is greater than the redundancy threshold, the position of the inspection target in the remote sensing map is calculated based on the GPS information, camera attitude, viewing angle, and target pixel coordinates corresponding to the image frame.

2. The positioning method according to claim 1, characterized in that, The step of matching the image frame with the remote sensing map includes: Based on the geographical range, extract the map region corresponding to the image frame from the remote sensing map; The image frame is matched with the map region.

3. The positioning method according to claim 1, characterized in that, The step of obtaining the location of the inspection target in the remote sensing map based on the matching result includes: If a match is successful, the image frame is projected onto the map area, and the location of the inspection target in the remote sensing map is obtained.

4. The positioning method according to claim 1, characterized in that, The step of obtaining the location of the inspection target in the remote sensing map based on the matching result also includes: If the matching fails, the position of the inspection target in the remote sensing map is calculated based on the GPS information, camera attitude, viewing angle, and target pixel coordinates corresponding to the image frame.

5. The positioning method according to claim 3, characterized in that, The step of projecting the image frame onto the map area includes: Obtain the homography matrix of the image frame; The pixel coordinates of the image frame are transformed into the reference coordinate system of the remote sensing map based on the homography matrix.

6. A flight device, characterized in that, include: The image module is used to acquire image frames of the inspection area; A cruise module is used to carry the image module for cruise, so that the image module can acquire image frames of the inspection area. The positioning module is used to acquire a remote sensing map and an image frame containing the inspection target from the image frames of the inspection area, and to match the image frames of the inspection area with the remote sensing map, and to obtain the position of the inspection target in the remote sensing map based on the matching result; The positioning module is configured to: obtain the geographical area covered by the image frame based on the image information of the image frame; obtain a redundant geographical area, wherein the redundant geographical area covers the geographical area and is larger than the geographical area; extract a map region corresponding to the image frame from the remote sensing map based on the redundant geographical area; if the matching fails, obtain the redundancy of the redundant geographical area relative to the geographical area; if the redundancy is greater than a redundancy threshold, calculate the position of the inspection target in the remote sensing map based on the GPS information, camera attitude, viewing angle, and target pixel coordinates corresponding to the image frame.

7. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores program data, and the processor is used to execute the program data to implement the positioning method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The device stores program instructions that, when executed by a processor, implement the positioning method as described in any one of claims 1-5.

Images