Obstacle measurement method and device based on unmanned aerial vehicle, electronic equipment and medium

Abstract

The embodiment of the invention discloses an obstacle measurement method and device based on an unmanned aerial vehicle, electronic equipment and a storage medium. The method comprises the steps of obtaining image information and point cloud information collected by the unmanned aerial vehicle; constructing an environment three-dimensional model corresponding to the current environment according to the image information and the point cloud information; recognizing obstacles and power transmission lines in the environment three-dimensional model based on a preset recognition network; and measuring the distance between the obstacle and the power transmission line based on the point cloud information. According to the scheme, the obstacle measurement precision can be improved under the complex background and variable conditions.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and specifically to an obstacle measurement method, device, electronic device and storage medium based on a drone. Background Art

[0002] With the advancement of science and technology, UAV LiDAR technology can obtain high-precision three-dimensional point cloud data, which can be used for three-dimensional reconstruction of transmission lines and selection and design of new line directions, and provide detailed information on measurement objects such as line corridors, crossings, conductor sag, line distances, tower bodies, surface attachments and their spatial relationships.

[0003] In power grid applications, a large number of objects need to be classified and identified, such as poles, transmission lines, etc. Although deep learning technology has made significant progress in target recognition, the measurement accuracy of obstacles by UAV LiDAR technology is poor under complex backgrounds and changing conditions. Summary of the invention

[0004] The embodiments of the present application provide a drone-based obstacle measurement system, method, electronic device, and storage medium, which can improve the measurement accuracy of obstacles under complex backgrounds and changing conditions.

[0005] In a first aspect, an embodiment of the present application provides an obstacle measurement method based on a drone, comprising:

[0006] Obtain image information and point cloud information collected by drones;

[0007] Constructing a three-dimensional model of the environment corresponding to the current environment according to the image information and the point cloud information;

[0008] Identify obstacles and power transmission lines in the three-dimensional model of the environment based on a preset recognition network;

[0009] The distance between the obstacle and the transmission line is measured based on the point cloud information.

[0010] Optionally, in some embodiments of the present application, constructing a three-dimensional model of the environment corresponding to the current environment according to the image information and the point cloud information includes:

[0011] Extracting image features corresponding to the current environment from the image information;

[0012] Extracting point cloud features corresponding to each object in the current environment from the point cloud information;

[0013] Based on the image features and point cloud features, a three-dimensional model of the environment corresponding to the current environment is constructed.

[0014] Optionally, in some embodiments of the present application, constructing a three-dimensional model of the environment corresponding to the current environment based on the image features and the point cloud features includes:

[0015] Based on the point cloud features, construct an initial three-dimensional model corresponding to the current environment;

[0016] An environmental three-dimensional model corresponding to the current environment is constructed according to the image features, a preset multi-view stereo matching algorithm and the initial three-dimensional model.

[0017] Optionally, in some embodiments of the present application, constructing an environmental three-dimensional model corresponding to the current environment according to the image features, a preset multi-view stereo matching algorithm and the initial three-dimensional model includes:

[0018] Mapping the image features to the initial three-dimensional model to obtain a mapped three-dimensional model;

[0019] Calculating the disparity corresponding to the image information according to the image features and a preset multi-view stereo matching algorithm;

[0020] The mapped three-dimensional model is adjusted according to the parallax to obtain an environmental three-dimensional model corresponding to the current environment.

[0021] Optionally, in some embodiments of the present application, the identifying obstacles and power lines in the three-dimensional model of the environment based on a preset recognition network includes:

[0022] Get the preset recognition network;

[0023] Determine the target obstacle corresponding to the recognition strategy;

[0024] The height and shape of each object in the three-dimensional model of the environment based on a preset recognition network;

[0025] Determine target obstacles and power lines based on the height and shape of each object.

[0026] Optionally, in some embodiments of the present application, measuring the distance between the obstacle and the transmission line based on the point cloud information includes:

[0027] Determine, according to the point cloud information, an obstacle point cloud corresponding to the obstacle and a transmission line point cloud corresponding to the transmission line;

[0028] The distance between the obstacle and the transmission line is measured according to the cluster center corresponding to the obstacle point cloud and the cluster center corresponding to the transmission line point cloud.

[0029] Optionally, in some embodiments of the present application, after measuring the distance between the obstacle and the transmission line based on the point cloud information, the method further comprises:

[0030] Obtain safe distance standards for transmission lines;

[0031] The measurement results are analyzed to determine whether there is a safety hazard at the distance between the obstacle and the transmission line.

[0032] In a second aspect, an embodiment of the present application provides an obstacle measurement device based on a drone, comprising:

[0033] The acquisition module is used to obtain the image information and point cloud information collected by the drone;

[0034] A construction module, used to construct a three-dimensional model of the environment corresponding to the current environment according to the image information and the point cloud information;

[0035] An identification module, used to identify obstacles and power transmission lines in the three-dimensional model of the environment based on a preset identification network;

[0036] A measuring module is used to measure the distance between the obstacle and the transmission line based on the point cloud information.

[0037] Correspondingly, the present application also provides an electronic device, comprising a memory, a processor, and a processing program stored in the memory and executable on the processor, wherein the processor executes the program according to any of the steps of the method described above.

[0038] The present application also provides a storage medium, wherein the storage medium stores a processor program, and the processor program implements any of the above methods when the processor program is executed by a processor.

[0039] The present application provides an obstacle measurement method, device, electronic device and storage medium based on drones. After acquiring image information and point cloud information collected by drones, a three-dimensional model of the environment corresponding to the current environment is constructed based on the image information and point cloud information. Then, obstacles and power lines in the three-dimensional model of the environment are identified based on a preset recognition network. Finally, the distance between the obstacle and the power line is measured based on the point cloud information. In the obstacle measurement solution based on drones provided in the present application, obstacles and power lines in the three-dimensional model of the environment are identified based on a preset recognition network, which can improve the measurement accuracy of obstacles under complex backgrounds and changing conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 It is a flow chart of the obstacle measurement method based on a drone provided in an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of the structure of an obstacle measurement device based on a drone provided in an embodiment of the present application;

[0043] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of systems and methods consistent with the examples detailed in the attached claims, or with some aspects of the present application.

[0045] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover descriptions such as non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0046] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0048] The embodiments of the present application provide a drone-based obstacle measurement method, device, electronic device, and storage medium.

[0049] Among them, the obstacle measurement device based on drone can be deployed in a terminal, and the terminal can be a smart phone, a tablet, etc. As mentioned above, the terminal 102 is often equipped with a display device, and the display device can also be a display, a display screen, a touch screen, etc. The touch screen can also be a touch screen, a touch panel, etc., but it is not limited to this. The terminal can communicate with the server in a wired or wireless manner. The server can include an independently running server or a distributed server, or a server cluster composed of multiple servers. The terminal can include a mobile phone, a tablet computer or a personal processor (PC, Personal Computer).

[0050] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0051] The present application provides an obstacle measurement method based on a drone, comprising: obtaining image information and point cloud information collected by a drone; constructing an environmental three-dimensional model corresponding to the current environment based on the image information and point cloud information; identifying obstacles and power lines in the environmental three-dimensional model based on a preset recognition network; and measuring the distance between the obstacles and the power lines based on the point cloud information.

[0052] See also Figure 1 , Figure 1 : is a flow chart of the obstacle measurement method based on drone provided in this application, and the obstacle measurement method based on drone specifically includes the following process:

[0053] 101. Obtain image information and point cloud information collected by the drone.

[0054] For example, a drone carries a high-definition camera to shoot the target scene and obtain the original image data. Then, the original image data is pre-processed using image processing technology, including denoising, enhancement, segmentation and other operations to highlight the key information in the image. The point cloud information collected by the drone is obtained through the laser radar (LiDAR) technology, which can provide the spatial coordinates of each sampling point on the surface of the object, including three-dimensional coordinates (X, Y, Z), color, classification value, intensity value, time and other information. Point cloud data has a wide range of applications in many fields such as autonomous driving, robot navigation, architectural design, virtual reality, etc. due to its unique advantages. The LiDAR system carried by the drone can quickly reconstruct the three-dimensional model of the measured target and various drawing data such as lines, surfaces, and volumes. Point cloud data can be obtained by using laser scanners, depth cameras, binocular cameras, and Structure from Motion (SfM) technology.

[0055] Point cloud data can be stored in various formats, for example:

[0056] PTS: Store point cloud data directly in 3D coordinate order.

[0057] LAS: LiDAR data format that supports multiple hardware and software providers to output a unified format.

[0058] PCD: The official format specified by the PCL library, supporting multidimensional point types.

[0059] XYZ: Text format, containing coordinate and normal information.

[0060] PCAP: A general data stream format, commonly used for lidar data collection.

[0062] Point cloud data processing software such as "Point Cloud Cube" (PCM V2.0) provides a wealth of data processing functions, including point cloud classification, filtering, clipping, digital model generation (DEM, DSM, CHM), etc. In addition, there are professional software such as ContextCapture and Bentley Pointools that can be used for point cloud data management, editing and analysis.

[0063] In practical applications, such as power grid inspection, UAV LiDAR technology can quickly complete the collection and processing of urban distribution network point cloud data, provide high-quality point cloud data, support the planning of UAV automated flight routes, and improve inspection efficiency.

[0064] 102. Construct a three-dimensional model of the environment corresponding to the current environment based on the image information and point cloud information.

[0065] For example, extract visual features from images, such as SIFT, SURF, or ORB feature points. Extract geometric features from point clouds, such as normals, curvature, and shape indices. Use multi-view stereo matching technology to combine image and point cloud data and perform 3D reconstruction through software such as ContextCapture, PhotoScan, or Pix4D mapper. Map image data to the surface of a 3D model to add realistic texture and color information to the model.

[0066] Optionally, in some embodiments of the present application, the step of “constructing a three-dimensional model of the environment corresponding to the current environment according to the image information and the point cloud information” may specifically include:

[0067] Extract image features corresponding to the current environment from the image information;

[0068] Extract point cloud features corresponding to each object in the current environment from the point cloud information;

[0069] Based on image features and point cloud features, a three-dimensional model of the environment corresponding to the current environment is constructed.

[0070] For example, specifically, the images collected by the drone are preprocessed, including denoising, adjusting brightness, color correction, etc. Computer vision algorithms are used to detect key points in the image, such as corners, edges, etc. Then, descriptors are generated for the detected key points, which can represent the features of the image and are used for subsequent matching and reconstruction. At the same time, the point cloud data is preprocessed by filtering, denoising, ground and non-ground point classification, and geometric features such as normals, curvature, shape index, etc. are extracted from the point cloud. Then, the image features are matched with the point cloud features to provide a basis for three-dimensional reconstruction. Then, the depth information of the feature points in the image is estimated by a stereo matching algorithm. Sparse three-dimensional reconstruction is performed using the matched feature points to generate a sparse point cloud. A dense point cloud is generated by a dense matching algorithm, such as block matching or grid matching. Finally, the dense point cloud is converted into a three-dimensional mesh model, which can be achieved by methods such as voxelization and surface extraction (such as Marching Cubes algorithm). The image data is mapped to the surface of the three-dimensional mesh model to add real texture and color information to the model. The three-dimensional mesh model is simplified to reduce the complexity of the model and improve rendering efficiency. Optionally, in some embodiments of the present application, holes and incomplete parts in the model can be repaired.

[0071] Optionally, in some embodiments of the present application, the step of “building a three-dimensional model of the environment corresponding to the current environment based on the image features and the point cloud features” may specifically include:

[0072] Based on the point cloud features, an initial 3D model corresponding to the current environment is constructed;

[0073] According to the image features, the preset multi-view stereo matching algorithm and the initial 3D model, a 3D model of the environment corresponding to the current environment is constructed.

[0074] For example, specifically, the acquired point cloud data is preprocessed by filtering, denoising, ground and non-ground point classification, etc. A preliminary 3D model is constructed using 3D reconstruction software (such as ContextCapture, PhotoScan). Visual features such as SIFT, SURF or ORB feature points are extracted from images collected by drones. Also, geometric features such as normals, curvature, and shape index are extracted from point cloud data. Multi-view stereo matching technology is used to combine image and point cloud data for dense reconstruction using software such as OpenMVS. The generated 3D model is edited and optimized to improve the visual effect and accuracy of the model. Image data is mapped to the surface of the 3D model to add real texture and color information to the model.

[0075] Optionally, in some embodiments of the present application, the step of “constructing a three-dimensional model of the environment corresponding to the current environment according to the image features, the preset multi-view stereo matching algorithm and the initial three-dimensional model” may specifically include:

[0076] Mapping the image features to the initial three-dimensional model to obtain a mapped three-dimensional model;

[0077] Calculate the disparity corresponding to the image information based on the image features and the preset multi-view stereo matching algorithm;

[0078] The mapped three-dimensional model is adjusted according to the parallax to obtain the environmental three-dimensional model corresponding to the current environment.

[0079] For example, after extracting salient features from the point cloud, such as the edges and planes of buildings. Use the point cloud data to generate a rough 3D model framework. Extract visual features, such as SIFT, SURF, or ORB feature points, from the images collected by the drone. Use feature point matching techniques, such as FLANN or BFMatcher, to associate image features with point cloud data. Apply a multi-view stereo matching (MVS) algorithm, such as OpenMVS, to combine image data from multiple perspectives to generate a dense point cloud. Finally, use the MVS algorithm to calculate disparity and convert the disparity information into depth information to generate an initial 3D model. According to the preset multi-view stereo matching algorithm, calculate the disparity of each pixel in the image sequence. Disparity indicates the position difference of the same object in different images. Restore the depth information of the 3D scene by analyzing the difference between the left and right images. According to the calculated disparity information, adjust the geometric details of the initial 3D model, such as adding, modifying, or deleting point cloud data in the model. Use optimization tools in 3D reconstruction software, such as Global Mapper or MeshLab, to optimize and edit the model. Map the image data to the surface of the 3D model to add real texture and color information to the model. Use refinement tools in 3D modeling software to simplify the mesh, fill holes, and smooth the surface of the model.

[0080] 103. Based on the preset recognition network, obstacles and power transmission lines in the three-dimensional model of the environment are identified.

[0081] For example, the visual features extracted from the image are combined with the geometric features extracted from the point cloud. The fused features are input into the trained model to identify obstacles and power lines. The output of the model is analyzed to determine the location of obstacles and power lines.

[0082] Optionally, in some embodiments of the present application, the step of “identifying obstacles and power transmission lines in the three-dimensional model of the environment based on a preset recognition network” may specifically include:

[0083] Get the preset recognition network;

[0084] Determine the target obstacle corresponding to the recognition strategy;

[0085] The height and shape of each object in the three-dimensional model of the environment based on the preset recognition network;

[0086] Determine target obstacles and power lines based on the height and shape of each object.

[0087] For example, specifically, a preset recognition network is obtained, such as the YOLO series (YOLOv8, YOLOv7, YOLOv6, YOLOv5). This recognition network has been widely used in various object detection tasks, including the recognition of obstacles and transmission lines, and the target of the recognition task is clearly defined. For example, in the inspection of transmission lines, the target obstacles may include trees, buildings, vehicles, etc., which may pose a threat to the transmission lines. Then, the geometric features such as the height and shape of each object in the three-dimensional model of the environment are extracted using the point cloud data. These features are crucial for the recognition of obstacles and transmission lines. The visual features extracted from the image (such as SIFT, SURF) and the geometric features extracted from the point cloud are combined and input into the preset recognition network for the recognition of obstacles and transmission lines. The multi-view stereo matching technology is used to combine the image data from multiple perspectives to calculate the disparity of each object in the scene. The disparity information can be used to estimate the depth of the object, thereby assisting in the recognition of obstacles and transmission lines. Using the calculated disparity information, the depth of the objects in the three-dimensional model of the environment is estimated and the position is adjusted to obtain a more accurate spatial position of obstacles and transmission lines. Analyze the results of the identification network output to determine the location, size and shape of obstacles and power lines.

[0088] 104. Measure the distance between obstacles and transmission lines based on point cloud information.

[0089] The acquired point cloud data is filtered and denoised to improve the data quality. Then, the point cloud data is classified to distinguish different ground objects such as ground, vegetation, buildings and transmission lines. The point cloud data of the transmission line is extracted from the classified point cloud. The power line points can be identified by cluster analysis, height information or reflection intensity of the point cloud. Using the extracted power line point cloud data, a three-dimensional model of the transmission line is constructed through a fitting algorithm (such as the least squares method). Objects that pose obstacles to the transmission line, such as trees, buildings, etc., are extracted from the point cloud data. Finally, the distance between the transmission line and the obstacle is calculated.

[0090] Optionally, in some embodiments of the present application, the step of “measuring the distance between the obstacle and the transmission line based on the point cloud information” may specifically include:

[0091] According to the point cloud information, determine the obstacle point cloud corresponding to the obstacle and the transmission line point cloud corresponding to the transmission line;

[0092] According to the cluster center corresponding to the obstacle point cloud and the cluster center corresponding to the transmission line point cloud, the distance between the obstacle and the transmission line is measured.

[0093] For example, specifically, the raw point cloud data is filtered, denoised and classified, such as ground points, obstacle points and transmission line points, and then the obstacle point cloud and transmission line point cloud are extracted from the preprocessed point cloud. This can be achieved by setting a height threshold or other spatial distribution features, and then the extracted point cloud is clustered to determine the cluster centers of obstacles and transmission lines. A Euclidean clustering algorithm such as KD-Tree or region growing method can be used, and then the center of each cluster is determined, which will be the geometric center of the obstacle and transmission line, and finally, the measurement is performed based on the straight-line distance between the cluster centers.

[0094] Optionally, in some embodiments of the present application, after the step of “measuring the distance between the obstacle and the transmission line according to the cluster center corresponding to the obstacle point cloud and the cluster center corresponding to the transmission line point cloud”, the following may be specifically included:

[0095] Obtain safe distance standards for transmission lines;

[0096] Analyze the measurement results to determine whether the distance between the obstacle and the transmission line poses a safety hazard.

[0097] According to the "110kV ~ 750kV Overhead Transmission Line Construction and Acceptance Specification GB 50233-2014", the safety distance standard of the transmission line is obtained. Calculate the straight-line distance between the obstacle point cloud cluster center and the transmission line point cloud cluster center. According to the measurement results and the safety distance standard, analyze whether there are safety hazards. If the distance between the obstacle and the transmission line is less than the specified safety distance, there is a safety hazard.

[0098] The above is the obstacle measurement process based on a drone provided in the embodiment of the present application.

[0099] In summary, after acquiring the image information and point cloud information collected by the drone, the embodiment of the present application constructs a three-dimensional model of the environment corresponding to the current environment based on the image information and point cloud information, then identifies obstacles and power lines in the three-dimensional model of the environment based on a preset recognition network, and finally measures the distance between the obstacles and the power lines based on the point cloud information. In the obstacle measurement solution based on drones provided in the present application, obstacles and power lines in the three-dimensional model of the environment are identified based on a preset recognition network, which can improve the measurement accuracy of obstacles under complex backgrounds and changing conditions.

[0100] In order to better implement the obstacle measurement method based on drone of the embodiment of the present application, the embodiment of the present application also provides an obstacle measurement device based on drone. The meaning of the terms is the same as that in the obstacle measurement system based on drone mentioned above, and the specific implementation details can refer to the description in the system embodiment.

[0101] See also Figure 2 , Figure 2 The schematic diagram of the structure of the obstacle measurement device based on a drone provided in the embodiment of the present application, wherein the obstacle measurement device based on a drone may specifically include an acquisition module 201, a construction module 202, an identification module 203 and a measurement module 204, which may be specifically as follows:

[0102] Acquisition module 201, used to acquire image information and point cloud information collected by the drone;

[0103] A construction module 202 is used to construct a three-dimensional model of the environment corresponding to the current environment based on the image information and the point cloud information;

[0104] The recognition module 203 is used to recognize obstacles and power transmission lines in the three-dimensional model of the environment based on a preset recognition network;

[0105] The measurement module 204 is used to measure the distance between the obstacle and the transmission line based on the point cloud information.

[0106] Optionally, in some embodiments of the present application, the construction module 202 may specifically include:

[0107] A first extraction unit, used to extract image features corresponding to the current environment from the image information;

[0108] The second extraction unit is used to extract point cloud features corresponding to each object in the current environment from the point cloud information;

[0109] The construction unit is used to construct a three-dimensional model of the environment corresponding to the current environment based on image features and point cloud features.

[0110] Optionally, in some embodiments of the present application, the construction unit may specifically include:

[0111] A mapping subunit, used for mapping the image features into the initial three-dimensional model to obtain a mapped three-dimensional model;

[0112] A calculation subunit, used to calculate the disparity corresponding to the image information according to the image features and a preset multi-view stereo matching algorithm;

[0113] The adjustment subunit is used to adjust the mapped three-dimensional model according to the parallax to obtain the environmental three-dimensional model corresponding to the current environment.

[0114] Optionally, in some embodiments of the present application, the identification module 203 may be specifically used for:

[0115] Get the preset recognition network;

[0116] Determine the target obstacle corresponding to the recognition strategy;

[0117] The height and shape of each object in the three-dimensional model of the environment based on a preset recognition network;

[0118] Determine target obstacles and power lines based on the height and shape of each object.

[0119] Optionally, in some embodiments of the present application, the measurement module 204 may be specifically used for:

[0120] According to the point cloud information, determine the obstacle point cloud corresponding to the obstacle and the transmission line point cloud corresponding to the transmission line;

[0121] According to the cluster center corresponding to the obstacle point cloud and the cluster center corresponding to the transmission line point cloud, the distance between the obstacle and the transmission line is measured.

[0122] Optionally, in some embodiments of the present application, the measurement module 204 may also be used to:

[0123] Obtain safe distance standards for transmission lines;

[0124] Analyze the measurement results to determine whether the distance between the obstacle and the transmission line poses a safety hazard.

[0125] The embodiment of the present application provides an obstacle measurement device based on a drone. After the acquisition module 201 acquires the image information and point cloud information collected by the drone, the construction module 202 constructs a three-dimensional model of the environment corresponding to the current environment based on the image information and the point cloud information. Then, the recognition module 203 recognizes obstacles and power lines in the three-dimensional model of the environment based on a preset recognition network. Finally, the measurement module 204 measures the distance between the obstacle and the power line based on the point cloud information. In the obstacle measurement solution based on a drone provided in the present application, obstacles and power lines in the three-dimensional model of the environment are recognized based on a preset recognition network, which can improve the measurement accuracy of obstacles under complex backgrounds and changing conditions.

[0126] In addition, the present application also provides an electronic device, such as Figure 3 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0127] The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more processor-readable storage media, a power supply 303, and an input unit 304. Those skilled in the art will appreciate that Figure 3 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0128] The processor 301 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 302, and calling data stored in the memory 302, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0129] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and obstacle measurement methods based on drones by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0130] The electronic device also includes a power supply 303 for supplying power to each component. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 303 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0131] The electronic device may further include an input unit 304, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0132] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 301 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application programs stored in the memory 302, thereby realizing various functions, as follows:

[0133] Acquire image information and point cloud information collected by the drone; construct a three-dimensional model of the environment corresponding to the current environment based on the image information and point cloud information; identify obstacles and power lines in the three-dimensional model of the environment based on a preset recognition network; and measure the distance between obstacles and power lines based on the point cloud information.

[0134] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0135] After acquiring the image information and point cloud information collected by the drone, the embodiment of the present application constructs a three-dimensional model of the environment corresponding to the current environment based on the image information and the point cloud information, then identifies obstacles and power lines in the three-dimensional model of the environment based on a preset recognition network, and finally measures the distance between the obstacles and the power lines based on the point cloud information. In the obstacle measurement solution based on drones provided in the present application, obstacles and power lines in the three-dimensional model of the environment are identified based on a preset recognition network, which can improve the measurement accuracy of obstacles under complex backgrounds and changing conditions.

[0136] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a processor-readable storage medium and loaded and executed by a processor.

[0137] To this end, an embodiment of the present application provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any of the obstacle measurement methods based on drones provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0138] Acquire image information and point cloud information collected by the drone; construct a three-dimensional model of the environment corresponding to the current environment based on the image information and point cloud information; identify obstacles and power lines in the three-dimensional model of the environment based on a preset recognition network; and measure the distance between obstacles and power lines based on the point cloud information.

[0139] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0140] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0141] Since the instructions stored in the storage medium can execute the steps in any one of the obstacle measurement methods based on drones provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the obstacle measurement methods based on drones provided in the embodiments of the present application can be achieved. For details, please refer to the previous embodiments and will not be repeated here.

[0142] The above is a detailed introduction to the obstacle measurement method, device, electronic device and storage medium based on a drone provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An obstacle measurement method based on a drone, characterized in that: include: Obtain image information and point cloud information collected by drones; Constructing a three-dimensional model of the environment corresponding to the current environment according to the image information and the point cloud information; Identify obstacles and power transmission lines in the three-dimensional model of the environment based on a preset recognition network; The distance between the obstacle and the transmission line is measured based on the point cloud information.

2. The obstacle measurement method according to claim 1, characterized in that: The step of constructing a three-dimensional model of the environment corresponding to the current environment according to the image information and the point cloud information includes: Extracting image features corresponding to the current environment from the image information; Extracting point cloud features corresponding to each object in the current environment from the point cloud information; Based on the image features and point cloud features, a three-dimensional model of the environment corresponding to the current environment is constructed.

3. The obstacle measurement method according to claim 2, characterized in that: The step of constructing a three-dimensional model of the environment corresponding to the current environment based on the image features and the point cloud features includes: Based on the point cloud features, construct an initial three-dimensional model corresponding to the current environment; An environmental three-dimensional model corresponding to the current environment is constructed according to the image features, a preset multi-view stereo matching algorithm and the initial three-dimensional model.

4. The obstacle measurement method according to claim 3, characterized in that: The step of constructing a three-dimensional model of the environment corresponding to the current environment according to the image features, a preset multi-view stereo matching algorithm and the initial three-dimensional model includes: Mapping the image features to the initial three-dimensional model to obtain a mapped three-dimensional model; Calculating the disparity corresponding to the image information according to the image features and a preset multi-view stereo matching algorithm; The mapped three-dimensional model is adjusted according to the parallax to obtain an environmental three-dimensional model corresponding to the current environment.

5. The obstacle measurement method according to any one of claims 1 to 4, characterized in that: The identifying of obstacles and power transmission lines in the three-dimensional model of the environment based on a preset identification network includes: Get the preset recognition network; Determine the target obstacle corresponding to the recognition strategy; The height and shape of each object in the three-dimensional model of the environment based on a preset recognition network; Determine target obstacles and power lines based on the height and shape of each object.

6. The obstacle measurement method according to any one of claims 1 to 4, characterized in that: The measuring the distance between the obstacle and the transmission line based on the point cloud information includes: Determine, according to the point cloud information, an obstacle point cloud corresponding to the obstacle and a transmission line point cloud corresponding to the transmission line; The distance between the obstacle and the transmission line is measured according to the cluster center corresponding to the obstacle point cloud and the cluster center corresponding to the transmission line point cloud.

7. The obstacle measurement method according to any one of claims 1 to 4, characterized in that: After measuring the distance between the obstacle and the transmission line based on the point cloud information, the method further comprises: Obtain safe distance standards for transmission lines; The measurement results are analyzed to determine whether there is a safety hazard at the distance between the obstacle and the transmission line.

8. An obstacle measurement device based on a drone, characterized in that: include: The acquisition module is used to obtain the image information and point cloud information collected by the drone; A construction module, used to construct a three-dimensional model of the environment corresponding to the current environment according to the image information and the point cloud information; An identification module, used for identifying obstacles and power transmission lines in the three-dimensional model of the environment based on a preset identification network; A measuring module is used to measure the distance between the obstacle and the transmission line based on the point cloud information.

9. An electronic device, characterized in that: include: A memory, a processor, and a processing program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the obstacle measurement method based on a drone as described in any one of claims 1 to 7 are performed.

10. A storage medium, characterized in that: A computer processing program is stored which can be loaded by a processor and execute the obstacle measurement method based on a drone as described in any one of claims 1 to 7.

Images