Unmanned aerial vehicle infrastructure construction supervision method and system based on 5G

Abstract

The invention discloses an unmanned aerial vehicle infrastructure construction supervision method and system based on 5G, and belongs to the field of unmanned aerial vehicle monitoring. The unmanned aerial vehicle infrastructure construction supervision system comprises an unmanned aerial vehicle module, a 5G communication module, a data processing module, a cloud storage module, a mobile terminal module, a system management module and a visual management module; the unmanned aerial vehicle module is provided with a high-definition camera, an infrared sensor and a laser range finder, and high-quality construction site data is obtained; the data processing module processes and analyzes the data acquired by the unmanned aerial vehicle; the visual management module compares data acquired by the unmanned aerial vehicle with a construction plan, can generate a real-time visual chart of the construction progress, and displays the construction progress to a supervisor in a visual manner; real-time image transmission is achieved through the system, supervisors can know the progress condition of a project in real time, discover problems in time and take corresponding measures, construction efficiency and quality are improved, and cost and risks are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of drone monitoring, and specifically relates to a 5G-based drone infrastructure construction supervision method and system. Background Art

[0002] In recent years, drones have been widely used in disaster monitoring, ecological environment assessment, rescue command, dangerous area detection and other fields due to their unique advantages such as light size and easy operation, which effectively reduces the cost of manual detection and reduces safety hazards. The application research around drones has also become a hot topic at present. Relying on the gradually mature computer vision technology, drone line patrols to collect images of problematic components of transmission lines and conduct intelligent detection of problematic components, which has a certain theoretical and application basis. In the power grid, drone technology is mainly used in line inspection. In recent years, it has replaced manual inspections and become the main method of power grid line inspection. In power construction, drones are mainly used for traction ropes in the process of laying out wires. The lightweight and strong traction ropes are tied under drones in the form of multi-rotors, and then fly over the tower suspension point to facilitate subsequent work. This method has been widely used in power construction.

[0003] Airborne LiDAR technology is one of the revolutionary achievements in the field of photogrammetry and remote sensing in the past decade, and is currently the most advanced three-dimensional aerial remote sensing technology; laser point cloud mapping technology is essentially a form of point cloud technology; point cloud technology refers to a technology that uses a massive set of points to represent the coordinates and distribution of objects in space. By drawing a large number of points in the air and using these points to form a data set, a three-dimensional model is established to represent the surface characteristics of the space; in short, laser panoramic laser point cloud mapping technology can make three-dimensional simulation of virtual space possible. Laser point cloud obtains ground information and can truly and accurately reflect the terrain and landforms, effectively making up for the shortcomings of aerial photogrammetry technology; The full life cycle management of infrastructure construction can be divided into different stages. Taking the design, construction, completion and acceptance stages as an example, the existing infrastructure construction supervision system cannot intuitively feedback the progress of the project and cannot quickly track the progress of the project construction. In order to achieve intelligent management of the power infrastructure construction process, improve construction technology and reduce on-site costs.

[0004] To this end, technicians in this field have proposed a 5G-based drone infrastructure construction supervision method and system to solve the problems raised by the background technology. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a 5G-based drone infrastructure construction supervision method and system to solve the problems in the prior art that the infrastructure construction supervision system cannot intuitively feedback the progress of the project and cannot quickly track the progress of the project construction.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: A 5G-based drone infrastructure construction supervision system, including a drone module, a data processing module and a visual management module; The drone module is equipped with a high-definition camera, an infrared sensor and a laser rangefinder to obtain images of the construction site, extract features from the images of the construction site to form three-dimensional point cloud data, and obtain the construction site data through the three-dimensional point cloud data; The data processing module analyzes the construction site data acquired by the drone and data collected from other data sources to obtain the construction progress; The visual management module compares the construction progress with the construction plan, generates a real-time visual chart of the construction progress, and displays the construction progress to supervisors in a visual manner.

[0007] Furthermore, it also includes an early warning module, which is used to monitor and warn of abnormal conditions at the construction site in real time, discover potential construction quality problems and safety hazards through data analysis and model prediction, and send early warning information.

[0008] Furthermore, it also includes a 5G communication module, which is used to realize data transmission between the drone and the data processing module and the visualization management module.

[0009] Furthermore, it also includes a cloud storage module, which is used to store drone construction site images, three-dimensional point cloud data and analysis results of the data processing module.

[0010] Furthermore, it also includes a system management module, which is used to manage and maintain users, permissions, equipment, etc. of the supervision system; the system administrator can divide roles and set permissions for users.

[0011] A 5G-based drone infrastructure construction supervision method includes the following steps: Acquire construction site images, extract features from the construction site images to form three-dimensional point cloud data, and obtain construction site data through the three-dimensional point cloud data; Analyze the construction site data collected by drones and other data sources to obtain construction progress; Compare the construction progress with the construction plan, generate and display a real-time visual chart of the construction progress.

[0012] Furthermore, feature extraction is performed on the construction site image to form three-dimensional point cloud data, including the following steps: S1. Camera calibration: Use a known calibration plate or calibration object to determine the internal and external parameters of the camera through the images taken by the camera; S2, feature extraction: extract feature points from the image taken by the camera; S3, feature matching: match the feature points in different images to obtain points with the same name; S4, triangulation: using the image coordinates of the same-name points and the internal and external parameters of the camera, the coordinates of each feature point in three-dimensional space are calculated through the principle of triangulation; S5. Point cloud generation: combine the three-dimensional coordinates of all feature points to form three-dimensional point cloud data.

[0013] Furthermore, in step S2, a SIFT feature extraction algorithm is used to perform feature extraction.

[0014] Furthermore, the SIFT feature extraction algorithm includes the following steps: S201, constructing a Gaussian pyramid: constructing the original image into multiple Gaussian pyramids of different scales; S202, constructing a DoG pyramid: performing a difference operation on two adjacent layers of the Gaussian pyramid to obtain a DoG pyramid; S203, extracting feature points: In each layer of the DoG pyramid, by comparing the gray value difference between each pixel and its surrounding pixels, find the local maximum point as the feature point.

[0015] Furthermore, the SIFT feature extraction algorithm also includes calculating a feature descriptor: the process of calculating the feature descriptor is: for each feature point, a neighborhood window is selected with it as the center, and a directional histogram is calculated in the window; the directional histogram is used as the feature descriptor of the feature point.

[0016] Compared with the prior art, the present invention has at least the following beneficial technical effects: 1. The present invention realizes real-time image transmission through this system: using a high-definition camera mounted on a drone and the high-speed and low-latency characteristics of a 5G base station, the images collected by the drone are transmitted to a remote terminal in real time, which is convenient for experts to conduct remote diagnosis and guidance; the on-site images are remotely backed up, which is convenient for on-site data playback and tracing. Supervisors can understand the progress of the project in real time, find problems in time and take corresponding measures, improve construction efficiency and quality, and reduce costs and risks; 2. Realized edge computing image recognition in the construction environment: UAV images can be used to monitor the construction environment and civilized construction in real time through the visualization management module. It can not only generate real-time visualization charts of the construction progress and display the construction progress to supervisors in a visual way, but also identify and record the safety measures on site, personnel clothing, and construction site protection; 3. Construction progress management is realized: the construction area is collected by drones carrying laser radars to complete the visual management of the construction progress. At the same time, it is compared with the design drawings to detect the gaps in construction technology.

[0017] 4. The three-dimensional point cloud data acquisition algorithm of the present invention obtains high-precision three-dimensional point cloud data through photogrammetry, uses the SIFT feature extraction algorithm to extract features, finally obtains feature points, and calculates the direction histogram as a feature descriptor, thereby realizing the extraction and description of image features; the features extracted by the above method have the advantages of rotation invariance, scale invariance, illumination invariance, strong anti-noise ability and good repeatability, and can effectively improve the accuracy and reliability of the infrastructure construction supervision system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the overall structure of the present invention; Figure 2 Schematic diagram of the steps for obtaining 3D point cloud data. DETAILED DESCRIPTION

[0019] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0020] Example 1 As attached Figure 1 To Attachment Figure 2 As shown: The present invention provides a 5G-based drone infrastructure construction supervision system, including a drone module, a 5G communication module, a data processing module, a cloud storage module, a mobile terminal module, a system management module and a visualization management module.

[0021] The mobile terminal module supports mobile applications. Through the drone module, supervisors can view the data and progress of the construction site anytime and anywhere, thus improving work efficiency. The drone module is equipped with a high-definition camera, infrared sensor and laser rangefinder to obtain high-quality construction site images, extract features through SIFT, form three-dimensional point cloud data, and obtain construction site data through three-dimensional point cloud data: such as the terrain and geomorphic features of the construction site, the buildings and facilities at the construction site, and the personnel and equipment at the construction site; based on the three-dimensional design data of the infrastructure project, the automatic flight route of the drone at the construction site is planned in advance to ensure flight safety and route reuse in the full life cycle management. After the drone arrives at the site, it can take off directly and fly automatically, with a unified shooting angle and position. In special circumstances, it can switch to manual mode to achieve the best shooting effect; through the real-time picture of the drone and the use of edge computing image recognition technology, the construction site personnel positioning, whether the safety helmet is worn, and the automatic identification and alarm of the safety belt wearing of the tower climbers are completed, the picture is displayed in real time, and the alarm information is recorded; The data processing module is responsible for collecting data from drones and other data sources and processing and analyzing the data to extract valuable information such as construction progress, quality issues, and safety hazards.

[0022] Drones regularly take photos or videos of the construction site and analyze the progress of the construction by comparing images at different time points; the data processing module can analyze data collected from other data sources, such as project plans, construction schedules, etc., and compare them with the actual construction situation to obtain information on the construction progress.

[0023] Drones conduct detailed monitoring and inspection of construction sites, acquire images and data, and send them to the data processing module, which can analyze these images or data to detect construction quality problems, such as material defects, irregular construction, etc. Quality inspection reports and project acceptance records collected from other data sources can also provide information about quality problems. Drones can detect safety hazards at construction sites through real-time video transmission or image analysis, such as people not wearing safety helmets and activities in dangerous areas.

[0024] Other data sources, such as safety inspection reports, accident records, etc., can also provide information about safety hazards; The visual management module compares the data collected by drones with the construction progress and the construction plan, generates a real-time visual chart of the construction progress, and displays the construction progress to supervisors in a visual way; The visual management module can be used to monitor and evaluate quality based on data collected by drones. Image recognition and data analysis technology can be used to monitor and evaluate construction quality. Problems with material quality and construction technology at the construction site can be identified and evaluated based on preset quality standards.

[0025] Through the above technical solutions, the system can collect data from the construction site in real time, convert it into visual information and display it, helping supervisors to understand the progress of the project in real time; through smart sensors and Internet of Things technology, the system can accurately collect data from the construction site and improve the accuracy and reliability of the data; the visual management module uses a visual display platform to display the data from the construction site in the form of charts, graphs, etc., helping supervisors to understand the progress of the project more intuitively; the system supports mobile applications, allowing supervisors to view the data and progress of the construction site anytime and anywhere, thereby improving work efficiency.

[0026] It also includes an early warning module; the early warning module performs real-time monitoring and early warning of abnormal conditions at the construction site, discovers potential construction quality problems and safety hazards through data analysis and model prediction based on the construction site images and three-dimensional point cloud data collected by the drone, and sends early warning information to supervisors in a timely manner.

[0027] The 5G communication module provides high-speed and stable data transmission for each module of the system. The low latency and high bandwidth characteristics of the 5G network ensure the real-time and reliability of data transmission, and realize real-time communication between the drone module and the 5G base station.

[0028] The cloud storage module is used to store data collected by the drone, three-dimensional point cloud data, analysis results of the data processing module and analysis results of the visualization management module.

[0029] The system management module is used to manage and maintain the users, permissions, equipment, etc. of the supervision system; the system administrator can divide the roles and set permissions for users to ensure the security and stability of the system.

[0030] Real-time image transmission is achieved through this system: a high-definition camera is mounted on the drone, and the high speed and low latency characteristics of the 5G base station are utilized to transmit the images collected by the drone in real time to a remote terminal. The remote terminal can be a mobile phone or computer, which is convenient for experts to conduct remote diagnosis and guidance. On-site images are remotely backed up to facilitate on-site data playback and tracing.

[0031] Edge computing image recognition in construction environment: UAV images can be used to monitor the construction environment and civilized construction in real time through the visualization management module. It can not only generate real-time visualization charts of the construction progress and display the construction progress in a visual way to supervisors, but also identify and record on-site safety measures, personnel clothing, construction site protection, etc.

[0032] Construction progress management: Use drones to carry laser radars to collect images of the construction area, and cooperate with data processing modules and visualization management modules to complete the visualization management of the construction progress. At the same time, compare it with the design drawings to detect construction process gaps.

[0033] Example 2 As attached Figure 2 As shown: A three-dimensional point cloud data acquisition method based on photogrammetry, used in conjunction with the above-mentioned 5G-based drone infrastructure construction supervision system, includes the following steps: S1. Camera calibration: The drone module is equipped with a high-definition camera, infrared sensor and laser rangefinder to obtain high-quality construction site data. The camera's internal and external parameters, including the camera's focal length, principal point coordinates and distortion coefficient, are determined through images taken by the camera using known calibration plates or objects. S2. Feature extraction: Extract feature points from the captured images and use the traditional SIFT feature extraction algorithm for feature extraction. The SIFT feature extraction algorithm has good scale invariance and rotation invariance. The extracted features have the advantages of rotation invariance, scale invariance, illumination invariance, strong anti-noise ability and good repeatability. It is suitable for image matching at different scales and rotation angles. It can effectively improve the accuracy and reliability of the infrastructure construction supervision system. S3, feature matching: match the feature points in different images to obtain points with the same name; S4, triangulation: using the image coordinates of the same-name points and the internal and external parameters of the camera, the coordinates of each feature point in three-dimensional space are calculated through the principle of triangulation; S5, point cloud generation: combine the three-dimensional coordinates of all feature points to form three-dimensional point cloud data; S6, point cloud optimization: optimize the generated 3D point cloud data and remove noise points in the point cloud, such as outliers, wrong matching points, etc., to improve the quality of the point cloud; S7. Point cloud stitching: When obtaining a larger range of three-dimensional point cloud data, stitch multiple point cloud data together.

[0034] In step S2, the SIFT feature extraction algorithm steps are: S201, constructing a Gaussian pyramid: constructing the original image into multiple Gaussian pyramids of different scales to extract image features at different scales; S202, constructing a DoG pyramid: performing a difference operation on two adjacent layers of a Gaussian pyramid to obtain a DoG pyramid for extracting edge and detail information of the image; S203, extracting feature points: in each layer of the DoG pyramid, by comparing the gray value difference between each pixel and its surrounding pixels, find the local maximum point as the feature point; S204, calculating feature descriptors: for each feature point, a neighborhood window is selected with the feature point as the center, and a directional histogram is calculated within the window; the directional histogram is used as the feature descriptor of the feature point; by using feature descriptors, the similarities between a large number of feature points can be quickly compared, thereby achieving fast feature matching.

[0035] From the above, it can be seen that the 3D point cloud data acquisition algorithm can obtain high-precision 3D point cloud data through photogrammetry, thereby realizing comprehensive supervision and refined management of infrastructure construction. The SIFT feature extraction algorithm includes the steps of constructing a Gaussian pyramid, constructing a DoG pyramid, extracting feature points and calculating feature descriptors. The algorithm extracts image features at different scales by constructing a Gaussian pyramid and a DoG pyramid, and finds local maximum points as feature points by comparing the gray value difference between each pixel and its surrounding pixels. Finally, the directional histogram is calculated as a feature descriptor to realize the extraction and description of image features. The features extracted by the above method have the advantages of rotation invariance, scale invariance, illumination invariance, strong anti-noise ability and good repeatability, which can effectively improve the accuracy and reliability of the infrastructure construction supervision system.

[0036] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. (Add this paragraph to the instructions for cases involving big data processing, especially user data privacy. Some cases involving user privacy may be corrected) In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0037] The embodiments of the present invention are provided for the purpose of illustration and description. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present invention.

[0038] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A 5G-based drone infrastructure construction supervision system, characterized in that: Includes drone module, data processing module and visual management module; The drone module is equipped with a high-definition camera, an infrared sensor and a laser rangefinder to obtain images of the construction site, extract features from the images of the construction site to form three-dimensional point cloud data, and obtain the construction site data through the three-dimensional point cloud data; The data processing module analyzes the construction site data acquired by the drone and data collected from other data sources to obtain the construction progress; The visual management module compares the construction progress with the construction plan, generates a real-time visual chart of the construction progress, and displays the construction progress to supervisors in a visual manner.

2. According to the 5G-based drone infrastructure construction supervision system of claim 1, it is characterized in that: It also includes an early warning module, which is used to monitor and warn of abnormal conditions at the construction site in real time, discover potential construction quality problems and safety hazards through data analysis and model prediction, and send early warning information.

3. A 5G-based drone infrastructure construction supervision system according to claim 1 or 2, characterized in that: It also includes a 5G communication module, which is used to realize data transmission between the drone and the data processing module and the visualization management module.

4. According to the 5G-based drone infrastructure construction supervision system of claim 1, it is characterized in that: It also includes a cloud storage module, which is used to store drone construction site images, three-dimensional point cloud data and analysis results of the data processing module.

5. According to the 5G-based UAV infrastructure construction supervision system of claim 1, it is characterized in that: It also includes a system management module, which is used to manage and maintain users, permissions, equipment, etc. of the supervision system; the system administrator can divide roles and set permissions for users.

6. A 5G-based drone infrastructure construction supervision method, characterized in that: The following steps are involved: Acquire construction site images, extract features from the construction site images to form three-dimensional point cloud data, and obtain construction site data through the three-dimensional point cloud data; Analyze the construction site data collected by drones and other data sources to obtain construction progress; Compare the construction progress with the construction plan, generate and display a real-time visual chart of the construction progress.

7. A 5G-based UAV infrastructure construction supervision method according to claim 6, characterized in that: The method of extracting features from the construction site image to form three-dimensional point cloud data includes the following steps: S1. Camera calibration: Use a known calibration plate or calibration object to determine the internal and external parameters of the camera through the images taken by the camera; S2, feature extraction: extract feature points from the image taken by the camera; S3, feature matching: match the feature points in different images to obtain points with the same name; S4, triangulation: using the image coordinates of the same-name points and the internal and external parameters of the camera, the coordinates of each feature point in three-dimensional space are calculated through the principle of triangulation; S5. Point cloud generation: combine the three-dimensional coordinates of all feature points to form three-dimensional point cloud data.

8. The 5G-based UAV infrastructure construction supervision method according to claim 7 is characterized in that: In step S2, feature extraction is performed using a SIFT feature extraction algorithm.

9. The 5G-based drone infrastructure construction supervision method according to claim 8 is characterized in that: The SIFT feature extraction algorithm includes the following steps: S201, constructing a Gaussian pyramid: constructing the original image into multiple Gaussian pyramids of different scales; S202, constructing a DoG pyramid: performing a difference operation on two adjacent layers of the Gaussian pyramid to obtain a DoG pyramid; S203, extracting feature points: In each layer of the DoG pyramid, by comparing the gray value difference between each pixel and its surrounding pixels, find the local maximum point as the feature point.

10. A 5G-based UAV infrastructure construction supervision method according to claim 9, characterized in that: The SIFT feature extraction algorithm also includes calculating feature descriptors: the process of calculating feature descriptors is: for each feature point, a neighborhood window is selected with it as the center, and a direction histogram is calculated in the window; the direction histogram is used as the feature descriptor of the feature point.

Images