Low-altitude economic three-dimensional scene construction method and system combined with digital stretching technology

Abstract

The invention belongs to the technical field of three-dimensional modeling, and discloses a low-altitude economic three-dimensional scene construction method and system combined with a digital stretching technology. The method comprises the following steps: acquiring image information of urban facilities on a preset flight path; the image information is acquired by five-eye camera equipment carried by the unmanned aerial vehicle; the image information comprises a first view, a second view, a third view, a fourth view, a fifth view, ambient brightness, a shooting angle, a shooting distance and a depth map; preprocessing the image information to obtain preprocessed image information; based on the preprocessed image information, performing three-dimensional construction on the urban facilities in combination with a digital stretching technology to obtain a three-dimensional structure model of the urban facilities; based on the three-dimensional structure model, generating planning layout information of the urban facilities; and the accuracy of overall layout planning during city expansion is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional modeling technology, and more specifically, to a method and system for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology. Background Art

[0002] The existing Chinese patent application with publication number CN113793412A discloses a 3D modeling method and system for a nuclear power plant based on a 2D plane template diagram, in order to solve the current problem that the modeling of nuclear power plants is complex, time-consuming and error-prone; the invention first simplifies the 2D plane template diagram to form the contour line of the bottom plane, and offsets the plane contour according to the layer height, and then stretches it according to the layer height. In this way, the stretching of the plane diagrams of each layer is completed, and the bottom plate is formed at the floor elevation. After the bottom plate is cut to generate the floor plate, the redundant geometric faces are deleted to generate the 3D geometric model of the nuclear power plant structure. The invention realizes the function of obtaining the required basic data through the 2D plane template diagram; integrates the design drawings into the design simulation project to accurately reflect the real state; realizes automation through the program, and can be used in the digital collaborative design platform of nuclear power structure in the future.

[0003] The existing technology still has the following problems:

[0004] When a city is expanding, the 3D structural models of various urban facilities can provide a reasonable and accurate basis for layout planning. However, traditional 3D modeling methods, such as manual measurement, monocular vision, or 2D plane templates, often have problems of insufficient accuracy and missing details when modeling urban facilities, resulting in insufficient accuracy in the overall layout planning of the city.

[0005] In view of this, the present invention proposes a low-altitude economical three-dimensional scene construction method and system combined with digital stretching technology to solve the above problems. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology, comprising:

[0007] Collect image information of urban facilities along the preset flight path;

[0008] Preprocessing the image information to obtain preprocessed image information;

[0009] Based on the pre-processed image information, the urban facilities are constructed in three dimensions in combination with digital stretching technology to obtain a three-dimensional structural model of the urban facilities;

[0010] Generate planning and layout information of urban facilities based on the three-dimensional structural model.

[0011] Furthermore, the image information is acquired by a five-eye camera device carried by the drone; the image information includes a first view, a second view, a third view, a fourth view, a fifth view, ambient brightness, a shooting angle, a shooting distance and a depth map;

[0012] The pretreatment method comprises:

[0013] Step S1, synchronously acquiring n groups of multi-channel data; the multi-channel data includes image channel data and sensor channel data; the image channel data includes a first view, a second view, a third view, a fourth view, a fifth view and a depth map; the sensor channel data includes ambient brightness, shooting angle and shooting distance; n is a positive integer;

[0014] Step S2: Establish a data buffer area on the drone end and align the multi-channel data at the same time according to the timestamp;

[0015] Step S3, based on n groups of multi-channel data, by comparing the ambient brightness with a preset brightness threshold, obtaining m groups of first processed data; m is a positive integer less than or equal to n;

[0016] Step S4, based on the m groups of first processed data, by comparing the roll angle with a preset roll angle threshold, obtaining q groups of second processed data; q is a positive integer less than or equal to m;

[0017] Step S5, performing clarity detection on the q groups of second processed data to obtain p groups of pre-processed image information; p is a positive integer less than or equal to q;

[0018] Step S6: transmitting the p groups of pre-processed image information to the cloud.

[0019] Further, the pre-processed image information is the processed image information and the corresponding contour probability map; the value of each pixel in the contour probability map represents the probability value of the pixel belonging to the actual contour;

[0020] The method for clarity detection comprises:

[0021] Step A1, removing random noise in the depth map corresponding to the q groups of second processed data;

[0022] Step A2, calling a function in OpenCV to perform Laplace filtering on the image to obtain a Laplace gradient map; the function is cv::Laplace() function;

[0023] Step A3: input the Laplace gradient map into the pre-trained contour detection model and output the contour probability map;

[0024] Step A4, calculating the proportion of pixels in the contour probability map that are greater than a preset probability threshold, and counting the number p of groups of contour probability maps whose pixel proportion is greater than or equal to the preset proportion;

[0025] Step A5: record the p groups of contour probability maps and the corresponding multi-channel data as pre-processed image information.

[0026] Furthermore, the training method of the contour detection model includes:

[0027] Step B1, obtaining a data set containing v sets of contour information from a sample set, wherein the contour information includes a Laplace gradient map and a corresponding contour probability map;

[0028] Step B2, dividing the data set into a training set and a test set according to a preset ratio, constructing a classifier, and performing a binary classification on each pixel in the Laplace gradient image, wherein the binary classification includes whether it is a contour or not a contour;

[0029] Step B3, using the Laplace gradient map in the training set as the input of the classifier, using the corresponding contour probability map in the training set as the output of the classifier, training the classifier to obtain an initial classifier; the classifier is a HED deep learning model;

[0030] Step B4: Use the test set to test the initial classifier, and output the initial classifier that meets the preset accuracy as the final contour detection model.

[0031] Furthermore, the method for constructing the three-dimensional structural model of the urban facilities includes:

[0032] Step Z1: convert the depth map into point cloud data through a stereo matching algorithm to construct a geometric basis of the three-dimensional space; the stereo matching algorithm is the SGBM algorithm;

[0033] Step Z2, combining the contour probability map and the point cloud data, optimizing the geometric basis, and obtaining the optimized geometric basis; the method for optimizing the geometric basis includes: according to the contour probability map, presetting the probability threshold, screening out the point cloud of the high probability area in the contour probability map, and performing key optimization; the high probability area is an area composed of corresponding pixels whose probability values ​​of pixels in the contour probability map are greater than the preset probability threshold;

[0034] Step Z3, based on the optimized geometric foundation, generate a three-dimensional mesh model through point cloud filtering and mesh reconstruction algorithm;

[0035] Step Z4: By using texture mapping technology, the first view, the second view, the third view, the fourth view, and the fifth view are integrated with the three-dimensional grid model to generate a three-dimensional structural model of the urban facilities; the three-dimensional structural model also includes corresponding multiple groups of environmental brightness, shooting angles, and shooting distances.

[0036] Furthermore, the shooting distance is the distance between the five-eye camera and any point on the top surface of the urban facility when the drone takes the photo; the shooting distance is obtained by converting the product of the baseline length and the focal length into an absolute value and dividing it by the parallax;

[0037] The method for calculating the distance from each pixel block to the five-eye camera in the depth map and the method for obtaining the shooting distance are both parallax methods, and the depth map is obtained by the five-eye camera carried by the drone.

[0038] Furthermore, the shooting angles include pitch angle, roll angle and yaw angle;

[0039] The method for acquiring the second processed data includes: comparing the roll angle with a preset roll angle threshold; if the roll angle is greater than or equal to the preset roll angle threshold, discarding the corresponding first processed data; if the roll angle is less than the preset roll angle threshold, retaining the corresponding first processed data; counting the number of retained first processed data groups, and recording them as the number of second processed data groups q.

[0040] Furthermore, the method for obtaining the first processed data includes:

[0041] Compare the ambient brightness with the preset brightness threshold; if the ambient brightness is greater than or equal to the preset brightness threshold, discard the corresponding multi-channel data; if the ambient brightness is less than the preset brightness threshold, retain the corresponding multi-channel data; count the number of retained multi-channel data groups and record them as the number m of groups of the first processed data.

[0042] Furthermore, the planning layout information includes the structure type, size ratio and spatial position of the target urban facilities in the city.

[0043] A low-altitude economical three-dimensional scene construction system combined with digital stretching technology, and a low-altitude economical three-dimensional scene construction method combined with digital stretching technology, comprising:

[0044] A data acquisition module, used to collect image information of urban facilities on a preset flight path;

[0045] An image processing module, used for preprocessing image information to obtain preprocessed image information;

[0046] The three-dimensional construction module constructs the urban facilities in three dimensions based on the pre-processed image information and combines the digital stretching technology to obtain the three-dimensional structural model of the urban facilities;

[0047] The information generation module generates the planning and layout information of urban facilities based on the three-dimensional structural model.

[0048] The technical effects and advantages of the low-altitude economical three-dimensional scene construction method and system combined with digital stretching technology of the present invention are as follows:

[0049] The present invention collects image information of urban facilities through a five-eye camera carried by an unmanned aerial vehicle, and then pre-processes the image information to obtain pre-processed image information; then, based on the pre-processed image information, the urban facilities are three-dimensionally constructed in combination with digital stretching technology to obtain a three-dimensional structural model of the urban facilities; finally, the planning and layout information of the urban facilities is generated through the high-precision three-dimensional structural model of the urban facilities, which effectively improves the accuracy of the overall layout planning during urban expansion.

[0050] The present invention combines contour probability maps with point cloud data to optimize the geometric basis, which not only significantly improves the edge accuracy and detail performance of the three-dimensional structural model of urban facilities, but also effectively reduces noise and errors, and enhances the geometric accuracy and visual realism of the three-dimensional structural model.

[0051] In the process of image information preprocessing, the present invention adopts multiple processing steps. First, the first processing data is obtained based on the comparison of ambient brightness, which helps to improve the overall quality of the image information; secondly, the second processing data is obtained based on the comparison of the roll angle to reduce the geometric distortion in the image and ensure the alignment accuracy of multiple views; finally, the second processing data is tested for clarity to obtain preprocessed image information to ensure that subsequent edge detection and three-dimensional reconstruction are based on high-quality image data. This process reduces the negative impact of blurred images on depth estimation and edge detection, and improves the detail performance and accuracy of the three-dimensional model. In addition, the multiple screening steps reduce the system's tolerance for low-quality or abnormal data, ensure that the data input to the three-dimensional modeling module is of higher quality, and reduce errors or distortions caused by data problems in the subsequent modeling process.

[0052] The present invention inputs the Laplace gradient map into a pre-trained contour detection model (such as HED) and outputs a high-quality contour probability map, which embodies the organic combination of classical image processing methods and modern deep learning technology. Through this combination, not only the accuracy and detail expression of contour detection are improved, but also a more solid data support is provided for the construction of subsequent three-dimensional models of urban facilities, ensuring the high precision and high quality of the final model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a schematic diagram of a low-altitude economical three-dimensional scene construction system combined with digital stretching technology according to Example 1 of the present invention;

[0054] Figure 2 This is a flow chart of a method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to Embodiment 2 of the present invention;

[0055] Figure 3 This is a flow chart of the pretreatment method of Example 1 of the present invention;

[0056] Figure 4 This is a flow chart of the method for clarity detection according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Example 1

[0059] See also Figure 1 As shown, the low-altitude economical three-dimensional scene construction system combined with digital stretching technology described in this embodiment includes: a data acquisition module, an image processing module, a three-dimensional construction module and an information generation module, and each module is connected by wired and / or wireless connection.

[0060] The urban facilities include scenic area facilities, residential facilities and transportation facilities, etc.; the scenic area facilities include observation decks and aerial plank roads, etc.

[0061] The data acquisition module is used to collect image information of scenic area facilities on a preset flight path; the image information is obtained by collecting the five-eye camera device carried by the drone; the image information includes the first view, the second view, the third view, the fourth view, the fifth view, the ambient brightness, the shooting angle, the shooting distance and the depth map; the shooting angle includes the pitch angle, the roll angle and the yaw angle.

[0062] It should be noted that the five lenses of the five-eye camera are arranged linearly, and the five cameras are arranged along a straight line with the same baseline length.

[0063] The first view, second view, third view, fourth view and fifth view are captured by the five lenses of the five-eye camera. Adjacent lenses capture the same scene through a fixed baseline to form parallax, which serves as the basis for calculating the depth values ​​of pixel points in subsequent depth maps. At the same time, multiple views enable the system to build a more detailed three-dimensional model, which helps to improve the accuracy of the subsequent model difference rate.

[0064] The ambient brightness affects the contrast and clarity when photographing scenic area facilities, thereby affecting the calculation accuracy of the subsequent model difference rate; the shooting angle is used for geometric correction of multiple views of scenic area facilities; the shooting distance determines the size and proportion of multiple views of scenic area facilities. At the same time, a shooting distance that is too far will reduce the accuracy of the depth map of the scenic area facilities, thereby affecting the three-dimensional reconstruction effect of the scenic area facilities.

[0065] The pitch angle is the rotation angle of the pentacular camera around the horizontal axis, indicating the upward or downward tilt of the pentacular camera, which will affect the accuracy when acquiring multiple views; the roll angle is the rotation angle of the pentacular camera around the vertical axis, indicating the left and right tilt of the pentacular camera. Too large a roll angle will change the shooting distance, thereby affecting the accuracy of the depth map; the yaw angle is the rotation angle of the pentacular camera around the vertical axis, indicating the left and right rotation of the pentacular camera.

[0066] The first view, the second view, the third view, the fourth view and the fifth view are all obtained by photographing the five cameras carried by the drone, and the shooting interval is preset to j seconds, where j can be any positive number; the ambient brightness is obtained by an ambient light sensor equipped with the five cameras carried by the drone; and the shooting angle is obtained by an inertial measurement unit (IMU) sensor on the drone.

[0067] It should be noted that when the drone is flying and shooting, the five cameras are fixed on the drone. At this time, the shooting angle is the attitude angle of the drone, which can be obtained through the inertial measurement unit sensor of the drone.

[0068] The shooting distance is the distance between the five-eye camera and any point on the top surface of the scenic spot facilities when the drone takes photos; the shooting distance D calculation method includes:

[0069]

[0070] Wherein, D is the shooting distance; L is the baseline length of the five-eye camera, which is the preset fixed distance between any two adjacent lenses in the five-eye camera; f is the preset focal length of any two adjacent lenses in the five-eye camera; d1 and d2 are the horizontal coordinate positions of the scenic area facilities in the views taken by any two adjacent lenses, respectively; d1-d2 is the parallax, which is the difference in the horizontal coordinate positions of the scenic area facilities in the views taken by any two adjacent lenses.

[0071] The method for calculating the distance from each pixel block to the five-eye camera in the depth map and the method for obtaining the shooting distance are both parallax methods, and the depth map is obtained by the five-eye camera carried by the drone.

[0072] The image processing module is used to preprocess the image information to obtain preprocessed image information; the preprocessed image information is the processed image information and the corresponding contour probability map.

[0073] See also Figure 3 As shown, the preprocessing method includes:

[0074] Step S1, synchronously acquiring n groups of multi-channel data; the multi-channel data includes image channel data and sensor channel data; the image channel data includes a first view, a second view, a third view, a fourth view, a fifth view and a depth map; the sensor channel data includes ambient brightness, a shooting angle and a shooting distance; n is a positive integer.

[0075] Step S2: Establish a data buffer area on the drone side to align the multi-channel data at the same time according to the timestamp to avoid errors in subsequent processing due to timing misalignment.

[0076] Step S3: Based on the n sets of multi-channel data, m sets of first processed data are obtained; the method for obtaining the first processed data includes:

[0077] Compare the ambient brightness with the preset brightness threshold; if the ambient brightness is greater than or equal to the preset brightness threshold, such as strong direct sunlight, discard the corresponding multi-channel data; if the ambient brightness is less than the preset brightness threshold, retain the corresponding multi-channel data; count the number of retained multi-channel data groups and record them as the number of groups m of the first processed data; m is a positive integer less than or equal to n.

[0078] Step S4: Based on the m groups of first processed data, q groups of second processed data are obtained; the method for obtaining the second processed data includes:

[0079] Compare the roll angle with the preset roll angle threshold; if the roll angle is greater than or equal to the preset roll angle threshold, resulting in distortion of the image channel data, the corresponding first processed data is discarded; if the roll angle is less than the preset roll angle threshold, the corresponding first processed data is retained; count the number of retained first processed data groups and record them as the number of second processed data groups q; q is a positive integer less than or equal to m.

[0080] Step S5: Perform clarity detection on the q groups of second processed data to obtain p groups of pre-processed image information.

[0081] See also Figure 4 As shown, the method for clarity detection includes:

[0082] Step A1: remove random noise in the depth map corresponding to the q groups of second processed data.

[0083] Step A2, calling a function in OpenCV to perform Laplace filtering on the image to obtain a Laplace gradient map; the function is the cv::Laplace() function; the pixel value in the Laplace gradient map represents the intensity of the local grayscale change in the Laplace gradient map, the larger the pixel value, the more prominent the local edge in the Laplace gradient map, and the clearer the Laplace gradient map; if the pixel values ​​in the Laplace gradient map are all small, the Laplace gradient map is relatively blurred.

[0084] Step A3: input the Laplace gradient map into a pre-trained contour detection model, and output a contour probability map; the value of each pixel in the contour probability map represents the probability value of the pixel belonging to the actual contour.

[0085] It should be noted that the contour probability map is a form of image representation generated by a machine learning model in image processing and computer vision.

[0086] It should be noted that the probability value is between 0 and 1, or mapped to a grayscale value range of 0 to 255 through appropriate scaling for visualization; the higher the value of a pixel, the more likely it is that the pixel is located at a contour position; the lower the value of a pixel, the less likely it is that the pixel is part of a contour.

[0087] The training method of the contour detection model includes:

[0088] Step B1, obtaining a data set containing v sets of contour information from a sample set, wherein the contour information includes a Laplace gradient map and a corresponding contour probability map;

[0089] Step B2, dividing the data set into a training set and a test set according to a preset ratio, constructing a classifier, and performing a binary classification on each pixel in the Laplace gradient image, wherein the binary classification includes whether it is a contour or not a contour;

[0090] Step B3, using the Laplace gradient map in the training set as the input of the classifier, using the corresponding contour probability map in the training set as the output of the classifier, training the classifier to obtain an initial classifier; the classifier is a HED deep learning model;

[0091] Step B4: Use the test set to test the initial classifier, and output the initial classifier that meets the preset accuracy as the final contour detection model.

[0092] It should be noted that the Laplace gradient map and the corresponding contour probability map correspond one to one in the spatial coordinates.

[0093] It should be noted that the HED deep learning model is a type of deep network specifically for edge detection. It can learn and fuse edge information at multiple scales to generate high-quality edge maps. Using the Laplacian gradient map as the input of the contour detection model makes it easier for HED-type networks to identify true contours and ignore noise and inconspicuous gradient areas.

[0094] Step A4, calculate the proportion of pixels in the contour probability map that are greater than the preset probability threshold, and count the number of groups p of contour probability maps whose pixel proportion is greater than or equal to the preset proportion; p is a positive integer less than or equal to q; if the pixel proportion is less than the preset proportion, it means that there are too few high-confidence edge areas and the contour of the scenic area facilities is not obvious.

[0095] Step A5: record the p groups of contour probability maps and the corresponding multi-channel data as pre-processed image information.

[0096] Step S6: transmitting the p groups of pre-processed image information to the cloud for constructing a three-dimensional structural model of subsequent scenic spot facilities.

[0097] The three-dimensional construction module constructs the scenic area facilities in three dimensions based on the pre-processed image information and combines the digital stretching technology to obtain the three-dimensional structural model of the scenic area facilities.

[0098] It should be noted that digital stretching technology is a method that uses computer algorithms to perform geometric transformation and morphological processing on three-dimensional point cloud data; the main purpose is to optimize the distribution and structure of point cloud data, enhance the detail performance of key areas, reduce noise and errors, and thus improve the quality and accuracy of the final three-dimensional model. In the present invention, digital stretching technology is embodied in the subsequent combined optimization of contour probability maps and point cloud data, which significantly improves the geometric accuracy and detail performance of the three-dimensional structural model of scenic area facilities; this technology not only reduces noise and errors, improves the accuracy and visual realism of the three-dimensional structural model, but also enhances the overall robustness and processing efficiency of the system.

[0099] The method for constructing the three-dimensional structural model of the scenic spot facilities includes:

[0100] Step Z1: Convert the depth map into point cloud data through a stereo matching algorithm to construct a geometric basis of the three-dimensional space; the stereo matching algorithm is an SGBM algorithm, etc.

[0101] Step Z2: Combine the contour probability map and point cloud data to optimize the geometric basis, enhance the accuracy of the contour of the scenic area facilities, and obtain the optimized geometric basis; use the contour probability map to optimize the point cloud to highlight key structures and edges, and improve the detail performance and accuracy of the three-dimensional model.

[0102] The method for optimizing the geometric basis includes: according to a preset probability threshold of the contour probability map, screening out the point cloud of the high probability area in the contour probability map for key optimization; the high probability area is an area composed of corresponding pixels whose probability values ​​of pixels in the contour probability map are greater than the preset probability threshold.

[0103] Step Z3, based on the optimized geometric foundation, generate a three-dimensional mesh model through point cloud filtering and mesh reconstruction algorithm; the role of the point cloud filtering is to remove noise and outliers through filtering, improve the quality of the point cloud, and is a prerequisite for generating a high-quality three-dimensional mesh model; the mesh reconstruction algorithm is PSR algorithm and AtlasNet, etc.

[0104] Step Z4: Through texture mapping technology, the first view, the second view, the third view, the fourth view, and the fifth view are integrated with the three-dimensional grid model to enhance the realism and detail expression of the three-dimensional grid model, and generate a three-dimensional structural model of the scenic area facilities; the three-dimensional structural model also includes corresponding multiple sets of ambient brightness, shooting angles, and shooting distances.

[0105] It should be noted that the three-dimensional structure model includes multiple corresponding sets of environmental brightness, shooting angles and shooting distances, which helps to analyze the impact of the environment on the quality of three-dimensional model construction and facilitates the subsequent expansion of the robustness and adaptability of the model.

[0106] The information generation module generates the planning layout information of the scenic area facilities based on the three-dimensional structural model; the planning layout information includes the structural type, size ratio and spatial position of the target scenic area facilities in the scenic area; it helps to improve the rationality of the scenic area's future expansion planning.

[0107] Example 2

[0108] See also Figure 2 As shown, this embodiment provides a low-altitude economical three-dimensional scene construction method combined with digital stretching technology, including:

[0109] Collect image information of scenic area facilities on the preset flight path;

[0110] Preprocessing the image information to obtain preprocessed image information;

[0111] Based on the pre-processed image information, the scenic area facilities are constructed in three dimensions in combination with digital stretching technology to obtain a three-dimensional structural model of the scenic area facilities;

[0112] Based on the three-dimensional structural model, the planning and layout information of scenic area facilities is generated.

[0113] Example 3

[0114] When using a five-eye camera for shooting, the multi-channel data obtained synchronously includes image channel data and sensor channel data; and the image channel data includes the first view, the second view, the third view, the fourth view, the fifth view and the depth map; due to the large volume of multi-channel data, it will occupy a large amount of transmission bandwidth consumption during data transmission, affecting the data transmission efficiency, and further affecting the three-dimensional construction efficiency. The low-altitude economic three-dimensional scene construction system combined with digital stretching technology also needs to perform data lightweight processing on the multi-channel data between step S1 and step S2;

[0115] The lightweight processing method comprises:

[0116] Step 1: For the image channel data, according to a preset compression ratio, the first view, the second view, the third view, the fourth view and the fifth view are subjected to lossy compression in the WebP format, so as to significantly reduce the file size while maintaining high image quality; and the depth map is converted into a 16-bit PNG format to reduce the pressure of data transmission, and the compressed image channel data is obtained;

[0117] Step 2: Apply a data compression algorithm to the sensor channel data to reduce the bandwidth required for data transmission, thereby obtaining compressed sensor channel data; the data compression algorithm may be Delta coding and Huffman coding.

[0118] Step 3: Replace the original multi-channel data with the compressed image channel data and sensor channel data for subsequent multi-channel data alignment processing at the same time in the data buffer.

[0119] By performing lightweight processing on multi-channel data, the transmission bandwidth consumption during data transmission is reduced, and the efficiency of 3D modeling is effectively improved.

[0120] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology, characterized in that: include: Collect image information of urban facilities along the preset flight path; Preprocessing the image information to obtain preprocessed image information; Based on the pre-processed image information, the urban facilities are constructed in three dimensions in combination with digital stretching technology to obtain a three-dimensional structural model of the urban facilities; Generate planning and layout information of urban facilities based on the three-dimensional structural model.

2. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 1, characterized in that: The image information is acquired by a five-eye camera device carried by the drone; the image information includes a first view, a second view, a third view, a fourth view, a fifth view, ambient brightness, a shooting angle, a shooting distance and a depth map; The pretreatment method comprises: Step S1, synchronously acquiring n groups of multi-channel data; the multi-channel data includes image channel data and sensor channel data; the image channel data includes a first view, a second view, a third view, a fourth view, a fifth view and a depth map; the sensor channel data includes ambient brightness, shooting angle and shooting distance; n is a positive integer; Step S2: Establish a data buffer area on the drone end and align the multi-channel data at the same time according to the timestamp; Step S3, based on n groups of multi-channel data, by comparing the ambient brightness with a preset brightness threshold, obtaining m groups of first processed data; m is a positive integer less than or equal to n; Step S4, based on the m groups of first processed data, by comparing the roll angle with a preset roll angle threshold, obtaining q groups of second processed data; q is a positive integer less than or equal to m; Step S5, performing clarity detection on the q groups of second processed data to obtain p groups of pre-processed image information; p is a positive integer less than or equal to q; Step S6: transmitting the p groups of pre-processed image information to the cloud.

3. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 2, characterized in that: The pre-processed image information is the processed image information and the corresponding contour probability map; The value of each pixel in the contour probability map represents the probability value of the pixel belonging to the actual contour; The method for clarity detection comprises: Step A1, removing random noise in the depth map corresponding to the q groups of second processed data; Step A2, calling a function in OpenCV to perform Laplace filtering on the image to obtain a Laplace gradient map; the function is cv::Laplacian() function; Step A3: input the Laplace gradient map into the pre-trained contour detection model and output the contour probability map; Step A4, calculating the proportion of pixels in the contour probability map that are greater than a preset probability threshold, and counting the number p of groups of contour probability maps whose pixel proportion is greater than or equal to the preset proportion; Step A5: record the p groups of contour probability maps and the corresponding multi-channel data as pre-processed image information.

4. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 3, characterized in that: The training method of the contour detection model includes: Step B1, obtaining a data set containing v sets of contour information from a sample set, wherein the contour information includes a Laplace gradient map and a corresponding contour probability map; Step B2, dividing the data set into a training set and a test set according to a preset ratio, constructing a classifier, and performing a binary classification on each pixel in the Laplace gradient image, wherein the binary classification includes whether it is a contour or not a contour; Step B3, using the Laplace gradient map in the training set as the input of the classifier, using the corresponding contour probability map in the training set as the output of the classifier, training the classifier to obtain an initial classifier; the classifier is a HED deep learning model; Step B4: Use the test set to test the initial classifier, and output the initial classifier that meets the preset accuracy as the final contour detection model.

5. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 1, characterized in that: The method for constructing the three-dimensional structural model of the urban facility comprises: Step Z1: convert the depth map into point cloud data through a stereo matching algorithm to construct a geometric basis of the three-dimensional space; the stereo matching algorithm is the SGBM algorithm; Step Z2, combining the contour probability map and the point cloud data, optimizing the geometric basis, and obtaining the optimized geometric basis; the method for optimizing the geometric basis includes: according to the contour probability map, presetting the probability threshold, screening out the point cloud of the high probability area in the contour probability map, and performing key optimization; the high probability area is an area composed of corresponding pixels whose probability values ​​of pixels in the contour probability map are greater than the preset probability threshold; Step Z3, based on the optimized geometric foundation, generate a three-dimensional mesh model through point cloud filtering and mesh reconstruction algorithm; Step Z4: By using texture mapping technology, the first view, the second view, the third view, the fourth view, and the fifth view are integrated with the three-dimensional grid model to generate a three-dimensional structural model of the urban facilities; the three-dimensional structural model also includes corresponding multiple groups of environmental brightness, shooting angles, and shooting distances.

6. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 2, characterized in that: The shooting distance is the distance between the five-eye camera and any point on the top surface of the urban facility when the drone takes the photo; the shooting distance is obtained by converting the product of the baseline length and the focal length into an absolute value and dividing it by the parallax; The method for calculating the distance from each pixel block to the five-eye camera in the depth map and the method for obtaining the shooting distance are both parallax methods, and the depth map is obtained by the five-eye camera carried by the drone.

7. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 2, characterized in that: The shooting angles include pitch angle, roll angle and yaw angle; The method for acquiring the second processed data includes: comparing the roll angle with a preset roll angle threshold; if the roll angle is greater than or equal to the preset roll angle threshold, discarding the corresponding first processed data; if the roll angle is less than the preset roll angle threshold, retaining the corresponding first processed data; counting the number of retained first processed data groups, and recording them as the number of second processed data groups q.

8. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 2, characterized in that: The method for obtaining the first processed data includes: Compare the ambient brightness with the preset brightness threshold; if the ambient brightness is greater than or equal to the preset brightness threshold, discard the corresponding multi-channel data; if the ambient brightness is less than the preset brightness threshold, retain the corresponding multi-channel data; count the number of retained multi-channel data groups and record them as the number m of groups of the first processed data.

9. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 1, characterized in that: The planning layout information includes the structure type, size ratio and spatial position of the target urban facilities in the city.

10. A system for constructing low-altitude economical three-dimensional scenes in combination with digital stretching technology, implementing a method for constructing low-altitude economical three-dimensional scenes in combination with digital stretching technology as described in any one of claims 1 to 9, characterized in that: include: A data acquisition module, used to collect image information of urban facilities on a preset flight path; An image processing module, used for preprocessing image information to obtain preprocessed image information; The three-dimensional construction module constructs the urban facilities in three dimensions based on the pre-processed image information and combines the digital stretching technology to obtain the three-dimensional structural model of the urban facilities; The information generation module generates the planning and layout information of urban facilities based on the three-dimensional structural model.

Images