Low-altitude economic three-dimensional scene construction method and system combined with digital stretching technology

Abstract

The present invention belongs to the technical field of three-dimensional modeling, and discloses a method and system for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology; the method comprises: collecting image information of urban facilities on a preset flight path; the image information is collected by a five-eye camera device carried by an unmanned aerial vehicle; the image information comprises a first view, a second view, a third view, a fourth view, a fifth view, ambient brightness, a shooting angle, a shooting distance and a depth map; the image information is preprocessed to obtain preprocessed image information; based on the preprocessed image information, the urban facilities are three-dimensionally constructed in combination with digital stretching technology to obtain a three-dimensional structural model of the urban facilities; based on the three-dimensional structural model, planning layout information of the urban facilities is generated; the accuracy of the overall layout planning during urban expansion is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional modeling technology, and more specifically, to a method and system for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology. Background Art

[0002] The existing Chinese patent application, publication number CN113793412A, discloses a method and system for 3D modeling of nuclear power plants based on 2D planar templates. This method addresses the current issues with complex, time-consuming, and error-prone modeling of nuclear power plants. The invention first simplifies the 2D planar template to form the outline of the bottom plane. This outline is then offset based on the floor height, and then stretched based on the floor height. This method completes the stretching of each floor plan, forming a base plate at the floor elevation. The base plate is then cut to generate the floor plate, and redundant geometric surfaces are deleted to generate a 3D geometric model of the nuclear power plant structure. This invention enables the acquisition of required basic data from the 2D planar template; integrates design drawings into the design simulation process to accurately reflect the actual state; and achieves automation through programming, with the potential for future use in digital collaborative design platforms for nuclear power structures.

[0003] The existing technology still has the following problems:

[0004] When cities expand, 3D structural models of various urban facilities provide a reasonable and accurate basis for layout planning. However, traditional 3D modeling methods, such as manual measurement, monocular vision, or 2D templates, often suffer from insufficient accuracy and lack of detail when modeling urban facilities, resulting in inaccurate overall urban layout planning.

[0005] In view of this, the present invention proposes a low-altitude economical three-dimensional scene construction method and system combined with digital stretching technology to solve the above problems. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology, comprising:

[0007] Collect image information of urban facilities along the preset flight path;

[0008] Preprocessing the image information to obtain preprocessed image information;

[0009] Based on the pre-processed image information, the urban facilities are constructed in three dimensions using digital stretching technology to obtain a three-dimensional structural model of the urban facilities.

[0010] Generate planning and layout information of urban facilities based on the three-dimensional structural model.

[0011] Furthermore, the image information is acquired by a five-eye camera device carried by a drone; the image information includes a first view, a second view, a third view, a fourth view, a fifth view, ambient brightness, a shooting angle, a shooting distance, and a depth map;

[0012] The pretreatment method comprises:

[0013] Step S1, synchronously acquiring n sets of multi-channel data; the multi-channel data includes image channel data and sensor channel data; the image channel data includes a first view, a second view, a third view, a fourth view, a fifth view and a depth map; the sensor channel data includes ambient brightness, shooting angle and shooting distance; n is a positive integer;

[0014] Step S2: Establish a data buffer on the UAV side and align the multi-channel data at the same time according to the timestamp;

[0015] Step S3: Based on n sets of multi-channel data, obtain m sets of first processed data by comparing the ambient brightness with a preset brightness threshold; m is a positive integer less than or equal to n;

[0016] Step S4: Based on the m sets of first processed data, obtain q sets of second processed data by comparing the roll angle with a preset roll angle threshold; q is a positive integer less than or equal to m;

[0017] Step S5: Perform clarity detection on the q groups of second processed data to obtain p groups of pre-processed image information; p is a positive integer less than or equal to q;

[0018] Step S6: Transmit the p groups of pre-processed image information to the cloud.

[0019] Furthermore, the pre-processed image information is the processed image information and the corresponding contour probability map; the value of each pixel in the contour probability map represents the probability value of the pixel belonging to the actual contour;

[0020] The method for clarity detection comprises:

[0021] Step A1: removing random noise in the depth map corresponding to the q groups of second processed data;

[0022] Step A2: Call a function in OpenCV to perform Laplace filtering on the image to obtain a Laplace gradient map; the function is cv::Laplace() function;

[0023] Step A3: input the Laplace gradient map into the pre-trained contour detection model and output a contour probability map;

[0024] Step A4: Calculate the proportion of pixels in the contour probability map that are greater than a preset probability threshold, and count the number p of groups of contour probability maps whose pixel proportion is greater than or equal to the preset proportion;

[0025] Step A5: record the p groups of contour probability maps and the corresponding multi-channel data as pre-processed image information.

[0026] Furthermore, the training method of the contour detection model includes:

[0027] Step B1: obtaining a data set containing v sets of contour information from a sample set, wherein the contour information includes a Laplace gradient map and a corresponding contour probability map;

[0028] Step B2: Divide the data set into a training set and a test set according to a preset ratio, construct a classifier, and perform a binary classification on each pixel in the Laplace gradient image, wherein the binary classification includes whether it is a contour or not a contour;

[0029] Step B3: Using the Laplace gradient map in the training set as the input of the classifier and the corresponding contour probability map in the training set as the output of the classifier, the classifier is trained to obtain an initial classifier; the classifier is a HED deep learning model;

[0030] Step B4: Use the test set to test the initial classifier, and output the initial classifier that meets the preset accuracy as the final contour detection model.

[0031] Furthermore, the method for constructing the three-dimensional structural model of the urban facilities includes:

[0032] Step Z1: convert the depth map into point cloud data through a stereo matching algorithm to construct a geometric basis for the three-dimensional space; the stereo matching algorithm is the SGBM algorithm;

[0033] Step Z2: combining the contour probability map and the point cloud data to optimize the geometric basis and obtain an optimized geometric basis; the method for optimizing the geometric basis includes: presetting a probability threshold in the contour probability map, screening out point clouds in high-probability areas in the contour probability map, and performing focused optimization; the high-probability area is an area consisting of corresponding pixels in the contour probability map whose probability values are greater than the preset probability threshold;

[0034] Step Z3: Based on the optimized geometric foundation, a three-dimensional mesh model is generated through point cloud filtering and mesh reconstruction algorithm;

[0035] Step Z4: Using texture mapping technology, the first view, the second view, the third view, the fourth view, and the fifth view are fused with the three-dimensional grid model to generate a three-dimensional structural model of the urban facility; the three-dimensional structural model also includes corresponding multiple sets of ambient brightness, shooting angles, and shooting distances.

[0036] Furthermore, the shooting distance is the distance between the five-eye camera and any point on the top surface of the urban facility when the drone takes the photo; the shooting distance is obtained by converting the product of the baseline length and the focal length into an absolute value divided by the parallax;

[0037] The method for calculating the distance from each pixel block to the five-eye camera in the depth map and the method for obtaining the shooting distance are both parallax methods, and the depth map is obtained by the five-eye camera carried by the drone.

[0038] Furthermore, the shooting angle includes pitch angle, roll angle and yaw angle;

[0039] The method for obtaining the second processed data includes: comparing the roll angle with a preset roll angle threshold; if the roll angle is greater than or equal to the preset roll angle threshold, discarding the corresponding first processed data; if the roll angle is less than the preset roll angle threshold, retaining the corresponding first processed data; counting the number of retained first processed data groups and recording it as the number of second processed data groups q.

[0040] Furthermore, the method for obtaining the first processed data includes:

[0041] Compare the ambient brightness with the preset brightness threshold; if the ambient brightness is greater than or equal to the preset brightness threshold, discard the corresponding multi-channel data; if the ambient brightness is less than the preset brightness threshold, retain the corresponding multi-channel data; count the number of retained multi-channel data groups and record it as the number m of groups of the first processed data.

[0042] Furthermore, the planning layout information includes the structure type, size ratio and spatial location of the target urban facilities in the city.

[0043] A low-altitude economical three-dimensional scene construction system combining digital stretching technology, and a method for constructing a low-altitude economic three-dimensional scene combining digital stretching technology, comprising:

[0044] A data acquisition module is used to collect image information of urban facilities along a preset flight path;

[0045] An image processing module is used to preprocess image information to obtain preprocessed image information;

[0046] The 3D construction module constructs urban facilities in 3D based on pre-processed image information and combines digital stretching technology to obtain a 3D structural model of urban facilities;

[0047] The information generation module generates the planning and layout information of urban facilities based on the three-dimensional structural model.

[0048] The technical effects and advantages of the low-altitude economical three-dimensional scene construction method and system combined with digital stretching technology of the present invention are as follows:

[0049] The present invention uses a five-eye camera mounted on an unmanned aerial vehicle to collect image information of urban facilities, and then preprocesses the image information to obtain preprocessed image information; then, based on the preprocessed image information, the urban facilities are three-dimensionally constructed in combination with digital stretching technology to obtain a three-dimensional structural model of the urban facilities; finally, the planning layout information of the urban facilities is generated through the high-precision three-dimensional structural model of the urban facilities, effectively improving the accuracy of the overall layout planning during urban expansion.

[0050] This invention combines contour probability maps and point cloud data to optimize the geometric basis, which not only significantly improves the edge accuracy and detail expression of the three-dimensional structural model of urban facilities, but also effectively reduces noise and errors, and enhances the geometric accuracy and visual realism of the three-dimensional structural model.

[0051] In the process of image information preprocessing, the present invention adopts multiple processing steps. First, the first processed data is obtained based on the comparison of the ambient brightness, which helps to improve the overall quality of the image information; secondly, the second processed data is obtained based on the comparison of the roll angle, which reduces the geometric distortion in the image and ensures the accuracy of the alignment of multiple views; finally, the second processed data is subjected to clarity detection to obtain preprocessed image information, ensuring that subsequent edge detection and three-dimensional reconstruction are based on high-quality image data. This process reduces the negative impact of blurred images on depth estimation and edge detection, and improves the detail expression and accuracy of the three-dimensional model. In addition, the multiple screening steps reduce the system's tolerance for low-quality or abnormal data, ensure that the data input to the three-dimensional modeling module is of higher quality, and reduce errors or distortions caused by data problems in the subsequent modeling process.

[0052] The present invention inputs the Laplacian gradient map into a pre-trained contour detection model (such as HED) and outputs a high-quality contour probability map, which embodies the organic combination of classical image processing methods and modern deep learning technology. Through this combination, not only the accuracy and detail expression of contour detection are improved, but also more solid data support is provided for the subsequent construction of three-dimensional models of urban facilities, ensuring the high precision and quality of the final model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a schematic diagram of a low-altitude economical three-dimensional scene construction system incorporating digital stretching technology according to Example 1 of the present invention;

[0054] Figure 2 This is a flow chart of a method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to Example 2 of the present invention;

[0055] Figure 3 This is a flow chart of the pretreatment method of Example 1 of the present invention;

[0056] Figure 4 This is a flow chart of the method for clarity detection according to Example 1 of the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] Example 1

[0059] See also Figure 1 As shown, the low-altitude economic three-dimensional scene construction system combined with digital stretching technology described in this embodiment includes: a data acquisition module, an image processing module, a three-dimensional construction module and an information generation module, and each module is connected by wired and / or wireless connections.

[0060] The urban facilities include scenic area facilities, residential facilities and transportation facilities; the scenic area facilities include observation decks and aerial plank roads.

[0061] The data acquisition module is used to collect image information of scenic area facilities on a preset flight path; the image information is collected by the five-eye camera device carried by the drone; the image information includes the first view, second view, third view, fourth view, fifth view, ambient brightness, shooting angle, shooting distance and depth map; the shooting angle includes pitch angle, roll angle and yaw angle.

[0062] It should be noted that the five lenses of the five-eye camera are arranged in a linear layout, and the five cameras are arranged along a straight line with the same baseline length.

[0063] The first view, second view, third view, fourth view and fifth view are captured by the five lenses of the pentacular camera. Adjacent lenses capture the same scene through a fixed baseline to form parallax, which serves as the basis for calculating the depth values of pixel points in the subsequent depth map. At the same time, multiple views enable the system to construct a more detailed three-dimensional model, which helps to improve the accuracy of the subsequent model difference rate.

[0064] The ambient brightness affects the contrast and clarity when photographing scenic area facilities, thereby affecting the accuracy of subsequent model difference rate calculation; the shooting angle is used for geometric correction of multiple views of scenic area facilities; the shooting distance determines the size and proportion of multiple views of scenic area facilities. At the same time, a shooting distance that is too far will reduce the accuracy of the depth map of the scenic area facilities, thereby affecting the three-dimensional reconstruction effect of the scenic area facilities.

[0065] The pitch angle is the rotation angle of the pentacular camera around the horizontal axis, indicating the upward or downward tilt of the pentacular camera, which will affect the accuracy when acquiring multiple views; the roll angle is the rotation angle of the pentacular camera around the vertical axis, indicating the left and right tilt of the pentacular camera. Excessive roll angle will change the shooting distance and thus affect the accuracy of the depth map; the yaw angle is the rotation angle of the pentacular camera around the vertical axis, indicating the left and right rotation of the pentacular camera.

[0066] The first view, second view, third view, fourth view and fifth view are all obtained by shooting with the five cameras carried by the drone, and the shooting interval is preset to j seconds, where j can be any positive number; the ambient brightness is obtained by the ambient light sensor equipped with the five cameras carried by the drone; and the shooting angle is obtained by the inertial measurement unit (IMU) sensor on the drone.

[0067] It should be noted that when the drone is flying and shooting, the five cameras are fixed on the drone. At this time, the shooting angle is the attitude angle of the drone, which can be obtained through the inertial measurement unit sensor of the drone.

[0068] The shooting distance is the distance between the five-eye camera and any point on the top surface of the scenic area facilities when the drone takes a photo; the shooting distance D calculation method includes:

[0069]

[0070] Wherein, D is the shooting distance; L is the baseline length of the pentacular camera, which is the preset fixed distance between any two adjacent lenses in the pentacular camera; f is the preset focal length of any two adjacent lenses in the pentacular camera; d1 and d2 are the horizontal coordinate positions of the scenic area facilities in the views taken by any two adjacent lenses, respectively; d1-d2 is the parallax, which is the difference in the horizontal coordinate positions of the scenic area facilities in the views taken by any two adjacent lenses.

[0071] The method for calculating the distance from each pixel block to the five-eye camera in the depth map and the method for obtaining the shooting distance are both parallax methods, and the depth map is obtained by the five-eye camera carried by the drone.

[0072] The image processing module is used to preprocess the image information to obtain preprocessed image information; the preprocessed image information is the processed image information and the corresponding contour probability map.

[0073] See also Figure 3 As shown, the preprocessing method includes:

[0074] Step S1, synchronously acquire n sets of multi-channel data; the multi-channel data includes image channel data and sensor channel data; the image channel data includes a first view, a second view, a third view, a fourth view, a fifth view and a depth map; the sensor channel data includes ambient brightness, shooting angle and shooting distance; n is a positive integer.

[0075] Step S2: Establish a data buffer area on the drone side to align the multi-channel data at the same time according to the timestamp to avoid errors in subsequent processing due to timing misalignment.

[0076] Step S3: Based on the n sets of multi-channel data, m sets of first processed data are obtained. The method for obtaining the first processed data includes:

[0077] Compare the ambient brightness with the preset brightness threshold; if the ambient brightness is greater than or equal to the preset brightness threshold, such as strong direct sunlight, discard the corresponding multi-channel data; if the ambient brightness is less than the preset brightness threshold, retain the corresponding multi-channel data; count the number of retained multi-channel data groups and record them as the number m of groups of the first processed data; m is a positive integer less than or equal to n.

[0078] Step S4: Based on the m groups of first processed data, obtain q groups of second processed data; the method for obtaining the second processed data includes:

[0079] Compare the roll angle with a preset roll angle threshold; if the roll angle is greater than or equal to the preset roll angle threshold, resulting in distortion of the image channel data, the corresponding first processed data is discarded; if the roll angle is less than the preset roll angle threshold, the corresponding first processed data is retained; count the number of retained first processed data groups and record it as the number of second processed data groups q, where q is a positive integer less than or equal to m.

[0080] Step S5: Perform clarity detection on the q groups of second processed data to obtain p groups of pre-processed image information.

[0081] See also Figure 4 As shown, the method for clarity detection includes:

[0082] Step A1: remove random noise in the depth map corresponding to the q groups of second processed data.

[0083] Step A2: Call a function in OpenCV to perform Laplacian filtering on the image to obtain a Laplacian gradient map; the function is the cv::Laplacian() function; the pixel values in the Laplacian gradient map represent the intensity of local grayscale changes in the Laplacian gradient map. The larger the pixel value, the more prominent the local edge in the Laplacian gradient map, and the clearer the Laplacian gradient map; if the pixel values in the Laplacian gradient map are all small, the Laplacian gradient map is relatively blurred.

[0084] Step A3: input the Laplacian gradient map into a pre-trained contour detection model, and output a contour probability map; the value of each pixel in the contour probability map represents the probability value of the pixel belonging to the actual contour.

[0085] It should be noted that the contour probability map is a form of image representation generated by a machine learning model in image processing and computer vision.

[0086] It should be noted that the probability value is between 0 and 1, or mapped to a grayscale value range of 0 to 255 through appropriate scaling for visualization; the higher the value of a pixel, the more likely the pixel is to be located at a contour position; the lower the value of a pixel, the less likely the pixel is to be part of a contour.

[0087] The training method of the contour detection model includes:

[0088] Step B1: obtaining a data set containing v sets of contour information from a sample set, wherein the contour information includes a Laplace gradient map and a corresponding contour probability map;

[0089] Step B2: Divide the data set into a training set and a test set according to a preset ratio, construct a classifier, and perform a binary classification on each pixel in the Laplace gradient image, wherein the binary classification includes whether it is a contour or not a contour;

[0090] Step B3: Using the Laplace gradient map in the training set as the input of the classifier and the corresponding contour probability map in the training set as the output of the classifier, the classifier is trained to obtain an initial classifier; the classifier is a HED deep learning model;

[0091] Step B4: Use the test set to test the initial classifier, and output the initial classifier that meets the preset accuracy as the final contour detection model.

[0092] It should be noted that the Laplace gradient map and the corresponding contour probability map have a one-to-one correspondence in spatial coordinates.

[0093] It should be noted that the HED deep learning model is a type of deep network specifically for edge detection. It can learn and fuse edge information at multiple scales to generate high-quality edge maps. Using the Laplacian gradient map as the input of the contour detection model makes it easier for HED networks to identify true contours and ignore noise and unclear gradient areas.

[0094] Step A4: Calculate the proportion of pixels in the contour probability map that are greater than a preset probability threshold, and count the number p of groups of contour probability maps whose pixel proportion is greater than or equal to the preset proportion; p is a positive integer less than or equal to q; if the pixel proportion is less than the preset proportion, it means that there are too few high-confidence edge areas and the contours of the scenic area facilities are not clear.

[0095] Step A5: record the p groups of contour probability maps and the corresponding multi-channel data as pre-processed image information.

[0096] Step S6: transmit the p groups of pre-processed image information to the cloud for constructing a subsequent three-dimensional structural model of the scenic area facilities.

[0097] The three-dimensional construction module constructs the scenic area facilities in three dimensions based on the pre-processed image information and combines the digital stretching technology to obtain the three-dimensional structural model of the scenic area facilities.

[0098] It should be noted that digital stretching technology is a method that uses computer algorithms to perform geometric transformations and morphological processing on three-dimensional point cloud data. Its primary purpose is to optimize the distribution and structure of point cloud data, enhance the detail of key areas, reduce noise and errors, and thus improve the quality and accuracy of the final three-dimensional model. In this invention, digital stretching technology is embodied in the subsequent optimization of the combined contour probability map and point cloud data, significantly improving the geometric accuracy and detail of the three-dimensional structural model of scenic area facilities. This technology not only reduces noise and errors, improves the accuracy and visual realism of the three-dimensional structural model, but also enhances the overall robustness and processing efficiency of the system.

[0099] The method for constructing the three-dimensional structural model of the scenic area facilities includes:

[0100] Step Z1: Convert the depth map into point cloud data through a stereo matching algorithm to construct a geometric basis for the three-dimensional space; the stereo matching algorithm may be an SGBM algorithm or the like.

[0101] Step Z2: Combine the contour probability map and point cloud data to optimize the geometric basis, enhance the accuracy of the scenic area facility contours, and obtain the optimized geometric basis; use the contour probability map to optimize the point cloud to highlight key structures and edges, and improve the detail performance and accuracy of the three-dimensional model.

[0102] The method for optimizing the geometric basis includes: presetting a probability threshold according to the contour probability map, screening out the point cloud in the high probability area of the contour probability map, and performing key optimization; the high probability area is an area composed of corresponding pixels whose probability values of pixels in the contour probability map are greater than the preset probability threshold.

[0103] Step Z3: Based on the optimized geometric foundation, a three-dimensional mesh model is generated through point cloud filtering and mesh reconstruction algorithms. The function of the point cloud filtering is to remove noise and outliers through filtering, thereby improving the quality of the point cloud and being a prerequisite for generating a high-quality three-dimensional mesh model. The mesh reconstruction algorithms include the PSR algorithm and AtlasNet.

[0104] Step Z4: Using texture mapping technology, the first view, the second view, the third view, the fourth view, and the fifth view are integrated with the three-dimensional mesh model to enhance the realism and detail of the three-dimensional mesh model and generate a three-dimensional structural model of the scenic area facilities; the three-dimensional structural model also includes corresponding multiple sets of ambient brightness, shooting angles, and shooting distances.

[0105] It should be noted that the 3D structural model includes multiple corresponding sets of environmental brightness, shooting angles, and shooting distances, which helps to analyze the impact of the environment on the quality of 3D model construction and facilitates the subsequent expansion of the model's robustness and adaptability.

[0106] The information generation module generates the planning layout information of the scenic area facilities based on the three-dimensional structural model; the planning layout information includes the structural type, size ratio and spatial location of the target scenic area facilities in the scenic area, which helps to improve the rationality of the scenic area's future expansion planning.

[0107] Example 2

[0108] See also Figure 2 As shown, this embodiment provides a method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology, including:

[0109] Collect image information of scenic area facilities along the preset flight path;

[0110] Preprocessing the image information to obtain preprocessed image information;

[0111] Based on the pre-processed image information, the scenic area facilities are constructed in three dimensions in combination with digital stretching technology to obtain a three-dimensional structural model of the scenic area facilities;

[0112] Based on the three-dimensional structural model, the planning layout information of the scenic area facilities is generated.

[0113] Example 3

[0114] When shooting with a five-eye camera, the multi-channel data acquired synchronously includes image channel data and sensor channel data; the image channel data includes the first view, second view, third view, fourth view, fifth view, and depth map; due to the large volume of multi-channel data, it consumes a large amount of transmission bandwidth during data transmission, affecting data transmission efficiency and, in turn, 3D construction efficiency. The low-altitude economic 3D scene construction system combined with digital stretching technology also needs to perform data lightweight processing on the multi-channel data between steps S1 and S2;

[0115] The lightweight processing method includes:

[0116] Step 1: For the image channel data, use the WebP format for lossy compression at a preset compression ratio for the first, second, third, fourth, and fifth views, significantly reducing the file size while maintaining high image quality. Convert the depth map to 16-bit PNG format to reduce data transmission pressure, and obtain the compressed image channel data.

[0117] Step 2: Apply a data compression algorithm to the sensor channel data to reduce the bandwidth required for data transmission, thereby obtaining compressed sensor channel data; the data compression algorithm may be Delta coding and Huffman coding.

[0118] Step 3: Replace the original multi-channel data with the compressed image channel data and sensor channel data for subsequent multi-channel data alignment processing at the same time in the data buffer.

[0119] By performing lightweight processing on multi-channel data, the transmission bandwidth consumption during data transmission is reduced, and the efficiency of 3D modeling is effectively improved.

[0120] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A low-altitude economical three-dimensional scene construction method combining digital stretching technology is characterized by: include: Collect image information of urban facilities along the preset flight path using the five-lens camera equipment carried by the drone; Preprocess the image information to obtain preprocessed image information: Step S1, synchronously acquiring n sets of multi-channel data; n is a positive integer; Step S2: Establish a data buffer on the UAV side and align the multi-channel data at the same time according to the timestamp; Step S3: Based on n sets of multi-channel data, obtain m sets of first processed data by comparing the ambient brightness with a preset brightness threshold; m is a positive integer less than or equal to n; Step S4: Based on the m sets of first processed data, obtain q sets of second processed data by comparing the roll angle with a preset roll angle threshold; q is a positive integer less than or equal to m; Step S5: Perform clarity detection on the q groups of second processed data: Step A1: removing random noise in the depth map corresponding to the q groups of second processed data; Step A2: Call a function in OpenCV to perform Laplacian filtering on the image to obtain a Laplacian gradient map; the function is cv::Laplacian() function; Step A3: input the Laplace gradient map into the pre-trained contour detection model and output a contour probability map; Step A4: Calculate the proportion of pixels in the contour probability map that are greater than a preset probability threshold, and count the number p of groups of contour probability maps whose pixel proportion is greater than or equal to the preset proportion; Step A5: record the p groups of contour probability maps and corresponding multi-channel data as pre-processed image information; Obtain p groups of pre-processed image information; p is a positive integer less than or equal to q; Step S6: transmitting the p groups of pre-processed image information to the cloud; Based on the pre-processed image information, the urban facilities are constructed in three dimensions using digital stretching technology to obtain a three-dimensional structural model of the urban facilities: Step Z1: convert the depth map into point cloud data through a stereo matching algorithm to construct a geometric basis for the three-dimensional space; the stereo matching algorithm is the SGBM algorithm; Step Z2: combining the contour probability map and the point cloud data to optimize the geometric basis and obtain an optimized geometric basis; The method for optimizing the geometric basis includes: presetting a probability threshold in the contour probability map, screening out a point cloud in a high-probability area in the contour probability map, and performing focused optimization; the high-probability area is an area consisting of corresponding pixels in the contour probability map whose probability values are greater than the preset probability threshold; Step Z3: Based on the optimized geometric foundation, a three-dimensional mesh model is generated through point cloud filtering and mesh reconstruction algorithm; Step Z4: Using texture mapping technology, the first view, the second view, the third view, the fourth view, and the fifth view are fused with the three-dimensional mesh model to generate a three-dimensional structural model of the urban facility; the three-dimensional structural model also includes corresponding multiple sets of ambient brightness, shooting angles, and shooting distances; Based on the three-dimensional structural model, the planning layout information of urban facilities is generated, including the structural type, size ratio and spatial location of the target urban facilities in the city.

2. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 1, characterized in that: The image information includes a first view, a second view, a third view, a fourth view, a fifth view, ambient brightness, a shooting angle, a shooting distance, and a depth map; The multi-channel data includes image channel data and sensor channel data; The image channel data includes a first view, a second view, a third view, a fourth view, a fifth view and a depth map; the sensor channel data includes ambient brightness, a shooting angle and a shooting distance.

3. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 2, characterized in that: The pre-processed image information is the processed image information and the corresponding contour probability map; the value of each pixel in the contour probability map represents the probability value of the pixel belonging to the actual contour.

4. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 3, characterized in that: The training method of the contour detection model includes: Step B1: obtaining a data set containing v sets of contour information from a sample set, wherein the contour information includes a Laplace gradient map and a corresponding contour probability map; Step B2: Divide the data set into a training set and a test set according to a preset ratio, construct a classifier, and perform a binary classification on each pixel in the Laplace gradient image, wherein the binary classification includes whether it is a contour or not a contour; Step B3: Using the Laplace gradient map in the training set as the input of the classifier and the corresponding contour probability map in the training set as the output of the classifier, the classifier is trained to obtain an initial classifier; the classifier is a HED deep learning model; Step B4: Use the test set to test the initial classifier, and output the initial classifier that meets the preset accuracy as the final contour detection model.

5. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 2, characterized in that: The shooting distance is the distance between the five-eye camera and any point on the top surface of the urban facility when the drone takes the photo; the shooting distance is obtained by converting the product of the baseline length and the focal length into an absolute value divided by the parallax; The method for calculating the distance from each pixel block to the five-eye camera in the depth map and the method for obtaining the shooting distance are both parallax methods, and the depth map is obtained by the five-eye camera carried by the drone.

6. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 2, characterized in that: The shooting angles include pitch angle, roll angle and yaw angle; The method for obtaining the second processed data includes: comparing the roll angle with a preset roll angle threshold; if the roll angle is greater than or equal to the preset roll angle threshold, discarding the corresponding first processed data; if the roll angle is less than the preset roll angle threshold, retaining the corresponding first processed data; counting the number of retained first processed data groups and recording it as the number of second processed data groups q.

7. The method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology according to claim 1, characterized in that: The method for obtaining the first processed data includes: Compare the ambient brightness with the preset brightness threshold; if the ambient brightness is greater than or equal to the preset brightness threshold, discard the corresponding multi-channel data; if the ambient brightness is less than the preset brightness threshold, retain the corresponding multi-channel data; count the number of retained multi-channel data groups and record it as the number m of groups of the first processed data.

8. A system for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology, implementing the method for constructing a low-altitude economical three-dimensional scene in combination with digital stretching technology as described in any one of claims 1 to 7, characterized in that: include: The data acquisition module is used to collect image information of urban facilities along the preset flight path, which is obtained by the five cameras carried by the drone; The image processing module is used to preprocess the image information and obtain preprocessed image information: Step S1, synchronously acquiring n sets of multi-channel data; n is a positive integer; Step S2: Establish a data buffer on the UAV side and align the multi-channel data at the same time according to the timestamp; Step S3: Based on n sets of multi-channel data, obtain m sets of first processed data by comparing the ambient brightness with a preset brightness threshold; m is a positive integer less than or equal to n; Step S4: Based on the m sets of first processed data, obtain q sets of second processed data by comparing the roll angle with a preset roll angle threshold; q is a positive integer less than or equal to m; Step S5: Perform clarity detection on the q groups of second processed data: Step A1: removing random noise in the depth map corresponding to the q groups of second processed data; Step A2: Call a function in OpenCV to perform Laplacian filtering on the image to obtain a Laplacian gradient map; the function is cv::Laplacian() function; Step A3: input the Laplace gradient map into the pre-trained contour detection model and output a contour probability map; Step A4: Calculate the proportion of pixels in the contour probability map that are greater than a preset probability threshold, and count the number p of groups of contour probability maps whose pixel proportion is greater than or equal to the preset proportion; Step A5: record the p groups of contour probability maps and corresponding multi-channel data as pre-processed image information; Obtain p groups of pre-processed image information; p is a positive integer less than or equal to q; Step S6: transmitting the p groups of pre-processed image information to the cloud; The 3D construction module constructs urban facilities in 3D based on pre-processed image information and combines digital stretching technology to obtain a 3D structural model of urban facilities: Step Z1: convert the depth map into point cloud data through a stereo matching algorithm to construct a geometric basis for the three-dimensional space; the stereo matching algorithm is the SGBM algorithm; Step Z2: combining the contour probability map and the point cloud data to optimize the geometric basis and obtain an optimized geometric basis; The method for optimizing the geometric basis includes: presetting a probability threshold in the contour probability map, screening out a point cloud in a high-probability area in the contour probability map, and performing focused optimization; the high-probability area is an area consisting of corresponding pixels in the contour probability map whose probability values are greater than the preset probability threshold; Step Z3: Based on the optimized geometric foundation, a three-dimensional mesh model is generated through point cloud filtering and mesh reconstruction algorithm; Step Z4: Using texture mapping technology, the first view, the second view, the third view, the fourth view, and the fifth view are fused with the three-dimensional mesh model to generate a three-dimensional structural model of the urban facility; the three-dimensional structural model also includes corresponding multiple sets of ambient brightness, shooting angles, and shooting distances; The information generation module generates the planning layout information of urban facilities based on the three-dimensional structural model, including the structural type, size ratio and spatial location of the target urban facilities in the city.

Images