A new building detection method based on unmanned aerial vehicle aerial image

Abstract

The application discloses a new house detection method based on an unmanned aerial vehicle aerial image and belongs to the technical field of digital image processing.The application solves the problem that the existing method has low detection accuracy in house new construction determination due to the high similarity in appearance between new houses and non-new houses, the lack of new house samples and the positioning error of the unmanned aerial vehicle.The application introduces a deep learning model to predict and correct the positioning error in the house positioning process, and through coordinate registration and error correction technology, in combination with the geometric characteristics of the house, the IoU value between the house during the inspection and the existing house is used to realize high-precision house new construction determination, which can effectively reduce the deviation caused by the positioning error of the unmanned aerial vehicle, significantly improve the accuracy of the house new construction determination, and effectively avoid the problem that the accuracy of the identification method relying on the model alone is affected by the appearance similarity between the new house and the non-new house and the lack of new house samples.The application can be applied to new house detection.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology, specifically relating to a method for detecting newly built houses based on drone aerial images. Background Technology

[0002] With the acceleration of urbanization, the supervision of illegal buildings and newly constructed houses has become increasingly prominent. Traditional methods for inspecting new houses typically rely on manual inspections or static satellite imagery. These traditional methods suffer from limited coverage, low efficiency, and poor adaptability to environmental changes. Manual inspections not only require a large amount of human resources but are also easily affected by geographical environment, time, and weather factors, making it difficult to guarantee the timeliness and accuracy of inspection results. While satellite imagery can provide wide-area coverage, its low resolution and long update cycle often fail to reflect the dynamic changes of newly constructed houses in a timely manner. Drones, on the other hand, can efficiently collect aerial photographs of large areas in a short time, thereby acquiring high-resolution image data. Therefore, with the rapid development of drone technology, drone inspection has become a new trend.

[0003] Existing image target recognition methods employ models such as R-CNN and YOLO. While these methods have achieved some success, the high degree of similarity between newly built and existing buildings, coupled with a scarcity of newly built building samples, often leads to significant errors in distinguishing them. This limits the ability to determine a building's newness solely through image recognition. Furthermore, the rectangular shape of buildings and the positioning errors of drones result in significant shortcomings in the accuracy of identifying newly built buildings based on location, especially between inspection cycles. Due to drone positioning errors, relying solely on the building's center point often leads to misjudgments.

[0004] In summary, due to the high similarity in appearance between newly built and non-newly built houses, the scarcity of new house samples, and the impact of UAV positioning errors, existing methods still suffer from low detection accuracy in determining whether a house is newly built. Therefore, improving the accuracy and robustness of detection methods has become a key challenge in current new house detection technology. Summary of the Invention

[0005] This invention aims to address the problem that existing methods have low accuracy in determining the newness of buildings due to the high similarity in appearance between newly built and non-newly built buildings, the scarcity of new building samples, and the positioning errors of drones. Therefore, this invention proposes a new building detection method based on drone aerial images.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a method for detecting newly built houses based on UAV aerial images, the method specifically including the following steps:

[0007] Step 1: Control the drone to collect images of the area to be inspected, and use the built-in RTK device on the drone to simultaneously collect the drone's real-time flight parameters, real-time attitude data and real-time geographical location information; and obtain DEM data of the area to be inspected.

[0008] The images captured by the drone are used as input to the YOLOv8 model. The YOLOv8 model outputs the positions of the house targets in the images, that is, the initial pixel coordinates corresponding to the four vertices of each house target are obtained respectively.

[0009] Step 2: Based on the UAV's real-time flight parameters, real-time attitude data, and geographic location information, construct a mapping from the initial pixel coordinates of the house target vertices to the actual latitude and longitude of the house target vertices. Based on the established mapping, map the initial pixel coordinates of the house target vertices to the three-dimensional geographic coordinate system to obtain the actual latitude and longitude coordinates of the four vertices of each house target.

[0010] Step 3: Transform the actual latitude and longitude coordinates of the four vertices of each house target to the UTM coordinate system to obtain the coordinates of the vertices before registration in the UTM coordinate system, and use the neural network registration model to output the coordinate offset of the four vertices of each house target in the UTM coordinate system.

[0011] Step 4: Based on the output of the neural network registration model, register the coordinates of the four vertices of each house target in the UTM coordinate system to obtain the registered coordinates of the four vertices of each house target in the UTM coordinate system.

[0012] Historical geographic information data of houses is obtained from the DEM data of the area to be inspected. New houses are identified based on the historical geographic information data and the registered coordinates. The number of houses identified as new houses is recorded and the real geographic coordinates of the new houses are output.

[0013] The beneficial effects of this invention are:

[0014] This invention proposes a method for detecting newly built houses based on UAV aerial images. Unlike traditional image recognition methods that rely solely on the center point of a house to determine whether it is newly built, this invention introduces a deep learning model during the house positioning process to predict and correct positioning errors. By combining coordinate registration and error correction techniques with the geometric characteristics of the house, and utilizing the IoU value between the inspected house and existing houses, it achieves high-precision house construction determination. This effectively reduces the deviation caused by UAV positioning errors, significantly improves the accuracy of house construction determination, and effectively avoids the problems of detection accuracy being affected by the similarity between newly built and non-newly built houses and the scarcity of newly built house samples, which are common with recognition methods that rely solely on models. This avoids misjudgment and missed judgment, providing urban management departments with a more reliable and efficient house detection solution.

[0015] Moreover, the method of the present invention improves the adaptability under different environmental conditions. Errors caused by UAV flight paths, weather changes or other external factors can be effectively compensated by the model of the present invention, ensuring the consistency and stability of the detection results. Attached Figure Description

[0016] Figure 1 This is a flowchart of a new building detection method based on UAV aerial images according to the present invention;

[0017] Figure 2 This is a flowchart of the method for converting pixel coordinates to actual latitude and longitude coordinates;

[0018] Figure 3 This is a flowchart of the method for determining whether a building is newly constructed; Detailed Implementation

[0019] Before commencing the detection process using the method of this invention, it is necessary to determine the specific geographical area for image acquisition based on the inspection requirements, and to determine the drone's flight path, altitude, speed, and image overlap rate. It is also crucial to ensure that the drone uses a frontal, horizontal angle for image acquisition to guarantee that the acquired data is clear and unobstructed.

[0020] Specific Implementation Method 1: Combination Figure 1 and Figure 3 This embodiment describes a method for detecting newly constructed buildings based on UAV aerial images. The method specifically includes the following steps:

[0021] Step 1: Operate the drone to collect images of the area to be inspected, and use the RTK (Real-time kinematic) equipment built into the drone to simultaneously collect the drone's real-time flight parameters, real-time attitude data and real-time geographical location information; and obtain DEM (Digital Elevation Model) data collected by surveying tools in the area to be inspected.

[0022] It should be noted that the building geographic information data obtained from DEM data is used as the historical building geographic information data of the area to be inspected. This data does not change with the inspection time, and future inspections will be based on historical DEM data.

[0023] The images captured by the drone are used as input to the YOLOv8 model. The YOLOv8 model outputs the position of the house target in the image, that is, the initial pixel coordinates corresponding to the four vertices of each house target are obtained. Since the target position output by the YOLOv8 model is represented by a rectangular area, the four vertices of the house target will be obtained for each house target.

[0024] Step 2: Based on the UAV's real-time flight parameters, real-time attitude data, and geographic location information, construct a mapping from the initial pixel coordinates of the house target vertices to the actual latitude and longitude of the house target vertices. Based on the established mapping, map the initial pixel coordinates of the house target vertices to the three-dimensional geographic coordinate system to obtain the initial latitude and longitude coordinates of the four vertices of each house target.

[0025] Step 3: Transform the initial latitude and longitude coordinates of the four vertices of each house target to the UTM coordinate system (i.e., the Cartesian coordinate system in the plane, with due east as the positive x-axis and due north as the positive y-axis), and use the neural network registration model to output the coordinate offset of the four vertices of each house target in the UTM coordinate system.

[0026] Step 4: Based on the output of the neural network registration model, register the coordinates of the four vertices of each house target in the UTM coordinate system to obtain the registered coordinates of the four vertices of each house target in the UTM coordinate system.

[0027] Historical geographic information data of houses is obtained from the DEM data of the area to be inspected. New houses are identified based on the historical geographic information data and the registered coordinates. The number of houses identified as new houses is recorded and the real geographic coordinates of the new houses are output.

[0028] The following sections will explain several aspects of the YOLOv8 model training process:

[0029] (1) Obtaining the training dataset

[0030] In the training process of YOLOv8 model, in order to improve the generalization ability and robustness of the trained model, it is necessary to obtain images of houses with different lighting, different time periods, different seasons, and different sizes to enrich the training dataset.

[0031] In addition to the image data collected by drones on-site, this invention also uses an affine transformation algorithm to perform perspective distortion on the image data of houses to supplement the training dataset. Furthermore, the training set also includes negative sample images (i.e., images that do not contain houses) obtained from the Internet. Using negative sample images as negative samples input into the YOLOv8 model can prevent overfitting.

[0032] Furthermore, after the training set is constructed, the original image data needs to be preliminarily screened to remove blurry, duplicate, and abnormally exposed images to ensure data quality.

[0033] The filtered images are imported into labelImg (annotation) software. The house targets in the images are manually annotated using labelImg software, and a corresponding category label is assigned to each target. The annotation type is set to House. The annotation outputs the txt format label file required for YOLOv8 model training.

[0034] (2) Model hyperparameter settings

[0035] Before starting training, the hyperparameters need to be set according to the computer configuration: initial learning rate (lr0) = 0.0124, final learning rate (lrf) = 0.0117, momentum (momentum) = 0.98, weight decay (weight_decay) = 0.00048, number of training epochs (warmup_epochs) = 4.08, momentum value during training phase (warmup_momentum) = 0.711, and the learning rate of the bias term (w armup_bias_lr)=0.108, bounding box regression loss weight(box)=0.05, classification loss weight(cls)=0.5, positive and negative sample weights for classification loss(cls_pw)=1.0, target confidence loss weight(obj)=1.0, positive and negative sample weights for target confidence loss(obj_pw)=1.07, IoU threshold(iou_t)=0.2, anchor box matching threshold(anchor_t)=4.0, Focal The Gamma parameter of the Loss (focus loss function) is 0, the enhancement of hue (hsv_h) in the HSV color space is 0.015, the enhancement of saturation (hsv_s) in the HSV color space is 0.7, the enhancement of lightness (hsv_v) in the HSV color space is 0.4, the maximum angle of random image rotation (degrees) is 0.0, the maximum scale of random image translation (translate) is 0.1, the maximum scale of random image scaling (scale) is 0.5, and the maximum angle of random image cropping transformation is... (shear)=0.0, the magnitude of random perspective transformation of the image (perspective)=0.0, the probability of flipping the image vertically (flipud)=0.0, the probability of flipping the image horizontally (fliplr)=0.5, the probability of Mosaic enhancement (mosaic)=1.0, the probability of Mixup enhancement (mixup)=0.0, the probability of Copy-Paste enhancement (copy_paste)=0.0; and in order to make full use of the performance of the GPU, an early stopping mechanism is added to save model training time.

[0036] (3) Selection of evaluation indicators

[0037] Image data and its corresponding label files are input into the YOLOv8 model for training. After each training session, the model's precision, recall, and training time on the test set to be identified are output.

[0038] (Accuracy) can be viewed as a weighted average of the precision and recall of the YOLOv8 model, as shown in the following formula:

[0039]

[0040]

[0041]

[0042] in, This indicates the number of samples where the prediction result is positive and the label value is also positive. This represents the number of samples where the predicted result is negative and the label value is positive. This represents the number of samples where the predicted result is positive and the label value is negative.

[0043] Select the set of house detection model parameters that perform best on the test set as the trained model parameters.

[0044] Specific Implementation Method Two: Combining Figure 2 This embodiment is described below. The difference between this embodiment and specific embodiment one is that the specific process of step two is as follows:

[0045] Step 21: Correct the initial pixel coordinates corresponding to each vertex of the house target, specifically as follows:

[0046] Step 2.11: Calculate the normalized coordinates of the initial pixel coordinates of the vertices:

[0047]

[0048] in, Represents the initial pixel coordinates of the vertex. Represents normalized coordinates, This indicates the camera's focal length in the horizontal direction (in pixels). This indicates the focal length in the vertical direction of the camera. The pixel coordinates of the principal point (the intersection of the camera's principal axis and the image plane);

[0049] Step 2.12: Correct radial distortion of the normalized coordinates:

[0050]

[0051] in, , and Represents the radial distortion coefficient. express Distance to the center of the image Represents the coordinates after radial distortion correction;

[0052] Steps 2-13: Correct tangential distortion in the normalized coordinates:

[0053]

[0054] in, and Indicates the tangential distortion coefficient. Represents the coordinates after tangential distortion correction;

[0055] Step 214: Calculate the corrected normalized coordinates:

[0056]

[0057] in, Represents the corrected normalized coordinates;

[0058] Step 215: Project the corrected normalized coordinates back to pixel coordinates:

[0059]

[0060]

[0061] in, Indicates the corrected pixel coordinates;

[0062] Step 22: Calculate the actual distance offset of the corrected pixels based on the real-time flight parameters, real-time attitude data, and geographical location information of the UAV.

[0063] Step 221: Calculate the ground projection width (in meters) based on the UAV's relative height to the ground and horizontal field of view.

[0064]

[0065] in, Indicates the drone's altitude relative to the ground. Indicates the horizontal field of view of the drone. Indicates the width of the ground projection;

[0066] Step 222: Calculate the ground sampling distance (in meters per pixel) based on the ground projection width:

[0067]

[0068] in, Indicates the ground sampling distance. Indicates the horizontal resolution of the image (in pixels);

[0069] Step 223: Calculate the normalized coordinates of the corrected pixels:

[0070]

[0071] in, Represents the normalized coordinates of the corrected pixel;

[0072] Step 224: Calculate the offset of the normalized coordinates of the corrected pixels relative to the image center:

[0073]

[0074] in, Indicates the vertical resolution of the image. This represents the offset of the normalized coordinates of the corrected pixel relative to the image center;

[0075] Step 225: Calculate the actual distance offset of the corrected pixels (in meters):

[0076]

[0077] in, This indicates the actual distance offset of the pixel after correction;

[0078] It should be noted that the drone's relative altitude to the ground, horizontal field of view, drone heading angle, horizontal resolution, and vertical resolution of the image in step 22 are all changing in real time. When processing the current image, the data at the time the current image was captured is used.

[0079] Steps 2 and 3: Calculate the distance offset in the due east and due north directions based on the actual distance offset of the corrected pixels;

[0080] Specifically,

[0081] Step 231: Convert the drone's heading angle to radians:

[0082]

[0083] in, This indicates the heading angle of the drone (in degrees, counterclockwise is positive). Indicates radians;

[0084] Step 2.2: Determine the rotation matrix based on the radians. ;

[0085] Step 233: Calculate the distance offset in the east and north directions:

[0086]

[0087] in, This indicates the distance offset in the due east direction. Indicates the distance offset in the due north direction;

[0088] Step 24: Record the longitude of the center of the camera deployed on the drone as... The latitude of the center of the camera deployed on the drone is denoted as It should be noted that when processing vertices on the same image, the same set of methods is used. The value converts the distance offsets in the due east and due north directions into actual longitude and latitude offsets (ellipsoidal scale).

[0089]

[0090] in, Indicates the radius of curvature of the east-west axis. Represents the radius of meridional curvature. Indicates longitude offset. Indicates latitude offset;

[0091] Step 25: Calculate the actual latitude and longitude values ​​of the corrected pixels:

[0092]

[0093] in, This represents the actual longitude value of the pixel after correction. This represents the actual latitude value of the pixel after correction, i.e. This represents the initial latitude and longitude coordinates of the corrected pixel.

[0094] The other steps and parameters are the same as in Specific Implementation Method 1.

[0095] In this embodiment, the mapping relationship in step two-one can be organized, and the pixel coordinates before correction can be... To the corrected ideal pixel coordinates The mapping expression is written as:

[0096]

[0097] in, This indicates the distortion correction (radial + tangential) of the Brown–Conrady model. For the camera intrinsic parameter matrix, Representation matrix The reverse;

[0098] The camera intrinsic parameter matrix is ​​obtained through calibration:

[0099]

[0100] in, This represents the coordinates of the corresponding three-dimensional point;

[0101]

[0102] Summarizing steps 22 to 25, we obtain the corrected ideal pixel coordinates. to actual latitude and longitude Mapping expression:

[0103]

[0104] Using the above mapping expression, the four pixel vertices of each house target can be mapped to the three-dimensional geographic coordinate system to obtain the initial latitude and longitude coordinates of each vertex.

[0105] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that the specific process of steps two and three is as follows;

[0106]

[0107] in, This represents the rotation matrix.

[0108] Other steps and parameters are the same as in specific implementation method one or two.

[0109] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the radius of curvature of the ramusoidal region is:

[0110]

[0111] in, Indicates the semi-major axis. rice, Indicates the first eccentricity;

[0112]

[0113] in, Indicates flatness, .

[0114] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0115] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the meridional radius of curvature is:

[0116]

[0117] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0118] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that, before starting step three, the present invention first trains the neural network registration model:

[0119] Step 1: Obtain an image of a certain area, and denote the number of house targets detected from the image as m. Convert the actual latitude and longitude values ​​of the four vertices of each house target to the UTM coordinate system to obtain the coordinates in the UTM coordinate system (unit: meters).

[0120] The detected first The set of UTM coordinates of the four vertices of a house target is denoted as:

[0121]

[0122] The set of UTM coordinates of the four vertices of all house targets output during the inspection is the coordinate set before registration. ;

[0123] Obtain historical DEM data for the area. Based on this historical DEM data, obtain historical geographic information data for the houses. Then, based on this historical geographic information data, obtain the true latitude and longitude coordinates of the 4n vertices of n houses. Convert these true latitude and longitude coordinates to UTM coordinates to obtain the corresponding UTM coordinates for the four vertices of each house. The UTM coordinates corresponding to the actual latitude and longitude coordinates of the four vertices of the house are:

[0124]

[0125] Let the set of UTM coordinates corresponding to the actual latitude and longitude coordinates of all 4n vertices of the houses be denoted as ;

[0126] Step 2: Construct the training dataset ;

[0127] in, To register the input features of the neural network model, the input features It contains the longitude, latitude, altitude, yaw angle (drone_yaw), pitch angle (drone_pitch), roll angle (drone_raw), camera field of view (cam_fov), and focal length (focal_length) of the UAV before registration, as well as the coordinate set of the UAV before building target registration. UTM coordinates in;

[0128] For a housing target, the tag data The vertex of the house target in the set of real coordinates The set of UTM coordinates and the vertex coordinates of the house target before registration. The offset of the corresponding UTM coordinates in Because the offset values ​​are smaller and more concentrated, they are more suitable for the model to learn; label data This is the offset output by the neural network registration model;

[0129] Step 3: Set the maximum number of training rounds to 100, the batch size to 32, the early stop callback patience value to 10, the initial learning rate to 0.001, and the learning rate decay factor to 0.5. Divide the acquired dataset into three parts: training set, validation set, and test set in an 8:1:1 ratio. Learn the mapping relationship between input features and true coordinate offsets (training parameters). Evaluate the model performance on the test set and calculate the final mean squared error (MAE) to obtain the optimal weight file of the model used for coordinate error registration.

[0130] Specifically, the process of step three is as follows:

[0131] Perform steps 31 and 32 for each image separately:

[0132] Step 3: 1. Convert the actual latitude and longitude coordinates of the four vertices of each house target in the current image to the UTM coordinate system to obtain the coordinates of the four vertices of each house target in the UTM coordinate system;

[0133] Then, the input features corresponding to each vertex are constructed separately. The input features include the longitude of the UAV, the latitude of the UAV, the altitude of the UAV, the yaw angle of the UAV, the pitch angle of the UAV, the roll angle of the UAV, the field of view of the camera on the UAV, the focal length of the camera, and the coordinates of the vertex in the UTM coordinate system before registration.

[0134] Step 3.2: Use the input features corresponding to each vertex to form an input feature matrix X. Use the input feature matrix X as the input to the neural network registration model. The neural network registration model outputs the coordinate offset of each vertex in the UTM coordinate system.

[0135] It should be noted that the input feature matrix Each row represents the input feature corresponding to a vertex.

[0136] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0137] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that the neural network registration model is constructed based on a fully connected layer (MLP). The neural network registration model has four layers. The number of neurons in the first three fully connected layers is set to 128, 64, and 32, respectively. The activation function of each fully connected layer is set to ReLU. Batch normalization is performed on the first two hidden layers, and a regularization parameter (dropout) of 0.2 is set to prevent overfitting. The number of neurons in the output layer is 2, and the two neurons correspond to the offset in the UTM coordinate system to be predicted. and The output layer uses the Linear activation function;

[0138] Specifically, the working process of the neural network registration model is as follows:

[0139] (1) Input layer

[0140] Input feature matrix The neural network registration model is fed into the input layer. , Indicates the number of target houses to be registered;

[0141] (2) First hidden layer

[0142]

[0143] in, This represents the output matrix of the first hidden layer. , This represents the weight matrix of the first hidden layer. express transpose, This is the bias matrix for the first hidden layer. Indicates to Perform batch normalization. Represents the ReLU (Rectified Linear Unit) activation function. ;

[0144]

[0145] in, , This represents the number of neurons in the first hidden layer.

[0146]

[0147] in, ;

[0148] (3) Second hidden layer

[0149]

[0150] in, This represents the output matrix of the second hidden layer. , This represents the weight matrix of the second hidden layer. express transpose, This represents the bias matrix of the second hidden layer. Indicates to Perform batch normalization operations;

[0151]

[0152] Where l represents the number of neurons in the second hidden layer. ;

[0153]

[0154] in, ;

[0155] (4) Third hidden layer

[0156]

[0157] in, This represents the output matrix of the third hidden layer. , This represents the weight matrix of the third hidden layer. express transpose, This represents the third-layer hidden bias matrix;

[0158]

[0159] in, r is the number of neurons in the third hidden layer;

[0160]

[0161] in, ;

[0162] (5) Output layer:

[0163]

[0164] in, This represents the output matrix of the output layer (i.e., the output of the neural network registration model). , This represents the output layer weight matrix. express transpose, This represents the output layer bias matrix. For Linear Activation Function (LAF), ;

[0165]

[0166] in, ; s represents the number of neurons in the output layer;

[0167]

[0168] in, .

[0169] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0170] The neural network registration model of this embodiment can be represented as follows:

[0171]

[0172] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One to Seven in that the specific process of step four is as follows:

[0173] Step 4: 1. Obtain historical geographic information data of the actual houses based on the DEM data of the area to be inspected, and determine the set of rectangular areas where the actual houses are located within the area to be inspected based on the historical geographic information data of the actual houses.

[0174] That is, for any real house within the area to be inspected, according to the actual spatial shape of the house, the coordinates of the four vertices of the rectangular area where the real house is located are connected sequentially in the UTM coordinate system to obtain the closed rectangular area corresponding to the real house. The rectangular area corresponding to each real house is denoted as . ;

[0175] Let the set of closed rectangular regions containing all real houses within the area to be inspected be denoted as . :

[0176]

[0177] in, This indicates the actual number of houses within the area to be inspected;

[0178] Step 4.2: For the first [item] in the detected area to be inspected... The house, the first The four vertices of the house in the UTM coordinate system The axial coordinates are respectively compared with the output of the neural network registration model. Add the axial offsets together, and then... The four vertices of the house in the UTM coordinate system The axial coordinates are respectively compared with the output of the neural network registration model. The offsets in the axial directions are added together to obtain the registered first... The set of coordinates of the four vertices of a house ;

[0179] in:

[0180] Indicates the number after registration The first vertex of the house, Indicates the number after registration The second vertex of the house, Indicates the number after registration The third vertex of the house Indicates the number after registration The fourth vertex of the house;

[0181] Indicates the first registration The first vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The first vertex of the house in the UTM coordinate system Axis coordinate offset;

[0182] Indicates the first registration The first vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The first vertex of the house in the UTM coordinate system Axis coordinate offset;

[0183] Indicates the first registration The second vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The second vertex of the house in the UTM coordinate system Axis coordinate offset;

[0184] Indicates the first registration The second vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The second vertex of the house in the UTM coordinate system Axis coordinate offset;

[0185] Indicates the first registration The third vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The third vertex of the house in the UTM coordinate system Axis coordinate offset;

[0186] Indicates the first registration The third vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The third vertex of the house in the UTM coordinate system Axis coordinate offset;

[0187] Indicates the first registration The fourth vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The fourth vertex of the house in the UTM coordinate system Axis coordinate offset;

[0188] Indicates the first registration The fourth vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The fourth vertex of the house in the UTM coordinate system Axis coordinate offset;

[0189] The first After registration of the first house, the coordinates of its four vertices are connected sequentially to form the first... The closed quadrilateral area corresponding to each house Let the set of closed quadrilateral regions corresponding to all registered houses be denoted as . :

[0190]

[0191] in, This indicates the number of houses detected within the area to be inspected;

[0192] Step 4.3: Determine the new house based on the set of closed quadrilateral regions corresponding to the registered houses and the set of rectangular regions of the real houses;

[0193] Step 431, Calculate the first... After registration, the coordinates of the four vertices of each house are... Minimum value of axis coordinates , Maximum value of axis coordinates , Minimum value of axis coordinates and Maximum value of axis coordinates ;

[0194]

[0195]

[0196]

[0197]

[0198] Then according to , , and Determine the quadrilateral region Minimum bounding rectangle The minimum circumscribed rectangle This refers to the length and width relative to the UTM coordinate system. axis, A rectangle with its axis parallel to the bounding box, i.e., its minimum bounding box. It is the point , , and A closed rectangular region formed by connecting them sequentially;

[0199] The first A rectangular area corresponding to a real house The set of coordinates of the four vertices in the UTM coordinate system Recorded as:

[0200]

[0201] in:

[0202] Indicates the first A rectangular area corresponding to a real house The first vertex, Indicates the first A rectangular area corresponding to a real house The second vertex, Indicates the first A rectangular area corresponding to a real house. The third vertex, Indicates the first A rectangular area corresponding to a real house. The 4th vertex;

[0203] Represents the first vertex In the UTM coordinate system Axial coordinates, Represents the second vertex In the UTM coordinate system Axial coordinates, Represents the 3rd vertex In the UTM coordinate system Axial coordinates, Represents the 4th vertex In the UTM coordinate system Axis direction coordinates;

[0204] Represents the first vertex In the UTM coordinate system Axial coordinates, Represents the second vertex In the UTM coordinate system Axial coordinates, Represents the 3rd vertex In the UTM coordinate system Axial coordinates, Represents the 4th vertex In the UTM coordinate system Axis direction coordinates;

[0205] Calculate the first A rectangular area corresponding to a real house. Of the four vertices Minimum value of axis coordinates , Maximum value of axis coordinates , Minimum value of axis coordinates and Maximum value of axis coordinates :

[0206]

[0207]

[0208]

[0209]

[0210] Then according to , , and Determine the minimum bounding rectangle The minimum circumscribed rectangle This refers to the length and width relative to the UTM coordinate system. axis, A rectangle with its axis parallel to the bounding box, i.e., its minimum bounding box. It is the point , , and A closed rectangular region formed by connecting them sequentially;

[0211] Step 432: Calculate intermediate variables , , and :

[0212]

[0213]

[0214]

[0215]

[0216] Step 433: Calculate the rectangular region and Area of ​​intersection:

[0217] like and Then the rectangular region and No intersection, i.e., the area of ​​intersection. for:

[0218]

[0219] Otherwise, rectangular area and The area of ​​their intersection is:

[0220]

[0221] Step 4. Calculate the rectangular region. area and rectangular areas area :

[0222]

[0223]

[0224] Steps 4, 3, and 5: Calculate the rectangular region and The area of ​​the union of the sets;

[0225] Step 436: Calculate the rectangular region based on the area of ​​the union and the area of ​​the intersection. and IoU value;

[0226] Step 44: Determine if a new house is being built based on the IoU value:

[0227] For the The quadrilateral regions corresponding to the registered houses are calculated respectively. The IoU value between the minimum bounding rectangle region corresponding to each registered house and the minimum bounding rectangle region corresponding to each real house, if the... If the IoU value between the minimum bounding rectangle region corresponding to each registered house and the minimum bounding rectangle region corresponding to each real house is less than or equal to the threshold, then the th... The first registered house is a newly built house, the second The quadrilateral area corresponding to the first registered house is the location of the new house; otherwise, the... The registered houses are not newly built houses;

[0228] Similarly, the quadrilateral regions corresponding to each registered house are judged separately.

[0229] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0230] This invention sets rectangular regions rectangular regions and sets of rectangular regions The rectangular regions are all placed in the UTM coordinate system for detection to ensure detection accuracy. It should be noted that the threshold determination method in this embodiment is to perform statistical analysis on all IoU (Intersection over Union) values, calculate the mean and standard deviation of all IoU values, and determine the threshold based on the actual detection accuracy requirements of the inspection. This ensures that the threshold can cover the IoU range of all "detected rectangles and real rectangles of the same house" in the data, avoiding misjudgments caused by detection errors. The IoU threshold can also be flexibly adjusted according to the actual application scenario to adapt to the judgment needs of different regions and different types of houses. It has strong versatility and scalability, and also enhances the automation level of the system, reduces the need for manual intervention, and can be widely used in urban planning, illegal construction monitoring and other fields, with good practical application prospects.

[0231] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that the specific process of step four, three, and five is as follows:

[0232]

[0233] in, Represents a rectangular area and The area of ​​the union of the sets.

[0234] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0235] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that the specific process of step four, three, six is ​​as follows:

[0236]

[0237] in, Represents a rectangular area and The IoU value.

[0238] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.

[0239] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for detecting newly constructed buildings based on UAV aerial images, characterized in that, The method specifically includes the following steps: Step 1: Control the drone to collect images of the area to be inspected, and use the built-in RTK device on the drone to simultaneously collect the drone's real-time flight parameters, real-time attitude data and real-time geographical location information; and obtain DEM data of the area to be inspected. The images captured by the drone are used as input to the YOLOv8 model. The YOLOv8 model outputs the positions of the house targets in the images, that is, the initial pixel coordinates corresponding to the four vertices of each house target are obtained respectively. Step 2: Based on the UAV's real-time flight parameters, real-time attitude data, and geographic location information, construct a mapping from the initial pixel coordinates of the house target vertices to the actual latitude and longitude of the house target vertices. Based on the established mapping, map the initial pixel coordinates of the house target vertices to the three-dimensional geographic coordinate system to obtain the actual latitude and longitude coordinates of the four vertices of each house target. Step 3: Transform the actual latitude and longitude coordinates of the four vertices of each house target to the UTM coordinate system to obtain the coordinates of the vertices before registration in the UTM coordinate system, and use the neural network registration model to output the coordinate offset of the four vertices of each house target in the UTM coordinate system. The specific process of step three is as follows: Perform steps 31 and 32 for each image separately: Step 3:

1. Convert the actual latitude and longitude coordinates of the four vertices of each house target in the current image to the UTM coordinate system to obtain the coordinates of the four vertices of each house target in the UTM coordinate system; Then, the input features corresponding to each vertex are constructed separately. The input features include the longitude of the UAV, the latitude of the UAV, the altitude of the UAV, the yaw angle of the UAV, the pitch angle of the UAV, the roll angle of the UAV, the field of view of the camera on the UAV, the focal length of the camera, and the coordinates of the vertex in the UTM coordinate system before registration. Step 32: Use the input features corresponding to each vertex to form an input feature matrix X. Use the input feature matrix X as the input to the neural network registration model. The neural network registration model outputs the coordinate offset of each vertex in the UTM coordinate system. Step 4: Based on the output of the neural network registration model, register the coordinates of the four vertices of each house target in the UTM coordinate system to obtain the registered coordinates of the four vertices of each house target in the UTM coordinate system. Historical geographic information data of houses is obtained from the DEM data of the area to be inspected. New houses are identified based on the historical geographic information data and the registered coordinates. The number of houses identified as new houses is recorded and the real geographic coordinates of the new houses are output.

2. The method for detecting newly built houses based on UAV aerial images according to claim 1, characterized in that, The specific process of step two is as follows: Step 21: Correct the initial pixel coordinates corresponding to each vertex of the house target, specifically as follows: Step 2.11: Calculate the normalized coordinates of the initial pixel coordinates of the vertices: in, Represents the initial pixel coordinates of the vertex. Represents normalized coordinates, This indicates the focal length of the camera in the horizontal direction. This indicates the focal length in the vertical direction of the camera. Represents the pixel coordinates of the principal point; Step 2.12: Correct radial distortion of the normalized coordinates: in, , and Represents the radial distortion coefficient. express Distance to the center of the image Represents the coordinates after radial distortion correction; Steps 2-13: Correct tangential distortion in the normalized coordinates: in, and Indicates the tangential distortion coefficient. Represents the coordinates after tangential distortion correction; Step 214: Calculate the corrected normalized coordinates: in, Represents the corrected normalized coordinates; Step 215: Project the corrected normalized coordinates back to pixel coordinates: in, Indicates the corrected pixel coordinates; Step 22: Calculate the actual distance offset of the corrected pixels based on the real-time flight parameters, real-time attitude data, and geographical location information of the UAV. Step 221: Calculate the ground projection width based on the UAV's relative height to the ground and horizontal field of view: in, Indicates the drone's altitude relative to the ground. Indicates the horizontal field of view of the drone. Indicates the width of the ground projection; Step 222: Calculate the ground sampling distance based on the ground projection width: in, Indicates the ground sampling distance. Indicates the horizontal resolution of the image; Step 223: Calculate the normalized coordinates of the corrected pixels: in, Represents the normalized coordinates of the corrected pixel; Step 224: Calculate the offset of the normalized coordinates of the corrected pixels relative to the image center: in, Indicates the vertical resolution of the image. This represents the offset of the normalized coordinates of the corrected pixel relative to the image center; Step 225: Calculate the actual distance offset of the corrected pixels: in, This indicates the actual distance offset of the pixel after correction; Steps 2 and 3: Calculate the distance offset in the due east and due north directions based on the actual distance offset of the corrected pixels; Step 231: Convert the drone's heading angle to radians: in, Indicates the heading angle of the drone. Indicates radians; Step 2.2: Determine the rotation matrix based on the radians. ; Step 233: Calculate the distance offset in the east and north directions: in, This indicates the distance offset in the due east direction. Indicates the distance offset in the due north direction; Step 24: Record the longitude of the center of the camera deployed on the drone as... The latitude of the center of the camera deployed on the drone is denoted as The distance offsets in the due east and due north directions are converted into actual longitude and latitude offsets; in, Indicates the radius of curvature of the east-west axis. Represents the radius of meridional curvature. Indicates longitude offset. Indicates latitude offset; Step 25: Calculate the actual latitude and longitude values ​​of the corrected pixels: in, This represents the actual longitude value of the pixel after correction. This represents the actual latitude value of the pixel after correction.

3. The method for detecting newly built houses based on UAV aerial images according to claim 2, characterized in that, The specific process of steps two and three is as follows: in, This represents the rotation matrix.

4. The method for detecting newly built houses based on UAV aerial images according to claim 3, characterized in that, The radius of curvature of the tangent is: in, Indicates the semi-major axis. Indicates the first eccentricity; in, Indicates flatness.

5. The method for detecting newly built houses based on UAV aerial images according to claim 4, characterized in that, The meridional radius of curvature is: 。 6. The method for detecting newly built houses based on UAV aerial images according to claim 5, characterized in that, The working process of the neural network registration model is as follows: (1) Input layer Input feature matrix The neural network registration model is input through the input layer; (2) First hidden layer in, This represents the output matrix of the first hidden layer. This represents the weight matrix of the first hidden layer. express transpose, This is the bias matrix for the first hidden layer. Indicates to Perform batch normalization. Represents the ReLU activation function; (3) Second hidden layer in, This represents the output matrix of the second hidden layer. This represents the weight matrix of the second hidden layer. express transpose, This represents the bias matrix of the second hidden layer. Indicates to Perform batch normalization operations; (4) Third hidden layer in, This represents the output matrix of the third hidden layer. This represents the weight matrix of the third hidden layer. express transpose, This represents the bias matrix of the third hidden layer; (5) Output layer: in, This represents the output matrix of the output layer. This represents the output layer weight matrix. express transpose, This represents the output layer bias matrix. This is the activation function for Linear functions.

7. A method for detecting newly built houses based on UAV aerial images according to claim 6. Its characteristics are Therefore, the specific process of step four is as follows: Step 4:

1. Obtain historical geographic information data of the actual houses based on the DEM data of the area to be inspected, and determine the set of rectangular areas where the actual houses are located within the area to be inspected based on the historical geographic information data of the actual houses. That is, for any real house within the area to be inspected, the coordinates of the four vertices of the rectangular area where the real house is located in the UTM coordinate system are connected sequentially to obtain the closed rectangular area corresponding to the real house. The rectangular area corresponding to each real house is denoted as ; Let the set of closed rectangular regions containing all real houses within the area to be inspected be denoted as . : in, This indicates the actual number of houses within the area to be inspected; Step 4.2: For the first [item] in the detected area to be inspected... The house, the first The four vertices of the house in the UTM coordinate system The axial coordinates are respectively compared with the output of the neural network registration model. Add the axial offsets together, and then... The four vertices of the house in the UTM coordinate system The axial coordinates are respectively compared with the output of the neural network registration model. The offsets in the axial directions are added together to obtain the registered first... The set of coordinates of the four vertices of a house ; in: Indicates the number after registration The first vertex of the house, Indicates the number after registration The second vertex of the house, Indicates the number after registration The third vertex of the house Indicates the number after registration The fourth vertex of the house; Indicates the first registration The first vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The first vertex of the house in the UTM coordinate system Axis coordinate offset; Indicates the first registration The first vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The first vertex of the house in the UTM coordinate system Axis coordinate offset; Indicates the first registration The second vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The second vertex of the house in the UTM coordinate system Axis coordinate offset; Indicates the first registration The second vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The second vertex of the house in the UTM coordinate system Axis coordinate offset; Indicates the first registration The third vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The third vertex of the house in the UTM coordinate system Axis coordinate offset; Indicates the first registration The third vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The third vertex of the house in the UTM coordinate system Axis coordinate offset; Indicates the first registration The fourth vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The fourth vertex of the house in the UTM coordinate system Axis coordinate offset; Indicates the first registration The fourth vertex of the house in the UTM coordinate system Axial coordinates, Indicates the first The fourth vertex of the house in the UTM coordinate system Axis coordinate offset; The first After registration of the first house, the coordinates of its four vertices are connected sequentially to form the first... The closed quadrilateral area corresponding to each house Let the set of closed quadrilateral regions corresponding to all registered houses be denoted as . : in, This indicates the number of houses detected within the area to be inspected; Step 4.3: Determine the new house based on the set of closed quadrilateral regions corresponding to the registered houses and the set of rectangular regions of the real houses; Step 431: Calculate the first... After registration, the coordinates of the four vertices of each house are... Minimum value of axis coordinates , Maximum value of axis coordinates , Minimum value of axis coordinates and Maximum value of axis coordinates ; Then according to , , and Determine the quadrilateral region Minimum bounding rectangle The minimum circumscribed rectangle This refers to the length and width relative to the UTM coordinate system. axis, A rectangle parallel to the axis, i.e., the smallest bounding rectangle. It is the point , , and A closed rectangular region formed by connecting them sequentially; The first A rectangular area corresponding to a real house. The set of coordinates of the four vertices in the UTM coordinate system Recorded as: in: Indicates the first A rectangular area corresponding to a real house. The first vertex, Indicates the first A rectangular area corresponding to a real house. The second vertex, Indicates the first A rectangular area corresponding to a real house. The third vertex, Indicates the first A rectangular area corresponding to a real house. The 4th vertex; Represents the first vertex In the UTM coordinate system Axial coordinates, Represents the second vertex In the UTM coordinate system Axial coordinates, Represents the 3rd vertex In the UTM coordinate system Axial coordinates, Represents the 4th vertex In the UTM coordinate system Axis direction coordinates; Represents the first vertex In the UTM coordinate system Axial coordinates, Represents the second vertex In the UTM coordinate system Axial coordinates, Represents the 3rd vertex In the UTM coordinate system Axial coordinates, Represents the 4th vertex In the UTM coordinate system Axis direction coordinates; Calculate the first A rectangular area corresponding to a real house. Of the four vertices Minimum value of axis coordinates , Maximum value of axis coordinates , Minimum value of axis coordinates and Maximum value of axis coordinates : Then according to , , and Determine the minimum bounding rectangle The minimum circumscribed rectangle This refers to the length and width relative to the UTM coordinate system. axis, A rectangle parallel to the axis, i.e., the smallest bounding rectangle. It is the point , , and A closed rectangular region formed by connecting them sequentially; Step 432: Calculate intermediate variables , , and : Step 433: Calculate the rectangular region and Area of ​​intersection: like and Then the rectangular region and No intersection, i.e., the area of ​​intersection. for: Otherwise, rectangular area and The area of ​​their intersection is: Step 4. Calculate the rectangular region. area and rectangular areas area : Steps 4, 3, and 5: Calculate the rectangular region and The area of ​​the union of the sets; Step 436: Calculate the rectangular region based on the area of ​​the union and the area of ​​the intersection. and IoU value; Step 44: Determine if a new house is being built based on the IoU value: For the The quadrilateral regions corresponding to the registered houses are calculated respectively. The IoU value between the minimum bounding rectangle region corresponding to each registered house and the minimum bounding rectangle region corresponding to each real house, if the... If the IoU value between the minimum bounding rectangle region corresponding to each registered house and the minimum bounding rectangle region corresponding to each real house is less than or equal to the threshold, then the th... The first registered house is a newly built house, the second The quadrilateral area corresponding to the first registered house is the location of the new house; otherwise, the... The registered houses are not newly built houses; Similarly, the quadrilateral regions corresponding to each registered house are judged separately.

8. The method for detecting newly built houses based on UAV aerial images according to claim 7, characterized in that, The specific process of step four, three, and five is as follows: in, Represents a rectangular area and The area of ​​the union of the sets.

9. A method for detecting newly constructed buildings based on UAV aerial images according to claim 8, characterized in that, The specific process of step four, three, and six is ​​as follows: in, Represents a rectangular area and The IoU value.

Images