A method and system for identifying illegal buildings using drones based on multi-threaded training mechanism

Abstract

The main purpose of the present invention is to provide a method for identifying illegal buildings using a drone based on a multi-threaded training mechanism. The method first uses a drone adaptive image capturer to collect images and obtain multiple original high-altitude building images. The multiple original high-altitude building images are then preprocessed to generate multiple standard high-altitude building images and multiple groups of authentic illegal building discriminant vectors of standard high-altitude building images. According to the multiple standard high-altitude building images, an illegal building identification model based on a multi-threaded training mechanism is constructed to obtain multiple groups of predictive illegal building discriminant vectors. The multiple standard high-altitude building images, the multiple groups of predictive illegal building discriminant vectors and the multiple groups of authentic illegal building discriminant vectors are used as data samples to train the illegal building identification model based on the multi-threaded training mechanism, and finally the efficiency and performance of the illegal building identification model are improved. The present invention solves the problems of slow training speed, unstable convergence and poor recognition performance in traditional illegal building identification methods.

Description

Technical Field

[0001] The present invention relates to the field of illegal building image recognition, and in particular to an illegal building recognition method and system using a drone based on a multi-threaded training mechanism. Background Art

[0002] With the continuous development of urban economy and the increasing size of cities, higher requirements are put forward for urban management, such as comprehensive inspection of illegal urban buildings. Illegal urban buildings refer to buildings that do not comply with relevant laws and regulations on land planning and urban management and have changed their style without authorization, such as colored steel tile houses and dilapidated bungalows. Illegal urban buildings occupy urban development space, damage the image of the city, and even endanger people's safety. They are a problem that needs to be solved urgently in the development of urban management.

[0003] How to accurately and comprehensively identify illegal buildings in cities is of great significance for urban management and planning. Traditional illegal building identification methods rely on a large number of personnel for on-site inspections, but manual detection methods have some defects, such as low recognition efficiency and large resource consumption. In recent years, drone aerial photography technology has developed rapidly, with advantages such as strong flexibility, low cost and simple operation. The images taken by drones have high resolution and can clearly reflect the basic characteristics of buildings. Deep learning is widely used in fields such as speech recognition, image recognition and target detection. Combining drone images with deep learning to identify illegal buildings can greatly improve detection efficiency and save a lot of resources. However, learning-based image recognition methods also have limitations, such as slow training speed, weak generalization, unstable convergence and high missed detection rate. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method for identifying illegal buildings using a drone based on a multi-threaded training mechanism, which can accurately identify illegal buildings, improve the performance of illegal building identification, and has strong robustness.

[0005] One of the purposes of the present invention is achieved through the following technical solutions:

[0006] A method for identifying illegal buildings by using a drone based on a multi-threaded training mechanism, the method comprising the following steps:

[0007] S1: Use the UAV adaptive image capturer to collect images and obtain multiple original high-altitude building images;

[0008] S2: Preprocessing multiple original high-altitude building images to generate multiple standard high-altitude building images and multiple groups of authentic illegal building discriminant vectors of standard high-altitude building images;

[0009] S3: Based on multiple standard high-altitude building images, an illegal building recognition model based on a multi-threaded training mechanism is constructed to obtain multiple sets of predictive illegal building discriminant vectors;

[0010] S4: Multiple standard high-altitude building images, multiple sets of predictive illegal building discriminant vectors, and multiple sets of real illegal building discriminant vectors are used as data samples to train an illegal building recognition model based on a multi-threaded training mechanism;

[0011] S5: Use the trained illegal building recognition model to obtain the labels and display boxes of illegal buildings in the standard high-altitude building images.

[0012] Further, the step S2 specifically includes the following steps:

[0013] S21: grayscale multiple original high-altitude building images to generate the first-stage high-altitude building images;

[0014] S22: performing standardization processing on a plurality of first-stage high-altitude building images to generate a second-stage high-altitude building image;

[0015] S23: performing spatial transformation processing on the plurality of second-stage high-altitude building images to generate a plurality of standard high-altitude building images;

[0016] S24: marking illegal buildings in a plurality of standard high-altitude building images, and generating a plurality of sets of authentic illegal building identification labels and display frames for standard high-altitude building images;

[0017] S25: Design a vector conversion scheme to process the authenticity illegal building discrimination labels and display frames of multiple sets of standard high-altitude building images, and generate authenticity illegal building discrimination vectors of multiple sets of standard high-altitude building images.

[0018] Further, the step S3 specifically includes the following steps:

[0019] S31: construct multiple illegal building recognition models with the same network structure, parameter type and number of parameters in multiple threads;

[0020] S32: the illegal building recognition model of the first thread is used as the global learning model, and the illegal building recognition models in the remaining threads are used as branch calculation models;

[0021] S33: Inputting a plurality of standard high-altitude building images into a plurality of branch calculation models respectively to obtain a plurality of sets of predictive illegal building discrimination vectors.

[0022] Further, the step S4 specifically includes the following steps:

[0023] Step S41: using multiple standard high-altitude building images, multiple sets of predictive illegal building discriminant vectors, and multiple sets of real illegal building discriminant vectors as data samples, and obtaining multiple sets of different parameter gradients based on the loss function of the multiple branch calculation model;

[0024] Step S42: passing multiple groups of different parameter gradients into the global learning model to update the parameters of the global learning model;

[0025] Step S43: at fixed intervals t, all parameter values ​​of the global learning model are respectively transferred to the multiple branch calculation models to cover the original parameter values ​​of the branch calculation models, and the multiple branch calculation models obtain new parameter values;

[0026] Step S44: Repeat steps S33, S41, S42 and S43 until the end time Tend is reached, and obtain the trained illegal building recognition model.

[0027] Further, the step S5 specifically includes the following steps:

[0028] Step S51: inputting the standard high-altitude building image into the trained illegal building recognition model to obtain a predictive illegal building discriminant vector;

[0029] Step S52: converting the predictive illegal building discriminant vector into the label and display frame of the illegal building in the standard high-altitude building image.

[0030] Further, the vector conversion scheme in step S25 is:

[0031] The authenticity illegal building discrimination labels and display frames of multiple groups of standard high-altitude building images are encoded into authenticity illegal building discrimination vectors, where the labels are encoded using a one-hot vector, and the display frames are encoded using the length value, width value, and the horizontal and vertical coordinates of the center point of the display frame.

[0032] Furthermore, in step S31, the illegal building recognition model includes: an aggregate feature sub-model, a bidirectional combination feature sub-model and a feature recognition sub-model.

[0033] Furthermore, in step S33, the calculation formula for obtaining multiple sets of predictive illegal building discrimination vectors is as follows:

[0034] x1=(x0*w0+b0)

[0035] x2=Contact(Slice(x1))

[0036] x3=(BN(Conv(x2))*w1+b1)

[0037] y0=Conv(x3)z0=Conv(x3)

[0038] y1=StochasticPooling(y0)z1=MaxPooling(z0)

[0039] y2=y1*w y1 +b y1 +y1 z2=z1*w z1 +b z1 +z1

[0040] y3=(BN(y2)*w y2 +b y2 )z3=(BN(z2)*w z2 +b z2 )

[0041] h0=Contact(y3,z3)

[0042] h1=Conv(h0)

[0043] h2=Softmax(h1)

[0044] Among them, x0 is the pixel matrix of the standard high-altitude building image, w0 and b0 are the first weight parameter and the first bias parameter of the aggregation feature sub-model, x1 is the first intermediate vector of the aggregation feature sub-model, Slice() is the slicing function, Contact() is the cascade function, x2 is the second intermediate vector of the aggregation feature sub-model, Conv() is the convolution function, BN() is the batch processing function, w1 and b1 are the second weight parameter and the second bias parameter of the aggregation feature sub-model, x3 is the third intermediate vector of the aggregation feature sub-model; y0 is the forward feature extraction in the bidirectional combination feature sub-model Take the first intermediate vector of the network, z0 is the first intermediate vector of the reverse feature extraction network in the bidirectional combination feature submodel, StochasticPooling() is the random pooling function, MaxPooling() is the maximum pooling function, y1 is the second intermediate vector of the forward feature extraction network in the bidirectional combination feature submodel, z1 is the second intermediate vector of the reverse feature extraction network in the bidirectional combination feature submodel, y2 is the third intermediate vector of the forward feature extraction network in the bidirectional combination feature submodel, z2 is the third intermediate vector of the reverse feature extraction network in the bidirectional combination feature submodel, w y1 and b y1 are the first weight parameter and the first bias parameter of the forward feature extraction network in the bidirectional combined feature sub-model, respectively. z1 and b z1 are the first weight parameter and the first bias parameter of the reverse feature extraction network in the bidirectional combined feature sub-model, respectively. y2 and b y2are the second weight parameter and the second bias parameter of the forward feature extraction network in the bidirectional combined feature sub-model, respectively. z2 and b z2 They are respectively the second weight parameter and the second bias parameter of the reverse feature extraction network in the bidirectional combined feature sub-model, y3 is the fourth intermediate vector of the forward feature extraction network in the bidirectional combined feature sub-model, and z3 is the fourth intermediate vector of the reverse feature extraction network in the bidirectional combined feature sub-model; h0 is the first intermediate vector of the feature recognition sub-model, h1 is the second intermediate vector of the feature recognition sub-model, Softmax() is the activation function, and h3 is the predictive illegal building discriminant vector output by the activation function layer in the feature recognition sub-model.

[0045] Furthermore, in step S41, the loss function of the branch calculation model is:

[0046]

[0047] Where G is the loss function, k is a subscript with a value range of 1 to N, N is the number of data samples, and V k is the discriminant vector of the real illegal building, P k is the discriminant vector of predictive illegal buildings.

[0048] The second aspect of the present invention aims to provide a drone illegal building identification system based on a multi-threaded training mechanism, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the method described above when executing the computer program.

[0049] The beneficial effects of the present invention are as follows: the present invention collects high-resolution original high-altitude building images through an unmanned aerial vehicle adaptive image capturer, which can provide more image details and features for subsequent stages; by preprocessing the original high-altitude building images to generate standard high-altitude building images, it can provide richer image samples and facilitate subsequent processing by an illegal building recognition model based on a multi-threaded training mechanism; the illegal building recognition model based on the multi-threaded training mechanism utilizes the multi-threaded training mechanism to enable the illegal building recognition model to converge stably and improve the training speed of the illegal building recognition model, and at the same time, the illegal building recognition model can comprehensively and accurately extract various key features in the standard high-altitude building images, conduct a comprehensive analysis of each key feature, and finally accurately identify illegal buildings, thereby improving the illegal building recognition performance and having strong robustness.

[0050] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description and the preceding claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0052] Figure 1 The present invention is a flowchart of a method for identifying illegal buildings using a drone based on a multi-threaded training mechanism.

[0053] Figure 2 This is the original high-altitude building map of the embodiment of the present invention.

[0054] Figure 3 This is a standard high-altitude building diagram according to an embodiment of the present invention.

[0055] Figure 4 Schematic diagram of the authenticity of illegal building discrimination vectors of the standard high-altitude building image according to the embodiment of the present invention

[0056] Figure 5 This is a diagram of the architecture of the illegal building identification model according to an embodiment of the present invention.

[0057] Figure 6 This is a schematic diagram of testing the illegal building recognition model after training according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than for limiting the protection scope of the present invention.

[0059] The present invention provides a method for identifying illegal buildings by using a drone based on a multi-threaded training mechanism, such as Figure 1 As shown, the following steps are included:

[0060] Step S1: using a UAV adaptive image capturer to collect images and obtain multiple original high-altitude building images;

[0061] In this embodiment, the original high-altitude building image is as follows: Figure 2 Step S1 specifically includes the following steps:

[0062] Step S11: using a drone adaptive image capturer to capture multiple images of the entire high-altitude building area;

[0063] Step S12: cropping the plurality of overall high-altitude building area images to generate a plurality of original high-altitude building images of the same size.

[0064] Step S2: performing image preprocessing on a plurality of original high-altitude building images to generate a plurality of standard high-altitude building images and a plurality of groups of authenticity illegal building discrimination vectors of standard high-altitude building images;

[0065] Step S2: performing image preprocessing on a plurality of original high-altitude building images to generate a plurality of standard high-altitude building images and a plurality of groups of authenticity illegal building discrimination vectors of standard high-altitude building images;

[0066] In this embodiment, the standard high-altitude building image is as follows: Figure 3 Step S2 specifically includes the following steps:

[0067] Step S21: grayscale the multiple original high-altitude building images to generate the first-stage high-altitude building images. In this embodiment, a geometric mean method is designed to perform grayscale processing. The formula of the geometric mean method is as follows:

[0068]

[0069] Where i is the horizontal coordinate of the pixel in the image, j is the vertical coordinate of the pixel in the image, Fir(i,j) represents the grayscale value of the pixel with horizontal coordinate i and vertical coordinate j, R(i,j) represents the red channel brightness value of the pixel with horizontal coordinate i and vertical coordinate j, G(i,j) represents the green channel brightness value of the pixel with horizontal coordinate i and vertical coordinate j, and B(i,j) represents the blue channel brightness value of the pixel with horizontal coordinate i and vertical coordinate j.

[0070] The geometric mean method is designed to grayscale the original high-altitude building image, which can facilitate the calculation of the subsequent illegal building recognition model. The grayscale image can reflect the light intensity, which is convenient for feature extraction and edge information acquisition in the subsequent stage, thereby better identifying the object contour and avoiding the influence of image color changes.

[0071] Step S22: performing standardization processing on a plurality of first-stage high-altitude building images to generate second-stage high-altitude building images;

[0072] In this embodiment, the standardization process is performed by designing a mean removal method, and the mean removal method formula is as follows:

[0073]

[0074]

[0075]

[0076] Where E is the average gray value of all pixels in the image, n is the maximum horizontal coordinate of the pixel in the image, m is the maximum vertical coordinate of the pixel in the image, ∑ is the summation function, D is the variance of the pixel in the image, Sec(i,j) represents the standard value of the pixel with horizontal coordinate i and vertical coordinate j, max() is the maximum value function, and c is the penalty factor.

[0077] The de-meaning method is designed to standardize the high-altitude building images in the first stage, which can provide more standard image samples for the subsequent illegal building models, thereby accelerating the convergence of the model and improving the accuracy of the model.

[0078] Step S23: performing spatial transformation processing on the plurality of second-stage high-altitude building images by rotation and flipping to generate a plurality of standard high-altitude building images;

[0079] Step S24: marking illegal buildings in a plurality of standard high-altitude building images, and generating a plurality of sets of authentic illegal building identification labels and display frames for standard high-altitude building images;

[0080] Step S25: Process the authenticity illegal building discrimination labels and display frames of multiple sets of standard high-altitude building images to generate authenticity illegal building discrimination vectors of multiple sets of standard high-altitude building images. In this embodiment, a vector conversion scheme is designed to process the authenticity illegal building discrimination labels and display frames.

[0081] The vector conversion scheme in this embodiment is: the authenticity illegal building discrimination labels and display frames of multiple sets of standard high-altitude building images are encoded into authenticity illegal building discrimination vectors, where the labels are encoded using a one-hot vector, and the display frame is encoded using the length value, width value, and the horizontal and vertical coordinates of the center point of the display frame. The authenticity illegal building discrimination vectors of multiple sets of standard high-altitude building images are as follows: Figure 4 shown.

[0082] Step S3: constructing an illegal building recognition model based on a multi-threaded training mechanism according to a plurality of standard high-altitude building images to obtain a plurality of sets of predictive illegal building discriminant vectors;

[0083] In this embodiment, the illegal building identification model architecture is as follows Figure 5 The specific steps are as follows:

[0084] Step S31: constructing multiple illegal building recognition models with the same network structure, the same parameter type and the same number of parameters in multiple threads;

[0085] The illegal building identification model in this embodiment includes: an aggregate feature sub-model, a two-way combined feature sub-model and a feature identification sub-model;

[0086] The aggregate feature sub-model includes: an input layer, a slice layer, a cascade layer, a depth separation convolution layer, a batch processing layer and an activation function layer. The input end of the input layer serves as the input end of the aggregate feature sub-model, and its output end is connected to the input end of the slice layer; the slice layer segments a standard high-altitude building image and outputs nine low-resolution high-altitude building images; the output end of the slice layer is connected to the input end of the cascade layer; the cascade layer connects nine low-resolution high-altitude building images and outputs high-altitude building images with aggregate features; the output end of the cascade layer is connected to the input end of the depth separation convolution layer; the depth separation convolution layer extracts features from the high-altitude building images with aggregate features; the output end of the depth separation convolution layer is connected to the input end of the batch processing layer; the output end of the batch processing layer is connected to the input end of the activation function layer; the activation function layer outputs high-altitude building images with fine-grained features; the output end of the activation function layer is connected to the input end of the input layer in the bidirectional combination feature sub-model;

[0087] The bidirectional combined feature sub-model includes: an input layer, a forward feature extraction network, a reverse feature extraction network and a comprehensive feature output layer. The output end of the input layer is connected to the input end of the forward feature extraction network or the reverse feature extraction network; the output end of the forward feature extraction network or the reverse feature extraction network is connected to the input end of the comprehensive feature output layer; the comprehensive feature output layer splices the features output by the forward feature extraction network with the features output by the reverse feature extraction network; the output end of the comprehensive feature output layer is connected to the input end of the input layer in the feature recognition sub-model.

[0088] The forward feature extraction network includes: a deep separation convolution layer, a random pooling layer, a residual layer, a batch processing layer and an activation function layer. The input end of the deep separation convolution layer is used as the input end of the forward feature extraction network, and its output end is connected to the input end of the random pooling layer; the random pooling layer performs feature dimension reduction; the output end of the random pooling layer is connected to the input end of the residual layer; the output end of the residual layer is connected to the input end of the batch processing layer; the output end of the batch processing layer is connected to the input end of the activation function layer; the output end of the activation function layer is used as the output end of the forward feature extraction network.

[0089] The reverse feature extraction network includes: a depth separation convolution layer, a maximum pooling layer, a depth separation convolution layer, a batch processing layer and an activation function layer. The input end of the depth separation convolution layer is used as the input end of the reverse feature extraction network, and its output end is connected to the input end of the maximum pooling layer; the maximum pooling layer performs feature dimension reduction; the output end of the maximum pooling layer is connected to the input end of the depth separation convolution layer; the output end of the depth separation convolution layer is connected to the input end of the batch processing layer; the output end of the batch processing layer is connected to the input end of the activation function layer; the output end of the activation function layer is used as the output end of the reverse feature extraction network.

[0090] The feature recognition sub-model includes: an input layer, a convolution layer and an activation function layer. The output end of the input layer is connected to the input end of the convolution layer; the output end of the convolution layer is connected to the input end of the activation function layer; the activation function layer outputs a predictive illegal building discrimination vector.

[0091] Step S32: using the illegal building recognition model of the first thread as a global learning model, and the illegal building recognition models in the remaining threads as branch calculation models;

[0092] Step S33: Inputting a plurality of standard high-altitude building images into a plurality of branch calculation models respectively to obtain a plurality of sets of predictive illegal building discrimination vectors.

[0093] The calculation formula for obtaining multiple sets of predictive illegal building discrimination vectors in step S33 is as follows:

[0094] x1=(x0*w0+b0)

[0095] x2=Contact(slice(x1))

[0096] x3=(BN(Conv(x2))*w1+b1)

[0097] y0=Conv(x3)z0=Conv(x3)

[0098] y1=StochasticPooling(y0)z1=MaxPooling(z0)

[0099] y2=y1*w y +b y +y1 z2=z1*w z +b z +z1

[0100] y3=(BN(y2)*w y +b y )z3=(BN(z2)*w z +b z )

[0101] h0=Contact(y3,z3)

[0102] h1=Conv(h0)

[0103] h2=Softmax(h1)

[0104] Among them, x0 is the pixel matrix of the standard high-altitude building image, w0 and b0 are the first weight parameter and the first bias parameter of the aggregation feature sub-model, x1 is the first intermediate vector of the aggregation feature sub-model, Slice() is the slicing function, Contact() is the cascade function, x2 is the second intermediate vector of the aggregation feature sub-model, Conv() is the convolution function, BN() is the batch processing function, w1 and b1 are the second weight parameter and the second bias parameter of the aggregation feature sub-model, x3 is the third intermediate vector of the aggregation feature sub-model; y0 is the forward feature extraction in the bidirectional combination feature sub-model Take the first intermediate vector of the network, z0 is the first intermediate vector of the reverse feature extraction network in the bidirectional combination feature submodel, StochasticPooling() is the random pooling function, MaxPooling() is the maximum pooling function, y1 is the second intermediate vector of the forward feature extraction network in the bidirectional combination feature submodel, z1 is the second intermediate vector of the reverse feature extraction network in the bidirectional combination feature submodel, y2 is the third intermediate vector of the forward feature extraction network in the bidirectional combination feature submodel, z2 is the third intermediate vector of the reverse feature extraction network in the bidirectional combination feature submodel, w y1 and b y1 are the first weight parameter and the first bias parameter of the forward feature extraction network in the bidirectional combined feature sub-model, respectively. z1 and b z1 are the first weight parameter and the first bias parameter of the reverse feature extraction network in the bidirectional combined feature sub-model, respectively. y2 and b y2 are the second weight parameter and the second bias parameter of the forward feature extraction network in the bidirectional combined feature sub-model, respectively. z2 and b z2 They are respectively the second weight parameter and the second bias parameter of the reverse feature extraction network in the bidirectional combined feature sub-model, y3 is the fourth intermediate vector of the forward feature extraction network in the bidirectional combined feature sub-model, and z3 is the fourth intermediate vector of the reverse feature extraction network in the bidirectional combined feature sub-model; h0 is the first intermediate vector of the feature recognition sub-model, h1 is the second intermediate vector of the feature recognition sub-model, Softmax() is the activation function, and h3 is the predictive illegal building discriminant vector output by the activation function layer in the feature recognition sub-model.

[0105] Step S3 can achieve the following beneficial effects: First, the multi-threaded training mechanism can speed up the training speed of the illegal building recognition model and make the model converge more stably. Secondly, the aggregation feature sub-model in the illegal building recognition model can reduce the dimension of the standard high-altitude building image without discarding important features, convert the high-resolution image into a multi-channel low-resolution image, comprehensively extract features and aggregate them, and provide rich image feature information for subsequent stages. Furthermore, the bidirectional combined feature sub-model in the illegal building recognition model uses a forward feature extraction network and a reverse feature extraction network to further process the image feature information respectively. The forward feature extraction network can accurately capture the key feature information containing the target category, and the reverse feature extraction network can comprehensively capture the global feature information containing the overall outline of the target. Finally, the feature recognition sub-model in the illegal building recognition model can comprehensively analyze the key feature information and the global feature information, and then accurately identify the illegal buildings and their locations.

[0106] Step S4: using multiple standard high-altitude building images, multiple sets of predictive illegal building discriminant vectors, and multiple sets of authentic illegal building discriminant vectors as data samples to train an illegal building recognition model based on a multi-threaded training mechanism; specifically, in this embodiment, step S4 includes the following steps:

[0107] Step S41: using multiple standard high-altitude building images, multiple sets of predictive illegal building discriminant vectors, and multiple sets of real illegal building discriminant vectors as data samples, and obtaining multiple sets of different parameter gradients based on the loss function of the multiple branch calculation model;

[0108] The loss function of the branch calculation model is:

[0109]

[0110] Where G is the loss function, k is a subscript with a value range of 1 to N, N is the number of data samples, and V k is the discriminant vector of the real illegal building, P k is the discriminant vector of predictive illegal buildings.

[0111] Step S42: passing multiple groups of different parameter gradients into the global learning model to update the parameters of the global learning model;

[0112] The method of updating the parameters of the global learning model is: multiple sets of different parameter gradient back propagation update the parameters of the global learning model.

[0113] Step S43: at fixed intervals t, all parameter values ​​of the global learning model are respectively transferred to the multiple branch calculation models to cover the original parameter values ​​of the branch calculation models, and the multiple branch calculation models obtain new parameter values;

[0114] Step S44: Repeat steps S33, S41, S42 and S43 until the end time T is reached. end , obtain the trained illegal building recognition model;

[0115] In step S4, by aggregating the experience learned from multiple branch calculation models into the global learning model, the generalization ability of the illegal building recognition model can be improved, and illegal buildings in various high-altitude building scenes can be accurately identified. In addition, the multi-threaded training mechanism can speed up the training of the illegal building recognition model.

[0116] Step S5: using the trained illegal building recognition model to obtain the labels and display frames of illegal buildings in the standard high-altitude building image.

[0117] Step S51: input the standard high-altitude building image into the trained illegal building recognition model to obtain a predictive illegal building discriminant vector.

[0118] Step S52: According to the vector conversion scheme in step S25, the predictive illegal building discrimination vector is converted into a label and display box of the illegal building in the standard high-altitude building image.

[0119] In this embodiment, Figure 6 As shown, according to the trained illegal building recognition model, the labels and display frames of illegal buildings in the original high-altitude building images can be accurately identified, thereby improving the recognition efficiency and performance.

[0120] In this embodiment, Figure 2 The original high-altitude building images shown are from the training set. The illegal building recognition model is trained multiple times based on the training set. The trained illegal building recognition model can accurately identify the labels and display frames of illegal buildings for the original high-altitude images in the test set, thereby improving the recognition performance.

[0121] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.

[0122] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0123] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or portions thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0124] The computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents physical and tangible objects, including specific visual depictions of physical and tangible objects produced on the display.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for identifying illegal buildings by drone based on a multi-threaded training mechanism, characterized by: The method comprises the following steps: S1: Use the UAV adaptive image capturer to collect images and obtain multiple original high-altitude building images; S2: Preprocessing multiple original high-altitude building images to generate multiple standard high-altitude building images and multiple groups of authentic illegal building discriminant vectors of standard high-altitude building images; S3: Based on multiple standard high-altitude building images, an illegal building recognition model based on a multi-threaded training mechanism is constructed to obtain multiple sets of predictive illegal building discriminant vectors; the step S3 specifically includes the following steps: S31: construct multiple illegal building recognition models with the same network structure, parameter type and number of parameters in multiple threads; S32: the illegal building recognition model of the first thread is used as the global learning model, and the illegal building recognition models in the remaining threads are used as branch calculation models; S33: inputting a plurality of standard high-altitude building images into a plurality of branch calculation models respectively to obtain a plurality of sets of predictive illegal building discrimination vectors; S4: Multiple standard high-altitude building images, multiple sets of predictive illegal building discriminant vectors, and multiple sets of real illegal building discriminant vectors are used as data samples to train an illegal building recognition model based on a multi-threaded training mechanism; S5: Use the trained illegal building recognition model to obtain the labels and display boxes of illegal buildings in the standard high-altitude building images.

2. According to claim 1, a method for identifying illegal buildings by unmanned aerial vehicles based on a multi-threaded training mechanism is characterized in that: The step S2 specifically includes the following steps: S21: grayscale multiple original high-altitude building images to generate the first-stage high-altitude building images; S22: performing standardization processing on a plurality of first-stage high-altitude building images to generate second-stage high-altitude building images; S23: performing spatial transformation processing on the plurality of second-stage high-altitude building images to generate a plurality of standard high-altitude building images; S24: marking illegal buildings in a plurality of standard high-altitude building images, and generating a plurality of sets of authentic illegal building identification labels and display frames for standard high-altitude building images; S25: Design a vector conversion scheme to process the authenticity illegal building discrimination labels and display frames of multiple sets of standard high-altitude building images, and generate authenticity illegal building discrimination vectors of multiple sets of standard high-altitude building images.

3. A method for identifying illegal buildings using a drone based on a multi-threaded training mechanism according to claim 1 or 2, characterized in that: The step S4 specifically includes the following steps: Step S41: using multiple standard high-altitude building images, multiple sets of predictive illegal building discriminant vectors, and multiple sets of real illegal building discriminant vectors as data samples, and obtaining multiple sets of different parameter gradients based on the loss function of the multiple branch calculation model; Step S42: passing multiple groups of different parameter gradients into the global learning model to update the parameters of the global learning model; Step S43: at fixed intervals t, all parameter values ​​of the global learning model are respectively transferred to the multiple branch calculation models to cover the original parameter values ​​of the branch calculation models, and the multiple branch calculation models obtain new parameter values; Step S44: Repeat steps S33, S41, S42 and S43 until the end time Tend is reached, and obtain the trained illegal building recognition model.

4. A method for identifying illegal buildings by unmanned aerial vehicles based on a multi-threaded training mechanism according to claim 1 or 2, characterized in that: The step S5 specifically includes the following steps: Step S51: inputting the standard high-altitude building image into the trained illegal building recognition model to obtain a predictive illegal building discriminant vector; Step S52: converting the predictive illegal building discriminant vector into the label and display frame of the illegal building in the standard high-altitude building image.

5. The method for identifying illegal buildings by unmanned aerial vehicles based on a multi-threaded training mechanism according to claim 2, characterized in that: The vector conversion scheme in step S25 is: The authenticity illegal building discrimination labels and display frames of multiple groups of standard high-altitude building images are encoded into authenticity illegal building discrimination vectors, where the labels are encoded using a one-hot vector, and the display frames are encoded using the length value, width value, and the horizontal and vertical coordinates of the center point of the display frame.

6. The method for identifying illegal buildings by unmanned aerial vehicles based on a multi-threaded training mechanism according to claim 1, characterized in that: In the step S31, the illegal building recognition model includes: an aggregate feature sub-model, a two-way combined feature sub-model and a feature recognition sub-model.

7. The method for identifying illegal buildings by unmanned aerial vehicles based on a multi-threaded training mechanism according to claim 1, characterized in that: In step S33, the calculation formula for obtaining multiple sets of predictive illegal building discrimination vectors is as follows: ； ； ； ； ； ； ； ； ； ； ； ； ； ； in, x 0 is the pixel matrix of the standard high-altitude building image, w 0 and b 0 are the first weight parameter and the first bias parameter of the aggregate feature sub-model, x 1 is the first intermediate vector of the aggregate feature sub-model, Slice() is the slicing function, Contact() is the cascade function, x 2 is the second intermediate vector of the aggregate feature sub-model, Conv() is the convolution function, BN() is the batch processing function, w 1 and b 1 are the second weight parameter and the second bias parameter of the aggregate feature sub-model, x 3 is the third intermediate vector of the aggregated feature sub-model; y 0 is the first intermediate vector of the forward feature extraction network in the bidirectional combined feature sub-model, z 0 is the first intermediate vector of the reverse feature extraction network in the bidirectional combined feature sub-model, StochasticPooling() is the random pooling function, MaxPooling() is the maximum pooling function, y 1 is the second intermediate vector of the forward feature extraction network in the bidirectional combined feature sub-model, z 1 is the second intermediate vector of the reverse feature extraction network in the bidirectional combined feature sub-model, y 2 is the third intermediate vector of the forward feature extraction network in the bidirectional combined feature sub-model, z 2 is the third intermediate vector of the reverse feature extraction network in the bidirectional combined feature sub-model, and are the first weight parameter and the first bias parameter of the forward feature extraction network in the bidirectional combined feature sub-model, respectively. and are the first weight parameter and the first bias parameter of the reverse feature extraction network in the bidirectional combined feature sub-model, respectively. and are the second weight parameter and the second bias parameter of the forward feature extraction network in the bidirectional combined feature sub-model, respectively. and are the second weight parameter and the second bias parameter of the reverse feature extraction network in the bidirectional combined feature sub-model, y 3 is the fourth intermediate vector of the forward feature extraction network in the bidirectional combined feature sub-model, z 3 is the fourth intermediate vector of the reverse feature extraction network in the bidirectional combined feature sub-model; h 0 is the first intermediate vector of the feature recognition sub-model, h 1 is the second intermediate vector of the feature recognition sub-model, Softmax() is the activation function, h 2 is the predictive illegal building discriminant vector output by the activation function layer in the feature recognition sub-model.

8. The method for identifying illegal buildings by unmanned aerial vehicles based on a multi-threaded training mechanism according to claim 3 is characterized in that: In step S41, the loss function of the branch calculation model is: ； in G is the loss function, k is a subscript whose value range is from 1 to N , N is the number of data samples, is the discriminant vector of the real illegal building, is the discriminant vector of predictive illegal buildings.

9. A drone illegal building identification system based on a multi-threaded training mechanism, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

Images