Building identification method and equipment for fixed-wing UAV route survey

Abstract

The present invention discloses a building identification method and device for fixed-wing UAV line survey, belonging to the technical field of UAV line survey. The identification method comprises: obtaining feature matrix data of an aerial image of a building to be identified, expanding the feature matrix data to generate a first vector; position encoding the feature matrix data to generate a second vector; and classifying according to the first vector and the second vector, thereby completing the type identification of the building to be identified. The present invention performs position encoding on the feature matrix data so that the feature information input to the classifier is more complete and accurate, thereby improving the building type identification effect based on the aerial image of the fixed-wing UAV.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle (UAV) line survey, and in particular relates to a building identification method and equipment for fixed-wing UAV line survey. Background Art

[0002] The use of drones to assist operations has the advantages of low cost, high efficiency and strong safety, and has been widely used in the survey, inspection and maintenance of power grids. There are many factors to consider when surveying power grid lines, and the distribution of various buildings is one of them. Using aerial images collected from high altitudes by fixed-wing drones, the location distribution information of buildings can be quickly obtained. However, the current technology is still relatively inaccurate in automatically identifying building types based on the orthographic projection images obtained by fixed-wing drone aerial photography, and further improvement is needed. Summary of the invention

[0003] In view of this, the present invention provides a building recognition method and device for fixed-wing UAV route survey, so as to improve the accuracy of automatic recognition of building types based on fixed-wing UAV aerial images.

[0004] The technical scheme is as follows: A building identification method for fixed-wing UAV line survey includes the following steps:

[0005] Acquire characteristic matrix data of the aerial image of the building to be identified, expand the characteristic matrix data, and generate a first vector;

[0006] Performing position encoding on the feature matrix data to generate a second vector;

[0007] Classification is performed according to the first vector and the second vector, thereby completing the type identification of the building to be identified.

[0008] Furthermore, the process of position encoding the feature matrix data to generate a second vector includes:

[0009] After calculating the characteristic matrix data in sequence using a plurality of encoding units arranged in series, extracting the encoding matrix output by each encoding unit;

[0010] Performing full-space dimensionality reduction calculation on each of the coding matrices to generate a plurality of first coding vectors;

[0011] Performing a first convolution and a first excitation calculation on each first coding vector respectively to generate a plurality of second coding vectors with the same length;

[0012] Connect all the second encoding vectors to generate a third encoding vector;

[0013] Perform a second convolution and a second excitation calculation on the third encoding vector to generate the second vector;

[0014] The encoding unit includes a sub-pixel convolution layer, a first activation layer, a conventional convolution layer and a second activation layer which are sequentially connected in series. After the sub-pixel convolution layer is calculated, the spatial dimension size of the matrix data becomes the original N 2 times, the convolution kernel size of the conventional convolution layer is N*N, the stride is N, and N is a positive integer greater than 1.

[0015] Further, classification is performed according to the first vector and the second vector, thereby completing the type identification of the building to be identified, including the following steps:

[0016] The length of the first vector is equal to the length of the second vector, and elements in the first vector are multiplied by corresponding elements in the second vector to generate a third vector;

[0017] The third vector is calculated by the fully connected layer and the classifier in sequence to generate a classification result, and the type of the building to be identified is determined according to the classification result.

[0018] Furthermore, the process of obtaining the feature matrix data of the aerial image of the building to be identified includes the following steps:

[0019] The aerial image of the building to be identified is used as input, and multiple feature generators arranged in series are used for sequential calculation, and then the last feature generator generates the feature matrix data.

[0020] Furthermore, the internal calculation process of the feature generator includes the following steps:

[0021] Using a first link calculation unit and a second link calculation unit to process the data input into the feature generator respectively to generate a first link matrix and a second link matrix;

[0022] Performing a first merging process on the first link matrix and the second link matrix to generate a third link matrix;

[0023] Processing the third link matrix using a third link calculation unit and a fourth link calculation unit respectively to generate a fourth link matrix and a fifth link matrix;

[0024] Performing a second merging process on the first link matrix, the second link matrix, the fourth link matrix and the fifth link matrix to generate a sixth link matrix;

[0025] Performing a third merging process on the fourth link matrix, the fifth link matrix and the sixth link matrix to generate a seventh link matrix, wherein the seventh link matrix is ​​data output by the feature generator;

[0026] The first link calculation unit, the second link calculation unit, the third link calculation unit and the fourth link calculation unit each include an internal convolution layer and an internal activation layer which are sequentially connected in series.

[0027] Furthermore, the first link matrix implements a first merging process by adding corresponding elements of the second link matrix.

[0028] Furthermore, the process of performing a second merging process on the first link matrix, the second link matrix, the fourth link matrix and the fifth link matrix to generate a sixth link matrix includes:

[0029] connecting the first link matrix, the second link matrix, the fourth link matrix and the fifth link matrix to generate a first merged matrix;

[0030] The first merge matrix is ​​calculated in sequence through deformable convolution and merge activation layers to generate the sixth link matrix.

[0031] Furthermore, the third merging process is implemented by multiplying corresponding elements of the fourth link matrix, the fifth link matrix and the sixth link matrix.

[0032] In the present invention, the calculation process from the input of the aerial image of the building to be identified to the generation of the classification result by the classifier can be completed by a single neural network, and the neural network usually needs to be trained before the identification operation is performed.

[0033] The present invention also provides a building identification device for fixed-wing UAV line survey, including a computer program / instruction, which implements the method described above when the computer program / instruction is executed.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] Since digital images are composed of a large number of pixel points, the position of the pixel points is an important information of the image. In the existing image classification algorithm, the three-dimensional feature matrix is ​​usually expanded at the end of the network to obtain a one-dimensional vector. The inventor found that a considerable part of the pixel position information will be lost during the expansion process, which reduces the comprehensiveness of feature information extraction. The present application performs position encoding on the feature matrix data and saves the information in the second vector, so that the feature information input to the classifier is more complete and accurate, thereby improving the building type recognition effect based on fixed-wing drone aerial images. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the flow chart of the building identification method of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0038] This embodiment provides a building identification method for fixed-wing UAV route survey, such as Figure 1 As shown, the following steps are included:

[0039] In a building recognition system or device, an aerial image of a building to be identified is used as input, and multiple feature generators arranged in series are used to calculate in sequence, and the last feature generator generates output feature matrix data. The feature matrix data is expanded (which can be achieved through a flatten layer) to generate a first vector; the feature matrix data is position-encoded to generate a second vector; and the first vector and the second vector are classified to complete the type recognition of the building to be identified. The above-mentioned aerial image of the building is a visible light image. When the proportion of space occupied by the building in the image is small, the existing algorithm can be used to first perform target detection, and then segment the image containing the building part to reduce the proportion of the background image and reduce the interference of background information on the recognition.

[0040] The feature generator can be implemented by using many existing modules that can extract image features, such as a residual module and a dense connection module, etc. The number of feature generators can be set according to actual conditions. As an example but not limiting, in this embodiment, the number of feature generators is set to five.

[0041] The specific calculation process of the above-mentioned multiple feature generators is as follows: the first feature generator takes the aerial image of the building to be identified as input, and generates and outputs the first node data at the end of the first feature generator after calculation (the first node data is also the seventh link matrix generated in the first feature generator), the second feature generator takes the first node data as input, and generates and outputs the second node data at the end of the second feature generator after calculation (the second node data is also the seventh link matrix generated in the second feature generator), and so on, the next feature generator takes the data output at the end of the previous feature generator as input, until the fifth feature generator generates and outputs the fifth node data at the end (the fifth node data is also the seventh link matrix generated in the fifth feature generator), and the fifth node data is the feature matrix data.

[0042] As a preferred embodiment, in view of the characteristics of aerial images of fixed-wing drones, the present invention designs a selective merging mechanism across spatial domains within the feature generator to enhance the recognition and processing capabilities of random local features. Specifically, the calculation process within the feature generator of this embodiment includes the following steps:

[0043] The data input to the feature generator are processed by a first link calculation unit and a second link calculation unit respectively to generate a first link matrix and a second link matrix;

[0044] Performing a first merging process on the first link matrix and the second link matrix to generate a third link matrix;

[0045] Processing the third link matrix using a third link calculation unit and a fourth link calculation unit respectively to generate a fourth link matrix and a fifth link matrix;

[0046] Performing a second merging process on the first link matrix, the second link matrix, the fourth link matrix and the fifth link matrix to generate a sixth link matrix;

[0047] The fourth link matrix, the fifth link matrix and the sixth link matrix are combined for the third time to generate the seventh link matrix, which is the node data output at the end of the feature generator. Therefore, the fifth node data is the seventh link matrix generated by the fifth feature generator through calculation.

[0048] The first link calculation unit, the second link calculation unit, the third link calculation unit and the fourth link calculation unit all include internal convolution layers and internal activation layers which are sequentially connected in series. The internal activation layer can select functions such as ReLU, logistics, LeakyReLU or Tanh according to actual conditions. The internal convolution layer is an existing ordinary convolution layer. In this embodiment, the convolution kernel size of the internal convolution layer in the first link calculation unit is 3*3 and the stride is 2, the convolution kernel size of the internal convolution layer in the second link calculation unit is 5*5 and the stride is 2, the convolution kernel size of the internal convolution layer in the third link calculation unit is 3*3 and the stride is 1, and the convolution kernel size of the internal convolution layer in the fourth link calculation unit is 5*5 and the stride is 1.

[0049] Preferably, the first link matrix is ​​equal in size to the second link matrix, and the first link matrix implements the first merging process by adding the corresponding elements of the second link matrix. The fourth link matrix, the fifth link matrix and the sixth link matrix are equal in size, and the fourth link matrix, the fifth link matrix and the sixth link matrix are multiplied by the corresponding elements to implement the third merging process. The process of performing the second merging process on the first link matrix, the second link matrix, the fourth link matrix and the fifth link matrix to generate the sixth link matrix includes: connecting (also called concatenation, English concat) the first link matrix, the second link matrix, the fourth link matrix and the fifth link matrix to generate the first merging matrix; the first merging matrix is ​​calculated by deformable convolution and merge activation layer in sequence to generate the sixth link matrix. As an example, the convolution kernel size of the deformable convolution can be 3*3, and the merge activation layer can be LeakyReLU.

[0050] Furthermore, the process of position encoding the feature matrix data to generate the second vector includes:

[0051] After calculating the characteristic matrix data in sequence using multiple encoding units arranged in series, the encoding matrices output by each encoding unit are extracted. Preferably, there are three encoding units in this embodiment, and the specific operation process is: the characteristic matrix data is input into the first encoding unit, the first encoding unit generates a first encoding matrix after calculation, the second encoding unit uses the first encoding matrix as input, generates a second encoding matrix after calculation, and the third encoding unit uses the second encoding matrix as input, generates a third encoding matrix after calculation. The extracted encoding matrices are the first encoding matrix, the second encoding matrix, and the third encoding matrix mentioned above.

[0052] Full-space dimensionality reduction calculation is performed on each coding matrix respectively to generate three first coding vectors. As an example but not limiting, this embodiment performs global average pooling processing on each coding matrix along the spatial direction to achieve full-space dimensionality reduction calculation on each coding matrix.

[0053] Perform the first convolution and the first excitation calculation on each first coding vector respectively to generate three second coding vectors of the same length; the convolution kernel size of the first convolution is 1*1, and the first excitation can be performed using functions such as ReLU, logistics, LeakyReLU or Tanh.

[0054] All the second coding vectors are connected (also called concatenation) in sequence to generate a one-dimensional third coding vector.

[0055] Perform a second convolution and a second excitation calculation on the third encoded vector to generate a second vector; the convolution kernel size of the second convolution is 1*1, and the second excitation can be performed using functions such as ReLU, logistics, LeakyReLU or Tanh.

[0056] The encoding unit includes a sub-pixel convolution layer, a first activation layer, a conventional convolution layer, and a second activation layer, which are arranged in series. The first activation layer and the second activation layer can select ReLU, logistics, LeakyReLU, or Tanh functions for activation processing according to actual conditions. After the sub-pixel convolution layer is calculated, the spatial dimension size of the matrix data becomes the original N 2 times (the width and height become N times of the original, and the number of channels becomes 1 / N of the original 2 ), the convolution kernel (width * height) size of the conventional convolution layer is N*N, the stride is N, and N is a positive integer greater than 1. When the conventional convolution layer performs convolution operations, no padding is performed on the matrix data. In this way, for the same coding unit, the size of the matrix space dimension input to the coding unit is equal to the size of the matrix space dimension output by the coding unit, and the size of the matrix channel dimension input to the coding unit is N times the size of the matrix channel dimension output by the coding unit.2 times.

[0057] After adopting the above position coding scheme, in the process of position coding the feature matrix data, multiple sub-pixel convolution layers and conventional convolution layers are alternately set to cooperate to achieve the alternating compression of channel dimension and spatial dimension. In this process, the position information of the pixel is transcribed into multiple coding matrices, and then the coding matrices are integrated and dimensionally transformed, and the position information is further transcribed into the second vector. Through the above multi-dimensional hierarchical position coding mechanism, the position information loss and feature confusion caused by the expansion process are well compensated, and the comprehensiveness and accuracy of the acquisition of image feature information are improved.

[0058] Classifying the building to be identified according to the first vector and the second vector, thereby completing the type identification of the building to be identified, includes the following steps:

[0059] The length of the first vector is equal to the length of the second vector, and the elements in the first vector are multiplied by the corresponding elements in the second vector to generate a third vector;

[0060] After the third vector is calculated by the fully connected layer and the classifier in sequence, a classification result is generated, and the type of the building to be identified is determined according to the classification result.

[0061] The number of output nodes of the fully connected layer can be set according to the number of categories that need to be classified, and the classifier can select the softmax classifier for processing. For example, in the aerial images collected in non-urban areas, the following types of buildings usually need to be automatically identified: ordinary houses, schools, factories, plastic greenhouses, gas stations and communication base stations, etc. After the identification is completed, in the subsequent data processing and processing process, the category information of these identified buildings can be automatically marked at the corresponding position of the image, which greatly improves the convenience for staff to obtain the distribution information of buildings in the target area and improves the efficiency of line survey and planning.

[0062] Finally, it should be noted that the above description is only a preferred embodiment of the present invention. Under the guidance of the present invention, ordinary technicians in this field can make various similar expressions without violating the purpose and claims of the present invention, and such changes all fall within the scope of protection of the present invention.

Claims

1. A building identification method for fixed-wing UAV route survey, characterized in that: The following steps are involved: Acquire characteristic matrix data of the aerial image of the building to be identified, expand the characteristic matrix data, and generate a first vector; Performing position encoding on the feature matrix data to generate a second vector; Classification is performed according to the first vector and the second vector, thereby completing the type identification of the building to be identified.

2. The building identification method for fixed-wing UAV line survey according to claim 1 is characterized in that: The process of performing position encoding on the feature matrix data to generate a second vector includes: After calculating the characteristic matrix data in sequence using a plurality of encoding units arranged in series, extracting the encoding matrix output by each encoding unit; Performing full-space dimensionality reduction calculation on each of the coding matrices to generate a plurality of first coding vectors; Performing a first convolution and a first excitation calculation on each first coding vector respectively to generate a plurality of second coding vectors with the same length; Connect all the second encoding vectors to generate a third encoding vector; Perform a second convolution and a second excitation calculation on the third encoding vector to generate the second vector; The encoding unit includes a sub-pixel convolution layer, a first activation layer, a conventional convolution layer and a second activation layer which are sequentially connected in series. After the sub-pixel convolution layer is calculated, the spatial dimension size of the matrix data becomes the original N 2 times, the convolution kernel size of the conventional convolution layer is N*N, the stride is N, and N is a positive integer greater than 1.

3. The building identification method for fixed-wing UAV line survey according to claim 1 is characterized in that: Classifying according to the first vector and the second vector, thereby completing the type identification of the building to be identified, includes the following steps: The length of the first vector is equal to the length of the second vector, and elements in the first vector are multiplied by corresponding elements in the second vector to generate a third vector; The third vector is calculated by the fully connected layer and the classifier in sequence to generate a classification result, and the type of the building to be identified is determined according to the classification result.

4. The building identification method for fixed-wing UAV line survey according to claim 1 is characterized in that: The process of obtaining the feature matrix data of the aerial image of the building to be identified includes the following steps: The aerial image of the building to be identified is used as input, and multiple feature generators arranged in series are used for sequential calculation, and then the last feature generator generates the feature matrix data.

5. The building identification method for fixed-wing UAV line survey according to claim 4 is characterized in that: The internal calculation process of the feature generator includes the following steps: Using a first link calculation unit and a second link calculation unit to process the data input into the feature generator respectively to generate a first link matrix and a second link matrix; Performing a first merging process on the first link matrix and the second link matrix to generate a third link matrix; Processing the third link matrix using a third link calculation unit and a fourth link calculation unit respectively to generate a fourth link matrix and a fifth link matrix; Performing a second merging process on the first link matrix, the second link matrix, the fourth link matrix and the fifth link matrix to generate a sixth link matrix; Performing a third merging process on the fourth link matrix, the fifth link matrix and the sixth link matrix to generate a seventh link matrix, wherein the seventh link matrix is ​​data output by the feature generator; The first link calculation unit, the second link calculation unit, the third link calculation unit and the fourth link calculation unit each include an internal convolution layer and an internal activation layer which are sequentially connected in series.

6. The building identification method for fixed-wing UAV line survey according to claim 5 is characterized in that: The first link matrix implements a first merging process by adding corresponding elements of the second link matrix.

7. The building identification method for fixed-wing UAV line survey according to claim 5 is characterized in that: The process of performing a second merging process on the first link matrix, the second link matrix, the fourth link matrix and the fifth link matrix to generate a sixth link matrix includes: connecting the first link matrix, the second link matrix, the fourth link matrix and the fifth link matrix to generate a first merged matrix; The first merge matrix is ​​calculated in sequence through deformable convolution and merge activation layers to generate the sixth link matrix.

8. The building identification method for fixed-wing UAV line survey according to claim 5, characterized in that: The third merging process is implemented by multiplying corresponding elements of the fourth link matrix, the fifth link matrix and the sixth link matrix.

9. Building identification equipment for fixed-wing drone line survey, characterized in that: The method comprises a computer program / instruction, and when the computer program / instruction is executed, the method according to any one of claims 1 to 8 is implemented.

Images