A method and device for road vehicle recognition based on UAV aerial images

Abstract

The present invention relates to the field of computer vision, in particular to a method and device for road vehicle recognition based on drone aerial images. Aiming at the problem that unified object detection models cannot be used due to excessive differences in target scales in images collected by drones at different heights, the present invention adopts the idea of dividing large into small. According to the multiple relationship between the target scale in the dataset to be detected and the target scale domain of the detection model, the large image is regularly divided into small images, so that the size of the vehicle to be recognized is controlled within a suitable range, and then the results are merged and de-duplicated to obtain the final detection result. The present invention effectively solves the problem of the relative sizes of the image and the key object in the image, and is also beneficial to improving the overall recognition accuracy and the generalization of the algorithm to different scenarios, and can adapt to the processing of drone aerial images at different heights. At the same time, the method of the present invention also reduces the requirement for the number of database pictures in the model construction process.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and device for road vehicle recognition based on drone aerial images. Background Art

[0002] With the rapid development of China's highway transportation industry and the rapid increase in the number of automobiles, the problems of highway traffic safety and traffic efficiency have become increasingly severe. Traditional road monitoring systems based on video surveillance and vehicle detectors provide means for road operation management departments to perceive real-time road conditions and events. However, their installation positions are fixed, and the spatial limitations of their perception capabilities are relatively large. In recent years, civilian rotor drone technology has developed rapidly and become popular. Based on its characteristics of strong flexibility, high mobility, and large observation range, it can break through the spatial limitations of traditional road monitoring means and be used as an important means for mobile perception of key monitoring sections and on-site investigation and evidence collection in case of events.

[0003] However, there is a negative correlation between the altitude of drone aerial photography and the size of vehicles in the target image or video. If the altitude of drone aerial photography is too high, the vehicle targets in the captured video images will be too small. When the image is processed by a deep learning algorithm, the picture will be pre-processed and scaled into an image with a consistent scale. During the scaling process, the target size will also become smaller and a lot of pixel details will be lost, resulting in a decrease in the accuracy of target recognition. By increasing the input image size of the detection model to maintain the pixel details of the vehicle target, a large amount of computing power will be consumed during the target recognition process. At the same time, when keeping the input size of the detection model constant and increasing the pixel details of the vehicle target by reducing the altitude of drone aerial photography, the field of view of the drone will also decrease, that is, some advantages of aerial photography will be lost. Currently, many popular object detection algorithms support multi-scale detection. When the multi-scale targets in the database samples are covered sufficiently, the problem of recognizing targets of different sizes in the image can be solved. However, when the vehicle target scales in the database samples are single or the scale distribution is unbalanced, it is easy to cause a decrease in the performance and accuracy of the algorithm in actual applications.

[0004] Therefore, there is an urgent need for a method to detect multi-scale targets in drone aerial images under the condition that the target scales detectable by the target detection model are relatively single. Summary of the Invention

[0005] The purpose of the present invention is to overcome the problem in the prior art that the ratio of the target size detectable by the detection model to the image size is single, and to provide a method for road vehicle recognition in drone aerial images within a wide acquisition altitude range.

[0006] In order to achieve the above-mentioned invention purpose, the present invention provides the following technical solutions:

[0007] A method for road vehicle recognition based on drone aerial images, comprising the following steps:

[0008] S1: Obtain the aerial photo set to be detected, and randomly select a photos from the aerial photo set to construct a sub-dataset; the aerial photo set is generated by an unmanned aerial vehicle (UAV), and a is a positive integer;

[0009] S2: Obtain the pixel area of the road vehicles in the sub-dataset, compare it with the pixel area of the vehicles in the training image database of the vehicle target detection model, and calculate the segmentation multiple s; uniformly generate s - 1 horizontal segmentation marking lines and s - 1 vertical segmentation marking lines on the image to be detected, and generate two horizontal segmentation lines parallel to each horizontal segmentation marking line in the vertical direction thereof, and generate two segmentation lines parallel to each vertical segmentation marking line in the horizontal direction thereof;

[0010] Wherein, the vehicle target detection model is used for target detection of vehicle targets in S4, and the distance between the horizontal / vertical segmentation line and its corresponding horizontal / vertical segmentation marking line is a preset value;

[0011] S3: Segment the image to be detected along the horizontal segmentation lines and the vertical segmentation lines to obtain s * s sub-images that overlap each other;

[0012] S4: Respectively perform vehicle target detection on each sub-image, then convert the detection result to the reference system of the image to be detected, and store it in the non-overlapping area recognition result set and the overlapping area recognition result set; wherein, the detection results of the non-overlapping areas are stored in the non-overlapping area recognition result set, and the detection results of the overlapping areas are stored in the overlapping area recognition result set;

[0013] S: After removing duplicates from the detection results in the overlapping area recognition result set through the Hungarian algorithm, merge them with the non-overlapping area recognition result set to generate the recognition result of the image to be detected.

[0014] 2. The method for identifying road vehicles based on UAV aerial photos according to claim 1, wherein the calculation formula for the segmentation multiple s in S2 is:

[0015]

[0016] Wherein, A1 is the pixel area of the images in the sub-dataset, AC1 is the pixel area of the rectangular frames of the target vehicles in the sub-dataset, A2 is the pixel area of the images in the training image database of the vehicle target detection model, and AC2 is the pixel area of the rectangular frames of the target vehicles in the training image database of the vehicle target detection model, represents the ceiling operation.

[0017] As a preferred solution of the present invention, the calculation formula for the preset value of the distance between the dividing line and the dividing mark line in S2 is:

[0018] Δ = s × max(w2, h2),

[0019] where Δ is the distance between the horizontal dividing line and its corresponding horizontal dividing mark line and the vertical dividing line and its corresponding vertical dividing mark line, and w2 and h2 are respectively the average width and average height of the rectangular frames of large vehicles in the sub-dataset.

[0020] As a preferred solution of the present invention, the expression for the area where each sub-graph is located in S3 is:

[0021]

[0022] where θ mn is the set of points of the m-th sub-graph in the horizontal direction and the n-th sub-graph in the vertical direction, (x, y) is the coordinate of a certain point in the set θ mn ; w and h are respectively the width and height of the image to be detected, and Z is the set of integers;

[0023] The expression for the horizontal overlapping area generated by the horizontal dividing line is:

[0024]

[0025] where is the set of points on the horizontal overlapping area of the i-th horizontal dividing line, and (x, y) is the coordinate of a certain point in the set , and 1 ≤ i ≤ s - 1, i ∈ Z.

[0026] The expression for the vertical overlapping area generated by the vertical dividing line is:

[0027]

[0028] where is the set of points on the vertical overlapping area of the j-th vertical dividing line, and (x, y) is the coordinate of a certain point in the set ; 1 ≤ j ≤ s - 1, j ∈ Z.

[0029] As a preferred solution of the present invention, the vehicle target detection model for target detection of vehicles in S4 is the yolov5 model.

[0030] As a preferred solution of the present invention, the expression for converting the detection result of the sub-graph into the coordinate system of the image to be detected in S4 is:

[0031] TAR = η + tar

[0032] Among them, tar is the detection result of the sub-graph before conversion, tar = [x top , y top , x bottom , y bottom , c, p] T , (x top , y top ) is the position coordinate of the upper left corner boundary point on the diagonal of the target recognition rectangle in the sub-graph, (x bottom, , y bottom ) is the position coordinate of the lower right corner boundary point on the diagonal of the target recognition rectangle in the sub-graph, c is the category of target recognition, and p is the probability of recognition of this category; TAR is the detection result after conversion, TAR = [X top , Y top , X bottom , Y bottom , C, P] T , (X top , Y top ) is the position coordinate of the upper left corner boundary point on the diagonal of the target recognition rectangle in the to-be-detected image, (X bottom , Y bottom ) is the position coordinate of the lower right corner boundary point on the diagonal of the target recognition rectangle in the to-be-detected image, C is the category of target recognition, and P is the probability of recognition of this category; η is the conversion offset matrix, and its expression is as follows:

[0033]

[0034] Among them, m and n are the serial numbers of the sub-graph from left to right and from top to bottom respectively, represents the floor function operation.

[0035] As a preferred solution of the present invention, the tar of the detection results in the overlapping area in each sub-graph satisfies:

[0036]

[0037] Or:

[0038]

[0039] Among them, is the position coordinate of the upper left corner boundary point on the diagonal of the corresponding overlapping area in the sub-graph, is the position coordinate of the lower right corner boundary point on the diagonal of the corresponding overlapping area in the sub-graph.

[0040] As a preferred solution of the present invention, the S5 includes:

[0041] S51: Sequentially match the targets that overlap in the overlapping region recognition result set in the order from left to right and then from top to bottom; the matching process of the overlapping targets is as follows: calculate the cost matrix using IOU and category, and find the optimal overlapping match of two overlapping regions through the Hungarian algorithm;

[0042] S52: After fusing the overlapping targets, merge them with the non - overlapping region recognition result set, and generate the recognition result of the to - be - detected image.

[0043] As a preferred solution of the present invention, the calculation formula of the matching value is:

[0044] T = IOU×R,

[0045] where T is the matching value, IOU is used to judge whether there is an union of the target rectangles of two elements, the detection results of the two elements are tar1 and tar2 respectively, IOU = S ∩ / S ∪ , 0≤IOU≤1, S ∩ is the intersection of the detection result rectangles of the two elements, S ∪ is the union of the detection result rectangles of the two elements, and R is used to judge whether the two elements are of the same category, c1 and c2 are the categories of tar1 and tar2 respectively.

[0046] A road vehicle recognition device based on an unmanned aerial vehicle (UAV) aerial image includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in any one of the above.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] The present invention adopts the idea of dividing large into small. According to the multiple relationship between the target scale in the to - be - detected data set and the target scale domain of the detection model, the large image is regularly segmented into small images, so that the size of the recognized vehicle is controlled within a suitable range, and then the results are merged and de - duplicated to obtain the final detection result. The present invention effectively solves the problem of the relative size between the image and the key targets in the image, and is also beneficial to improving the overall recognition accuracy and the generalization of the algorithm to different scenarios, and can adapt to the processing of UAV aerial images at different heights. At the same time, the method of the present invention also reduces the requirement for the number of database pictures in the process of building the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic flowchart of a road vehicle recognition method based on an unmanned aerial vehicle (UAV) aerial image according to an embodiment of the present invention;

[0050] Figure 2 Schematic diagram of the aerial photo img1 in a method for identifying road vehicles based on drone aerial images according to Embodiment 3 of the present invention;

[0051] Figure 3 Schematic diagram of segmentation in step S3 of a method for identifying road vehicles based on drone aerial images according to Embodiment 3 of the present invention;

[0052] Figure 4 Schematic diagram of the relative positions of the sub - graphs and the original graph after segmentation in a method for identifying road vehicles based on drone aerial images according to Embodiment 3 of the present invention;

[0053] Figure 5 Schematic diagram of the vehicle recognition effects of multiple sub - graphs in step S4 of a method for identifying road vehicles based on drone aerial images according to Embodiment 3 of the present invention;

[0054] Figure 6 Schematic diagram of the recognition result after summarization in step S5 of a method for identifying road vehicles based on drone aerial images according to Embodiment 3 of the present invention;

[0055] Figure 7 Schematic diagram of the structure of a road vehicle recognition device based on drone aerial images using a method for identifying road vehicles based on drone aerial images according to Embodiment 1 or 2 of the present invention in Embodiment 4 of the present invention. Detailed implementation manners

[0056] The present invention will be further described in detail below in conjunction with test examples and specific implementation manners. However, this should not be understood as limiting the scope of the above - mentioned subject matter of the present invention to the following embodiments. Any technology implemented based on the content of the present invention belongs to the scope of the present invention.

[0057] Embodiment 1

[0058] As Figure 1 shown, a method for identifying road vehicles based on drone aerial images includes the following steps:

[0059] S1: Obtain an aerial photo set, and randomly select a (a is a positive integer) photos from the aerial photo set to construct a sub - data set; the aerial photo set is generated by a drone, and the drone can be a rotor drone;

[0060] S2: Obtain the pixel area of the road vehicles in the sub-dataset, compare it with the vehicle pixel area in the training image database of the vehicle target detection model, calculate the segmentation multiple s, uniformly generate s - 1 horizontal segmentation marking lines and s - 1 vertical segmentation marking lines on the to-be-detected image, and generate two horizontal segmentation lines parallel to the horizontal segmentation marking line in the vertical direction of each horizontal segmentation marking line, and generate two segmentation lines parallel to the vertical segmentation marking line in the horizontal direction of the vertical segmentation marking line;

[0061] Among them, the vehicle target detection model is used for target detection of vehicle targets in S4, and the distance between the horizontal / vertical segmentation line and its corresponding horizontal / vertical segmentation marking line is a preset value;

[0062] S3: Segment the to-be-detected image along the horizontal segmentation line and the vertical segmentation line to obtain s * s overlapping sub-images;

[0063] S4: Respectively perform vehicle target detection on each sub-image, then convert the detection result to the reference system of the to-be-detected image, and store it in the non-overlapping area recognition result set and the overlapping area recognition result set; among them, the detection results of non-overlapping areas are stored in the non-overlapping area recognition result set, and the detection results of overlapping areas are stored as the overlapping area recognition result set;

[0064] S5: After de-duplicating the detection results in the overlapping area recognition result set through the Hungarian algorithm, merge them with the non-overlapping area recognition result set to generate the recognition result of the to-be-detected image.

[0065] Embodiment 2

[0066] This embodiment is a specific implementation manner of the method described in Embodiment 1, including the following steps:

[0067] S1: Obtain an aerial photo set, and randomly select a photos from the aerial photo set to construct a sub-dataset; the aerial photo set is generated by an unmanned aerial vehicle (UAV) shooting.

[0068] In this embodiment, an UAV is used to take aerial photos of a highway where vehicles are traveling at a height of 50m - 500m to obtain an aerial video set. After decoding the aerial video set, an aerial photo set imgs0 is obtained, and 50 photos (at this time a = 50) of the photo set imgs0 are randomly selected to form a sub-dataset imgs1.

[0069] S2: Obtain the pixel area of road vehicles in the sub-dataset, compare it with the vehicle pixel area in the training image database of the vehicle target detection model, and calculate the segmentation multiple s; uniformly generate s - 1 horizontal segmentation marking lines and s - 1 vertical segmentation marking lines on the to-be-detected image, and generate two horizontal segmentation lines parallel to the horizontal segmentation marking lines in the vertical direction of each horizontal segmentation marking line, and generate two segmentation lines parallel to the vertical segmentation marking lines in the horizontal direction of the vertical segmentation marking lines. Specifically, it includes the following steps:

[0070] S21: Calculate and obtain the aerial photo segmentation multiple s by comparing the average value of the rectangular pixel areas of vehicle targets in the existing image dataset with the rectangular pixel dimensions of vehicle targets in imgs1. The calculation formula for the segmentation multiple s is:

[0071]

[0072] where A1 is the pixel area of the image in imgs1, AC1 is the pixel area of the rectangular frame of the target vehicle in imgs1, A2 is the image pixel area of the vehicle target detection model training image database, and AC2 is the pixel area of the rectangular target vehicle in the vehicle target detection model training image database, represents the ceiling operation. The vehicle target detection model is used for target detection of vehicle targets in S4. The existing yolov5 model is adopted, and the model training is pre-completed through the training image database.

[0073] S22: Convert the average maximum side length Δ of the length and width of large vehicle targets in imgs1 through the segmentation multiple s. Its calculation formula is:

[0074] Δ = s × max(w2, h2),

[0075] where Δ is the distance between the horizontal and vertical segmentation lines and their corresponding horizontal and vertical segmentation marking lines, and w2 and h2 are the average width and average height of the rectangular frames of large vehicles in imgs1 respectively.

[0076] S23: Take the image to be detected img1 from the aerial image set imgs0, with its size being (w, h); evenly divide img1 into s parts along the horizontal direction to obtain s - 1 horizontal division marking lines, expand Δ distance to both sides along the vertical direction of each horizontal division marking line within the img1 image, and obtain two parallel horizontal division lines on both sides of the horizontal division marking line in the vertical direction. Continue to evenly divide img1 into s parts along the vertical direction to obtain s - 1 vertical division marking lines, expand Δ distance to both sides along the horizontal direction of each vertical division marking line within the img1 image, and obtain two parallel vertical division lines on both sides of the vertical division marking line in the horizontal direction. Denote the region between the two parallel division lines generated by translating a certain horizontal or vertical division marking line i or j in the above steps as and

[0077] S3: Divide the image to be detected along the horizontal division line and the vertical division line to obtain s * s overlapping sub - images; and denote the sub - image at the m - th in the horizontal direction and the n - th in the vertical direction as θ mn .

[0078] Specifically, the expression for the region where each sub - image is located is:

[0079]

[0080] where θ mn is the set of points of the sub - image at the m - th in the horizontal direction and the n - th in the vertical direction, (x, y) is the coordinate of a certain point in the set θ mn , w and h are the width and height of the image to be detected respectively, and Z is the set of integers.

[0081] The expression for the horizontal overlapping region generated by the horizontal division line is:

[0082]

[0083] where is the set of points on the horizontal overlapping region of the i - th horizontal division line, (x, y) is the coordinate of a certain point in the set , 1 ≤ i ≤ s - 1, i ∈ Z.

[0084] The expression for the vertical overlapping region generated by the vertical division line is:

[0085]

[0086] where is the set of points on the vertical overlapping region of the j - th vertical division line, (x, y) is the coordinate of a certain point in the set The coordinates of a certain point; 1 ≤ j ≤ s - 1, j ∈ Z.

[0087] S4: Perform vehicle target detection on each sub - graph respectively, then convert the detection results to the reference system of the to - be - detected image, and store them in the non - overlapping area recognition result set and the overlapping area recognition result set; among them, the detection results of the non - overlapping areas are stored in the non - overlapping area recognition result set, and the detection results of the overlapping areas are stored as the overlapping area recognition result set; in S4, the yolov5 model is used to perform target detection on vehicles. Specifically, it includes the following steps:

[0088] S41: For each sub - graph θ mn Use the yolov5 target detection algorithm to detect vehicles, and save and mark the recognition result sets of each sub - graph as res mn . Generally, the output format of the yolov5 target detection algorithm is as follows: [x top , y top , x bottom , y bottom , c, p]; where (x top , y top ), (x bottom ,, y bottom ) are the position coordinates of the upper - left and lower - right points of the diagonal of the target recognition rectangle box, c is the finally recognized category, and p is the probability that this category is recognized.

[0089] S42: Convert all elements in res mn from the sub - graph reference system to the img1 reference system. Among them, the expression for converting the detection result of the sub - graph to the coordinate system in the to - be - detected image is:

[0090] TAR = η + tar,

[0091] where tar is the detection result of the sub - graph before conversion, tar = [x top , y top , x bottom , y bottom , c, p] T , (x top , y top ) is the position coordinate of the upper - left boundary point on the diagonal of the target recognition rectangle box in the sub - graph, (x bottom, y bottom ) is the position coordinate of the lower - right boundary point on the diagonal of the target recognition rectangle box in the sub - graph, c is the category of target recognition, and p is the probability that this category is recognized; TAR is the detection result after conversion, TAR = [X top , Y top , X bottom , Y bottom , C, P]T , (X top , Y top ) is the position coordinate of the upper left boundary point on the diagonal of the target recognition rectangle in the image to be detected, (X bottom, Y bottom ) is the position coordinate of the lower right boundary point on the diagonal of the target recognition rectangle in the image to be detected, C is the category of target recognition, and P is the probability of recognizing this category; η is the conversion bias matrix, and its expression is as follows:

[0092]

[0093] where m and n are the serial numbers of the subgraphs from left to right and from top to bottom respectively, represents the floor operation.

[0094] S43: Screen the vehicle targets in each result set res mn in the overlapping area and , and denote it as the overlapping area recognition result set

[0095] [[ID=3l]]The tar of the detection result of the overlapping area in each subgraph satisfies:

[0096]

[0097] Or:

[0098]

[0099] where is the position coordinate of the upper left boundary point on the diagonal of the corresponding overlapping area in the subgraph, is the position coordinate of the lower right boundary point on the diagonal of the corresponding overlapping area in the subgraph.

[0100] S5: After de-duplicating the detection results in the overlapping area recognition result set through the Hungarian algorithm, merge them with the non-overlapping area recognition result set to generate the recognition result of the image to be detected.

[0101] S51: Match the overlapping targets in the overlapping area recognition result set in the order from left to right and then from top to bottom in the horizontal and vertical directions; among them, the matching process of the overlapping targets is: traverse the elements of adjacent subgraphs in the overlapping area recognition result set in turn, and then calculate the cost matrix through IOU and category, that is, calculate the matching values pairwise to form the benefit matrix, and convert the calculation to the cost matrix; finally, use the Hungarian algorithm to calculate the optimal matching result. The result set of the overlapping area of two adjacent subgraphs can be represented as a matrix. If their shapes are different, they can be filled with 0.

[0102] S52: After fusing the overlapping targets, merge them with the non-overlapping region recognition result set, and generate the recognition result of the to-be-detected image. Among them, for the mutually matched result bounding boxes, the minimum bounding rectangle is taken as the new bounding box after fusion, and the classification category remains unchanged, and the classification probability takes the average of the two, and a new classification target is generated by fusion and added to the final result set.

[0103] The calculation formula of the matching value is:

[0104] T = IOU × R,

[0105] where T is the matching value, IOU is used to judge whether there is an intersection of the target bounding boxes of two elements, the detection results of the two elements are tar1 and tar2 respectively, IOU = S ∩ / S ∪ , 0 ≤ IOU ≤ 1, S ∩ is the intersection of the corresponding detection results of the two elements, and S ∪ is the union of the corresponding detection results of the two elements. R is used to judge whether the two elements are of the same category, c1 and c2 are the categories of tar1 and tar2 respectively.

[0106] Example 3

[0107] This example is a practical application example of the method described in Example 2. The recognition of small target vehicles is carried out by using a drone to aerial photograph a highway at a height of 50 - 500m. The specific steps are as follows:

[0108] S1: A drone aerial photographs the highway where the vehicle is traveling at a height of 50m - 500m to obtain a video set. The video set is decoded to obtain a picture set imgs0. The aerial photographs are as Figure 2 shown. Then, 50 pictures are randomly selected from the picture set imgs0 to form a sub-data set imgs1.

[0109] S2: Statistically calculate the average of the length and width dimensions of the vehicle target rectangles in the sub-data set imgs1 and the data set for training the yolov5 object detection model. The results are shown in Table 1 below:

[0110] Table 1 Target Size Statistics

[0111] Item Compact car Large vehicle Image pixel Existing image dataset (307,108) (424,121) (3840,2160) Sub-dataset imgs1 (57,24) (124,35) (3840,2160)

[0112] It can be calculated that the segmentation multiple s of the picture is 4, and Δ is 124.

[0113] S3: Segment the original picture. The annotation lines and segmentation lines for segmentation are as Figure 3As shown (where the dashed line is the segmentation marker line, and the rectangle formed by the solid line is the actual segmentation line for generating a certain sub - figure), the relative positions of the sub - figures and the original figure after segmentation are as Figure 4 shown, and adjacent sub - figures have overlapping areas.

[0114] S4: Use the yolov5 object detection algorithm with an input size of 640*640 to detect the target vehicles in each sub - figure. The detection results are as follows Figure 5 shown (i.e., the display of the vehicle recognition effects of multiple sub - figures).

[0115] S5: For the recognition results of each sub - figure, first screen the target vehicles located in the overlapping areas of adjacent sub - figures. Use IOU and category judgment to construct a cost matrix for calculation. It is recommended to use the Hungarian algorithm linear_sum_assignment in the scipy.optimize module of Python to find the best match, de - duplicate and fuse the mutually matched targets. Finally, summarize the detection results of all sub - figures to obtain the detection result of the original figure. The result is as Figure 6 shown. It can be seen that the method described in the present invention can well recognize the vehicle targets on the road aerial images, and solve the problem that the height of the drone aerial photography is relatively high and the vehicles in the image are too small from the perspective of image post - processing.

[0116] Embodiment 4

[0117] As Figure 7 shown, a road vehicle recognition device based on drone aerial images includes at least one processor, a memory communicatively connected to the at least one processor, and at least one input - output interface communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for road vehicle recognition based on drone aerial images described in the foregoing embodiments. The input - output interface may include a display, a keyboard, a mouse, and a USB interface for inputting and outputting data.

[0118] Those skilled in the art can understand that all or part of the steps to implement the above - mentioned method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer - readable storage medium. When the program is executed, it executes the steps including the above - mentioned method embodiments; and the foregoing storage medium includes: mobile storage devices, read - only memory (ROM), magnetic disks, or optical disks and other various media that can store program codes.

[0119] When the above integrated units of the present invention are implemented in the form of software functional units and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present invention, in essence or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as removable storage devices, ROM, magnetic disks, or optical discs that can store program codes.

[0120] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying road vehicles based on drone aerial images, characterized in that: The following steps are involved: S1: Obtain a set of aerial pictures to be detected, and randomly extract a pictures from the set to construct a sub-dataset; the set of aerial pictures is generated by taking pictures with a drone, and a is a positive integer; S2: Obtain the pixel area of the road vehicles in the sub-dataset, compare it with the vehicle pixel area in the training image database of the vehicle target detection model, and calculate the segmentation multiple s; evenly generate s-1 horizontal segmentation marking lines and s-1 vertical segmentation marking lines on the image to be detected, and generate two horizontal segmentation lines parallel to the horizontal segmentation marking lines in the vertical direction of each horizontal segmentation marking line, and generate two segmentation lines parallel to the vertical segmentation marking lines in the horizontal direction of the vertical segmentation marking lines; The vehicle target detection model is used to detect the vehicle target in S4, and the distance between the horizontal / vertical dividing line and the corresponding horizontal / vertical dividing mark line is a preset value; the preset value of the distance between the dividing line and the dividing mark line is calculated as follows: ， Wherein, Δ is the distance between the horizontal segmentation line and its corresponding horizontal segmentation mark line and the vertical segmentation line and its corresponding vertical segmentation mark line, w2 and h2 are the average width and average height of the rectangular boxes of large vehicles in the sub-dataset respectively; S3: Segment the image to be detected along the horizontal segmentation line and the vertical segmentation line to obtain s*s overlapping sub-images; S4: Perform vehicle target detection on each sub-image respectively, convert the detection results into the reference system of the image to be detected, and store them in a non-overlapping area recognition result set and an overlapping area recognition result set; wherein the detection results of the non-overlapping area are stored in the non-overlapping area recognition result set, and the detection results of the overlapping area are stored in the overlapping area recognition result set; S5: After deduplication processing is performed on the detection results in the overlapping area recognition result set using the Hungarian algorithm, the detection results are merged with the non-overlapping area recognition result set to generate a recognition result of the image to be detected; The S5 includes: S51: matching the overlapping targets of the overlapping region recognition result set in order from left to right and then from top to bottom; wherein the matching process of the overlapping targets is: using the IOU and category calculation cost matrix, and finding the optimal overlapping match of two overlapping regions by using the Hungarian algorithm; S52: After fusing the overlapping targets, merge them with the non-overlapping area recognition result set to generate a recognition result of the image to be detected; The matching value is calculated as: ， Among them, T is the matching value, IOU is used to judge whether the target rectangles of two elements have a union. The detection results of the two elements are tar1 and tar2 respectively. , is the intersection of the two elements’ corresponding detection result rectangles, It is the union of the rectangular boxes of the detection results of the two elements. R is used to determine whether the two elements are of the same category. , c1 and c2 are the categories of tar1 and tar2 respectively.

2. The method for identifying road vehicles based on drone aerial images according to claim 1, characterized in that: The calculation formula of the division multiple s in S2 is: ， Wherein, A1 is the image pixel area of the sub-dataset, AC1 is the pixel area of the target vehicle rectangular box in the sub-dataset, A2 is the image pixel area of the vehicle target detection model training image database, and AC2 is the pixel area of the target vehicle rectangular box in the vehicle target detection model training image database. Indicates a round-up operation.

3. The method for identifying road vehicles based on drone aerial images according to claim 2, characterized in that: The expression of the region where each subgraph in S3 is located is: ， in, is the set of the mth sub-graph point in the horizontal direction and the nth sub-graph point in the vertical direction, , (x,y) is a set The coordinates of a certain point; w and h are the width and height of the image to be detected, respectively, and Z is an integer set; The expression of the horizontal overlapping area generated by the horizontal dividing line is: ， in, is the set of points on the horizontal overlapping area of the i-th horizontal dividing line, (x, y) is the set The coordinates of a point, ; The expression of the vertical overlapping area generated by the vertical dividing line is: ， in, is the set of points on the vertical overlapping area of the j-th vertical dividing line, (x, y) is the set The coordinates of a point; .

4. The method for identifying road vehicles based on drone aerial images according to claim 1, characterized in that: The vehicle target detection model used for target detection of the vehicle in S4 is the yolov5 model.

5. The method for identifying road vehicles based on drone aerial images according to claim 4, characterized in that: The expression for converting the detection result of the sub-image in S4 into the coordinate system of the image to be detected is: Among them, tar is the detection result of the subgraph before conversion, ,(x top ,y top ) is the position coordinate of the upper left corner boundary point on the diagonal line of the target recognition rectangle in the sub-image, (x bottom, y bottom ) is the position coordinate of the lower right corner boundary point on the diagonal line of the target recognition rectangular box in the sub-graph, c is the target recognition category, p is the probability of the category being recognized; TAR is the detection result after conversion, ,(X top, Y top ) is the position coordinate of the upper left corner boundary point on the diagonal line of the target recognition rectangular frame in the image to be detected, (X bottom, Y bottom ) is the position coordinate of the lower right corner boundary point on the diagonal line of the target recognition rectangular box in the image to be detected, C is the category of target recognition, and P is the probability of the category being recognized; η is the conversion bias matrix, which is expressed as follows: ， Among them, m and n are the sequence numbers of the subgraphs from left to right and from top to bottom respectively. , Represents the floor operation.

6. The method for identifying road vehicles based on drone aerial images according to claim 5, characterized in that: The tar of the detection results in the overlapping area in each subgraph satisfies: ， or: ， Among them, (φ xtop, φ ytop ) is the position coordinate of the upper left corner boundary point on the diagonal line of the corresponding overlapping area in the sub-graph, (φ xbottom, φ ybottom ) is the position coordinate of the lower right corner boundary point on the diagonal line of the corresponding overlapping area in the sub-image.

7. A road vehicle identification device based on drone aerial images, characterized in that: The invention comprises at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 6.

Images