Character recognition method and system based on aerial image of unmanned aerial vehicle

Abstract

The invention discloses a character recognition method and system based on an aerial image of an unmanned aerial vehicle, and relates to the technical field of intelligent recognition, and the key points of the technical scheme are that the method comprises the steps: inputting the aerial image, carrying out the processing through CNN backbone, carrying out the convolution operation of a minimum-resolution feature map through different convolution kernels, splicing together, and obtaining an output feature map, the output feature map is sent to a multi-branch network to be processed, and a plurality of branch output vectors are obtained; splicing a plurality of branch output vectors together, carrying out two times of Linear through a character recognition head, and outputting probability distribution formed by the vectors through loss calculation; and in the probability distribution, selecting a character corresponding to a maximum probability value as output, and obtaining a character based on unmanned aerial vehicle aerial image recognition. The end-to-end neural network for identifying the serious perspective distortion character string is designed, the gesture of the character string does not need to be corrected through preprocessing, and meanwhile, the network is extremely simple, can be locally deployed based on an airborne edge computer of an unmanned aerial vehicle and is used for performing character identification in real time.

Description

Technical Field

[0001] The patent of this invention relates to the field of intelligent recognition technology, specifically, to a character recognition method and system based on drone aerial images. Background Art

[0002] Drone aerial photography uses unmanned aircraft as an aerial platform and airborne remote sensing equipment such as high-resolution CCD digital cameras, lightweight optical cameras, infrared scanners, laser scanners, magnetometers, etc. to obtain information, process the image information with a computer, and produce images according to certain accuracy requirements. The entire system has outstanding features in design and optimal combination. It is a new application technology that integrates high-altitude photography, remote control, telemetry technology, video image microwave transmission and computer image information processing.

[0003] Optical Character Recognition (OCR) refers to the process in which an electronic device examines characters printed on paper, determines their shape by detecting dark and light patterns, and then uses character recognition methods to translate the shape into computer text. That is, for printed characters, an optical method is used to convert the text in a paper document into a black and white dot matrix image file, and the text in the image is converted into a text format through recognition software for further editing and processing by word processing software. The workflow of optical character recognition is to convert the image so that the graphics in the image can continue to be saved, and if there is a table, the data in the table and the text in the image are all converted into computer text, so that the storage volume of image data can be reduced, the recognized text can be reused and analyzed, and of course, it can also save manpower and time due to keyboard input. From image to result output, it is necessary to go through image input, image pre-processing, text feature extraction, comparison and recognition, and finally correct the misrecognized text through manual correction and output of the result.

[0004] In the past, the understanding and recognition of aerial images were all done manually, which is undoubtedly a time-consuming, labor-intensive and inefficient method. In recent years, domestic and foreign scholars have proposed many deep neural network models for extracting visual features. At the same time, some scholars have begun to study the application of deep neural networks in the field of aerial images, and have proposed some networks for aerial image recognition and feature extraction. In terms of image content retrieval, deep neural networks have also been introduced to extract structured information from aerial images.

[0005] So far, deep convolutional neural networks have become a common method for target detection algorithms. Currently, high-performance object detection algorithms and the latest research are also based on deep convolutional neural networks. In order to improve the speed of target detection networks, Liu et al. proposed the SSD network. This network performs target category classification and position regression on feature maps of different sizes, and uses feature maps of different sizes to detect targets of different sizes, eliminating the RPN network and greatly improving the speed of the network; further, Redmon et al. proposed the YOLO network, which divides the original image into 7x7 areas, and directly performs target category classification and position regression on each area through a neural network, eliminating the classification and regression operations of feature maps of different sizes, which further improves the speed of the target detection network. In addition, many other image detection network models have been proposed.

[0006] With the continuous advancement of computer technology, multimedia technology and network technology, as well as the rapid development of deep learning, optical character recognition is generally considered to be a solved problem. However, due to the free navigation and high flight altitude of drones, character recognition based on drone aerial images still has the following problems to be solved: 1. Due to the limitation of wireless communication, drones cannot access the network anytime and anywhere. There are even some scenarios where it is impossible to access the cloud network. In these cases, cloud OCR services cannot be used and local character recognition must be performed based on the onboard computer. 2. When drones take photos of the ground, due to the freedom of heading and posture, the characters in the aerial photos are extremely perspective-distorted, with rotation, inversion, and mis-cutting. General OCR can only recognize strings arranged horizontally from left to right. Due to lighting, background interference, and other reasons, the perspective-distorted strings are corrected to be arranged horizontally with the characters facing upwards through preprocessing, which is not robust and has certain difficulties. The perspective distortion of the characters causes the general OCR algorithm to fail in recognition. Even some large cloud service providers cannot recognize strings with severe perspective distortion. 3. Drones have limited onboard processing capabilities and cannot run complex large-scale deep learning networks.

[0007] Therefore, the present invention aims to provide a character recognition method and system based on drone aerial images to solve the above problems. Summary of the invention

[0008] The purpose of the present invention is to provide a character recognition method and system based on drone aerial images. The present invention realizes end-to-end recognition by directly sending the character images obtained by aerial photography into the network for recognition. It does not need to be corrected into a rectangle arranged horizontally from left to right through preprocessing, thereby eliminating the risk of failure in preprocessing posture correction and improving robustness and accuracy. At the same time, the network structure of the present invention is clear and concise, and an ultra-lightweight but effective character recognition network is realized. It can be locally deployed based on the drone-mounted edge computer to realize real-time processing, and the loss calculation method in the present invention can effectively improve the accuracy of character recognition.

[0009] The present invention is implemented as follows: a character recognition method based on drone aerial images, comprising the following steps: S1, input aerial images, use CNN backbone to process, get minimum resolution feature map, then convolve the minimum resolution feature map with different convolution kernels, and splice them together to get output feature map; S2, sending the spliced ​​output feature map to the multi-branch network for processing to obtain multiple branch output vectors; S3, concatenate multiple branch output vectors together, pass through the first two linear operations of character recognition, and calculate the probability distribution of the output vectors through loss calculation; S4. In the probability distribution, the character corresponding to the maximum probability is selected as the output to obtain the character recognized based on the drone aerial image.

[0010] The present invention also provides a character recognition method based on drone aerial images. Compared with the above scheme, the method replaces the input aerial image with a batch of aerial images, and the batch of aerial images includes the aerial image itself, the aerial image rotated 90 degrees, and the aerial image rotated 180 degrees.

[0011] Furthermore, in step S1, the aerial images are batch-inputted into the CNN backbone through a data loader.

[0012] Furthermore, the data loader specifically adopts DataLoader, which performs image enhancement operation and normalization operation on the aerial images respectively, and then inputs them into the CNN backbone in batches.

[0013] Furthermore, the process of the image enhancement operation is: a) rotating and shearing the aerial image; b) then enhancing the illumination; c) finally filling the blank area in the aerial image with color.

[0014] Furthermore, the process of the normalization operation is as follows: a) decomposing the aerial image into three color channels of red, green and blue; b) dividing the three color channels by 255 respectively to improve data contrast; c) using the mean and variance to complete the normalization process to reduce the scale difference between different features of the aerial image.

[0015] Furthermore, in the step S1, the CNN backbone specifically uses mobilenet 0.25 as a backbone for processing the input aerial image.

[0016] Furthermore, the number of branches of the multi-branch network in step S2 is equal to the maximum recognition length N of the preset character string. After the concatenated output feature map is input into N branches with the same structure for processing, the N branches output a vector with a length of 256.

[0017] Furthermore, the loss calculation in step S3 is performed by combining the cross entropy loss and the FocalLoss loss, and the loss calculation formula is: in, is the network output probability corresponding to the true value category, The network output probability corresponding to other categories, For Category The weight of , Take a number greater than 1.

[0018] Furthermore, the process of selecting the character corresponding to the maximum probability in step S4 is specifically to select the serial number corresponding to the maximum probability, search for the character to be recognized at the corresponding position in the character table by the serial number, and then output the character.

[0019] Furthermore, the aerial image in step S1 is fixed with a special mark to increase the robustness of the network and improve the recognition accuracy of special characters.

[0020] The present invention also provides a character recognition system based on drone aerial images, which is characterized by comprising an image processing module, a feature conversion module, a probability calculation module and an image output module; The image processing module is used to obtain aerial images, process them using the CNN backbone, generate minimum resolution feature maps, and simultaneously use different convolution kernels to perform convolution operations and splice to obtain output feature maps; The feature conversion module uses a multi-branch network to process the spliced ​​output feature map to generate multiple branch output vectors; The probability calculation module is used to call multiple branch output vectors and splice them together, use the first two linear operations of character recognition to calculate the loss, and output the probability distribution of the vectors; The image output module selects the character corresponding to the maximum probability as the character output of the aerial image recognition by calling the probability distribution result.

[0021] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention realizes end-to-end recognition by directly sending the character images obtained by aerial photography to the network for recognition. It does not need to be corrected into a rectangle arranged horizontally from left to right through preprocessing, thereby eliminating the risk of failure of preprocessing posture correction and improving robustness and accuracy. 2. The network structure provided by the present invention is clear and concise, realizing an ultra-lightweight but effective character recognition network, which can be locally deployed based on the edge computer onboard the drone and can realize real-time processing; 3. The loss calculation method provided by the present invention is to combine the cross entropy loss and the FocalLoss loss, which can both reward the positive category and punish the negative category, while reducing the loss weight of the easy-to-classify samples, focusing on the classification of difficult samples, and has a fast convergence speed. Compared with the cross entropy loss and the FocalLoss loss, it can improve the accuracy of character recognition of aerial images; 4. The present invention uses the extremely simplified CNN network mobilenet0.25 as the backbone. For the network used for recognition, there is no need to build a feature map pyramid. Only a feature map with a resolution of 1 / 32 and 64 channels is taken. Compared with other common networks such as Resnet50, the number of parameters is reduced by more than 50 times, and the inference speed is significantly improved; 5. The present invention obtains multiple branch output vectors by sending the spliced ​​output feature map to a multi-branch network for processing, wherein each branch is composed of two convolutions and each branch encodes one character. Through the processing method of the present invention, all CNNs are used, thus avoiding the difficulty of training the RNN network, making it more suitable for the recognition of shorter character strings; 6. The network designed by the present invention has a branch number of N=3 and M=11, which can be used to identify numbers with a length not exceeding 3. The parameter volume is 1.15M, of which the backbone is 0.47M. The model is very small and has a fast inference speed, which is more suitable for edge processing device deployment; 7. The present invention sets a special mark at a fixed corner of the aerial image, which can improve the recognition accuracy of the special characters "6" and "9". After rotating 180 degrees, the special characters are converted into another character, avoiding the situation where they cannot be correctly recognized without the assistance of other characters, thereby increasing the robustness of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 1 is a flow chart of a method for character recognition based on drone aerial images in Embodiment 1 of the present invention; Figure 2 It is a schematic diagram of the data processing flow of a character recognition method based on drone aerial images in Example 1 of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0024] The implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0025] The same or similar numbers in the drawings of this embodiment correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limitations on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0026] Reference Figure 1-2 The figure shows a preferred embodiment of the present invention.

[0027] Embodiment 1: A character recognition method based on drone aerial images, comprising the following steps: S1. The aerial image is rotated and sheared through the data loader DataLoader, and then the lighting is enhanced. The blank area in the aerial image is filled with a color with an RGB value of (128, 128, 128). The aerial image is further decomposed into three color channels: red, green, and blue. The three color channels are then divided by 255 to improve the data contrast. At the same time, the mean (0.5, 0.5, 0.5) and variance (0.5, 0.5, 0.5) are used to complete the normalization process to reduce the scale difference between different features of the aerial image. The image enhancement and normalization operations of the aerial image are completed, and then the aerial images are batch-input into the CNN backbone to obtain the minimum resolution feature map. The minimum resolution feature map is then convolved with different convolution kernels and spliced ​​together to obtain the output feature map.

[0028] The data loader DataLoader specifically includes data loading and batch processing. DataLoader can load data from different sources, including files on the hard disk, databases, and networks. It automatically divides the data set into small batches to reduce memory requirements and ensure efficient data loading. At the same time, each batch consists of multiple aerial image samples, and data preprocessing and data enhancement can be performed in parallel, which helps to improve the efficiency of model training while ensuring that each batch of data is properly processed.

[0029] The specific data reading process of the data loader DataLoader is to first generate data indexes using dataset and sampler, then read a specified amount of data from the dataset based on these indexes, and perform normalization and cropping preprocessing on it, and then customize the data loading and batch processing methods by packaging the processed data into batch data or assigning data loading tasks to multiple sub-processes in parallel. During the data processing process, a batch of data is obtained from the DataLoader as the input of the aerial image.

[0030] The color with RGB value (128, 128, 128) is neutral gray, and the blank area in the aerial image is filled with gray with RGB value (128, 128, 128). Specifically, the aerial image is scaled according to the aspect ratio, the blank area is filled with (128, 128, 128), the size of the aerial image is adjusted so that one side of the image is equal to the target length and the other side is less than or equal to the target length, and then the scaled data is copied to the center of the canvas. Finally, the filled aerial image is returned for image enhancement and normalization operations.

[0031] In the process of illumination enhancement of aerial images, the aerial images are decomposed into three color channels of red, green and blue. The illumination enhancement of the images is completed by increasing the three channels at the same time. Then the three color channels are divided by 255 respectively to increase the gap between similar colors, which can be used to improve the data contrast. At the same time, the mean (0.5, 0.5, 0.5) and variance (0.5, 0.5, 0.5) are used to reduce the scale difference between different features of the aerial images and complete the image enhancement and normalization operations of the aerial images. In the normalization process, each aerial image input value can be converted into a standard normal distribution by subtracting the mean from the aerial image data point and dividing by the standard deviation. Subtracting the mean is the translation operation implemented on the aerial image, and dividing by the standard deviation is the scaling operation implemented on the aerial image. The normalized aerial image data fully considers the characteristics of time series data. Through translation and scaling operations, the distribution of the data is more stable, which is conducive to improving the accuracy of character recognition.

[0032] In this embodiment, the CNN backbone specifically uses an extremely simplified CNN network mobilenet0.25 as the backbone, which can realize onboard real-time processing. Specifically, by inputting the same aerial image at 3 rotation angles, it only takes 2ms. At the same time, for the recognition network, there is no need to build a feature map pyramid, and only takes a feature map with a resolution of 1 / 32 and 64 channels. Compared with other common networks such as Resnet50, the number of parameters is reduced by more than fifty times, and the inference speed is significantly improved.

[0033] S2. Send the concatenated output feature map to a multi-branch network for processing. Each branch is specifically composed of the functions Conv(64, 128, 3), ReLU(), Conv(128, 256, 1), and ReLU() in series, and each branch consists of two convolutions. Each branch encodes one character. The number of branches of the multi-branch network is equal to the maximum recognition length N of the preset string. After the concatenated output feature map is input into N branches with the same structure for processing, the N branches output a vector with a length of 256.

[0034] S3. Concatenate the N branch output vectors together to obtain an N×256 tensor, which is subjected to the first two linear operations of the character recognition head, i.e., permutation and transformation, and the probability distribution of the output vectors is calculated through loss. The character recognition head is specifically composed of the functions Linear(256, 128), ReLU(), and Linear(128, M), where M is the number of character categories to be recognized + 1. For strings whose length is less than the maximum recognition length N of the preset string, the end symbol is used to supplement the string.

[0035] In this embodiment, a network with N=3 branches and M=11 is designed to identify numbers with a length not exceeding 3. The number of parameters is 1.15M, of which the backbone is 0.47M. The model is very small and has a fast inference speed, which is suitable for deployment on edge processing devices.

[0036] At the same time, the loss calculation in this embodiment is specifically achieved by combining the cross entropy loss and the FocalLoss loss, which can both reward the positive category and punish the negative category, while also reducing the loss weight of easy-to-classify samples and focusing on the classification of difficult samples. The loss calculation formula is: in, is the network output probability corresponding to the true value category, The network output probability corresponding to other categories, For Category The weight of , Take a number greater than 1; and in practical applications, when the number of occurrences of various characters in the training samples is not much different, the inter-class weights can be Omitted, it means that all categories have equal weight.

[0037] S4. In the probability distribution, select the serial number corresponding to the maximum probability, use the serial number to find the character to be recognized at the corresponding position in the character table, and then output the character to obtain the character recognized based on the drone aerial image; in this embodiment, for each character, the network outputs a tensor of size [3, M], and takes the maximum value for the first dimension; among the three results obtained, further take the maximum value, and use its serial number (between 1 and M) as the recognition result, so that when the number of aerial image samples is insufficient, the processing has a very significant gain; and the serial number corresponding to the maximum probability in this scheme specifically adopts the one hot encoding method, by converting the categorical variable into a binary vector, it can be used for the machine learning algorithm to directly process the categorical data.

[0038] Preferably, in this embodiment, a special mark is set at a fixed corner of the aerial image. This processing can improve the recognition accuracy of the special characters "6" and "9" (the special characters are converted into another character after rotating 180 degrees, and cannot be correctly recognized without the assistance of other characters), and at the same time increase the robustness of the network.

[0039] Example 2: The difference between this example and Example 1 is that the input aerial image is replaced by a batch of aerial images, wherein the batch of aerial images includes the aerial image itself, the aerial image rotated 90 degrees, and the aerial image rotated 180 degrees; compared with Example 1, when the frame rate requirement is slightly lower and the accuracy requirement is higher, this example can further improve the accuracy on the basis of reducing the frame rate by inputting the batch of aerial images consisting of the aerial image itself, the aerial image rotated 90 degrees, and the aerial image rotated 180 degrees into CNNbackbone for processing.

[0040] This embodiment specifically includes the following steps: S1. The batch of aerial images is rotated and sheared through the data loader DataLoader, and then the lighting is enhanced. The blank areas in the aerial images are filled with colors with RGB values ​​of (128, 128, 128), and the aerial images are further decomposed into three color channels: red, green, and blue. The three color channels are then divided by 255 to improve the data contrast. At the same time, the mean (0.5, 0.5, 0.5) and variance (0.5, 0.5, 0.5) are used to complete the normalization process to reduce the scale difference between different features of the aerial images, complete the image enhancement and normalization operations on the aerial images, and then batch input the aerial images to the CNN backbone to obtain the minimum resolution feature map. The minimum resolution feature map is then convolved with different convolution kernels and spliced ​​together to obtain the output feature map.

[0041] The data loader DataLoader specifically includes data loading and batch processing. DataLoader can load data from different sources, including files on the hard disk, databases, and networks. It automatically divides the data set into small batches to reduce memory requirements and ensure efficient data loading. At the same time, each batch consists of multiple aerial image samples, and data preprocessing and data enhancement can be performed in parallel, which helps to improve the efficiency of model training while ensuring that each batch of data is properly processed.

[0042] The specific data reading process of the data loader DataLoader is to first generate data indexes using dataset and sampler, then read a specified amount of data from the dataset based on these indexes, and perform normalization and cropping preprocessing on it, and then customize the data loading and batch processing methods by packaging the processed data into batch data or assigning data loading tasks to multiple sub-processes in parallel. During the data processing process, a batch of data is obtained from the DataLoader as the input of the aerial image.

[0043] The color with RGB value (128, 128, 128) is neutral gray, and the blank area in the aerial image is filled with gray with RGB value (128, 128, 128). Specifically, the aerial image is scaled according to the aspect ratio, the blank area is filled with (128, 128, 128), the size of the aerial image is adjusted so that one side of the image is equal to the target length and the other side is less than or equal to the target length, and then the scaled data is copied to the center of the canvas. Finally, the filled aerial image is returned for image enhancement and normalization operations.

[0044] In the process of illumination enhancement of aerial images, the aerial images are decomposed into three color channels of red, green and blue. The illumination enhancement of the images is completed by increasing the three channels at the same time. Then the three color channels are divided by 255 respectively to increase the gap between similar colors, which can be used to improve the data contrast. At the same time, the mean (0.5, 0.5, 0.5) and variance (0.5, 0.5, 0.5) are used to reduce the scale difference between different features of the aerial images and complete the image enhancement and normalization operations of the aerial images. In the normalization process, each aerial image input value can be converted into a standard normal distribution by subtracting the mean from the aerial image data point and dividing by the standard deviation. Subtracting the mean is the translation operation implemented on the aerial image, and dividing by the standard deviation is the scaling operation implemented on the aerial image. The normalized aerial image data fully considers the characteristics of time series data. Through translation and scaling operations, the distribution of the data is more stable, which is conducive to improving the accuracy of character recognition.

[0045] In this embodiment, the CNN backbone specifically uses an extremely simplified CNN network mobilenet0.25 as the backbone, which can realize onboard real-time processing. Specifically, by inputting the same aerial image at 3 rotation angles, it only takes 2ms. At the same time, for the recognition network, there is no need to build a feature map pyramid, and only takes a feature map with a resolution of 1 / 32 and 64 channels. Compared with other common networks such as Resnet50, the number of parameters is reduced by more than fifty times, and the inference speed is significantly improved.

[0046] S2. Send the concatenated output feature map to a multi-branch network for processing. Each branch is specifically composed of the functions Conv(64, 128, 3), ReLU(), Conv(128, 256, 1), and ReLU() in series, and each branch consists of two convolutions. Each branch encodes one character. The number of branches of the multi-branch network is equal to the maximum recognition length N of the preset string. After the concatenated output feature map is input into N branches with the same structure for processing, the N branches output a vector with a length of 256.

[0047] S3. Concatenate the N branch output vectors together to obtain an N×256 tensor, which is subjected to the first two linear operations of the character recognition head, i.e., permutation and transformation, and the probability distribution of the output vectors is calculated through loss. The character recognition head is specifically composed of the functions Linear(256, 128), ReLU(), and Linear(128, M), where M is the number of character categories to be recognized + 1. For strings whose length is less than the maximum recognition length N of the preset string, the end symbol is used to supplement the string.

[0048] In this embodiment, a network with N=3 branches and M=11 is designed to identify numbers with a length not exceeding 3. The number of parameters is 1.15M, of which the backbone is 0.47M. The model is very small and has a fast inference speed, which is suitable for deployment on edge processing devices.

[0049] At the same time, the loss calculation in this embodiment is specifically achieved by combining the cross entropy loss and the FocalLoss loss, which can both reward the positive category and punish the negative category, while also reducing the loss weight of easy-to-classify samples and focusing on the classification of difficult samples. The loss calculation formula is: in, is the network output probability corresponding to the true value category, The network output probability corresponding to other categories, For Category The weight of , Take a number greater than 1; and in practical applications, when the number of occurrences of various characters in the training samples is not much different, the inter-class weights can be Omitted, it means that all categories have equal weight.

[0050] S4. In the probability distribution, select the serial number corresponding to the maximum probability, use the serial number to find the character to be recognized at the corresponding position in the character table, and then output the character to obtain the character recognized based on the drone aerial image; in this embodiment, for each character, the network outputs a tensor of size [3, M], and takes the maximum value for the first dimension; among the three results obtained, further take the maximum value, and use its serial number (between 1 and M) as the recognition result, so that when the number of aerial image samples is insufficient, the processing has a very significant gain; and the serial number corresponding to the maximum probability in this scheme specifically adopts the one hot encoding method, by converting the categorical variable into a binary vector, it can be used for the machine learning algorithm to directly process the categorical data.

[0051] Preferably, in the present embodiment, a special direction indicator is set at a fixed corner of the aerial image, specifically by setting an arrow pointing to the north. This processing can improve the recognition accuracy of the special characters "6" and "9" (the special characters are converted into another character after rotating 180 degrees, and cannot be correctly recognized without the assistance of other characters), and at the same time increase the robustness of the network.

[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A character recognition method based on drone aerial images, characterized in that: The following steps are involved: S1, input aerial images, use CNN backbone to process, get minimum resolution feature map, then convolve the minimum resolution feature map with different convolution kernels, and splice them together to get output feature map; S2, sending the spliced ​​output feature map to the multi-branch network for processing to obtain multiple branch output vectors; S3, concatenate multiple branch output vectors together, pass through the first two linear operations of character recognition, and calculate the probability distribution of the output vectors through loss calculation; S4. In the probability distribution, the character corresponding to the maximum probability is selected as the output to obtain the character recognized based on the drone aerial image.

2. The character recognition method based on drone aerial images according to claim 1 is characterized in that: The input aerial image is replaced by a batch of aerial images, wherein the batch of aerial images includes the aerial image itself, the aerial image rotated 90 degrees, and the aerial image rotated 180 degrees.

3. A character recognition method based on drone aerial images according to claim 1 or 2, characterized in that: In step S1, the aerial images are batch-inputted into the CNN backbone through a data loader.

4. The character recognition method based on drone aerial images according to claim 3 is characterized in that: The data loader specifically adopts DataLoader, which performs image enhancement and normalization operations on the aerial images respectively, and then inputs them into the CNN backbone in batches.

5. The character recognition method based on drone aerial images according to claim 4 is characterized in that: The process of the image enhancement operation is: a) rotating and shearing the aerial image; b) enhancing the illumination; c) filling the blank area in the aerial image with color.

6. The character recognition method based on drone aerial images according to claim 4 is characterized in that: The process of the normalization operation is as follows: a) decomposing the aerial image into three color channels: red, green, and blue; b) dividing the three color channels by 255 respectively to improve data contrast; c) using the mean and variance to complete the normalization process to reduce the scale difference between different features of the aerial image.

7. The character recognition method based on drone aerial images according to claim 1 or 2, characterized in that: In the step S1, the CNN backbone specifically uses mobilenet 0.25 as the backbone for processing the input aerial image.

8. The character recognition method based on drone aerial images according to claim 1 is characterized in that: The number of branches of the multi-branch network in step S2 is equal to the maximum recognition length N of the preset character string. After the spliced ​​output feature map is input into N branches with the same structure for processing, the N branches output a vector with a length of 256.

9. The character recognition method based on drone aerial images according to claim 1 or 2, characterized in that: The loss calculation in step S3 is achieved by combining the cross entropy loss and the FocalLoss loss. The loss calculation formula is: in, is the network output probability corresponding to the true value category, The network output probability corresponding to other categories, For Category The weight of , Take a number greater than 1.

10. A character recognition method based on drone aerial images according to claim 1 or 2, characterized in that: The process of selecting the character corresponding to the maximum probability in step S4 is specifically to select the serial number corresponding to the maximum probability, search for the character to be recognized at the corresponding position in the character table by the serial number, and then output the character.

11. A character recognition method based on drone aerial images according to claim 1 or 2, characterized in that: The aerial image in step S1 is fixed with a special mark to increase the robustness of the network and improve the recognition accuracy of special characters.

12. A character recognition system based on drone aerial images, characterized in that: It includes an image processing module, a feature conversion module, a probability calculation module and an image output module; The image processing module is used to obtain aerial images, process them using the CNN backbone, generate minimum resolution feature maps, and simultaneously use different convolution kernels to perform convolution operations and splice to obtain output feature maps; The feature conversion module uses a multi-branch network to process the spliced ​​output feature map to generate multiple branch output vectors; The probability calculation module is used to call multiple branch output vectors and splice them together, use the first two linear operations of character recognition to calculate the loss, and output the probability distribution of the vectors; The image output module selects the character corresponding to the maximum probability as the character output of the aerial image recognition by calling the probability distribution result.

Images