Unmanned aerial vehicle remote sensing surveying and mapping method and system

Abstract

The invention discloses an unmanned aerial vehicle remote sensing surveying and mapping method and system, and belongs to the technical field of building image recognition, and the method specifically comprises the following steps: carrying out the communication connection with an unmanned aerial vehicle, and sending the position information of a building image needed by a current terminal; acquiring a building image of the building sent by the unmanned aerial vehicle after receiving the positioning information, and performing angle correction on the building image to generate a building image at an overlook angle; identifying the building image, and extracting character information in building image information; sending the character information to a manual terminal for auditing, and storing the character information passing the auditing; according to the building image positioning algorithm based on the improved Sobel edge detection operator, the pseudo building image is removed by using character features, the building image characters are segmented through a threshold value method, and the characters are identified through feature comparison, so that accurate identification of building image character information is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method and system for remote sensing mapping using an unmanned aerial vehicle. Background Art

[0002] At present, the UAV mobile communication system can achieve intercommunication with ground mobile, fixed-line networks, the Internet, and user private networks. The communication range covers my country and surrounding areas. The UAV mobile communication system consists of communication UAVs, ground gateways, and user UAV mobile communication terminals. The communication UAV completes the forwarding of wireless signals, and the ground gateway completes the connection of user communication services and the management of the system. The UAV mobile communication terminal can call any ground fixed and mobile phone in the world, and can provide users with UAV mobile communication functions such as voice, images, and network data.

[0003] For some city slogans and advertisements, as well as road sign text, drone reconnaissance and image recognition can be used to quickly confirm whether the text information meets urban planning requirements, thereby saving manpower. However, in existing building image recognition systems, due to the imperfections of imaging systems, transmission media, recording equipment, and changes in weather conditions, building images are often contaminated by various noises, making it difficult to accurately identify characters in building images. Summary of the invention

[0004] The purpose of the present invention is to provide a UAV remote sensing mapping method and system to solve the following technical problems: In existing building image recognition systems, due to the imperfections of imaging systems, transmission media, recording equipment, and changes in weather conditions, building images are often contaminated by various noises, making it difficult to accurately identify characters in building images.

[0005] The purpose of the present invention can be achieved through the following technical solutions: A method and system for remote sensing mapping by unmanned aerial vehicle, comprising: Communicate with the drone and send the location information of the building image required by the current terminal; Acquire the building image of the building sent by the drone after receiving the positioning information, and perform angle correction on the building image to generate a building image from a bird's-eye view; Recognize the building image and extract text information from the building image information; The text information is sent to a manual end for review, and the reviewed text information is stored.

[0006] As a further solution of the present invention: the process of identifying the building image information includes: The building image is split into a plurality of image frames, the image frames are gray-processed to obtain gray-scale images, and the gray-scale images are subjected to noise reduction processing.

[0007] As a further solution of the present invention: the Sobel edge detection operator is used to perform edge detection on the grayscale image after denoising, and the maximum inter-class variance method is used to binarize the grayscale image after edge detection to obtain a binary image, the pixel value of the grayscale image after binarization is 0 or 255, and the binarized image is denoised again.

[0008] As a further solution of the present invention: the outline of the white connected area in the binary image is selected through a rectangular bounding box to obtain a pending rectangular area, the aspect ratio of the rectangular bounding box is A±a:B±b, A and B are preset coefficients, a and b are preset error thresholds, and the pending rectangular area is expanded and enlarged. If there are multiple pending rectangular areas, the pixel ratio of the grayscale value of 255 and the grayscale value of 0 in the pending rectangular area and the range of the number of grayscale jumps in the pending rectangular area are calculated respectively. If there is a pending rectangular area with a pixel ratio lower than 0.25 and the range of the number of jumps is between [5,30], the pending rectangular area is determined to be a building image, otherwise the selection continues.

[0009] As a further solution of the present invention: performing architectural image character segmentation on the architectural image, setting two thresholds T1 and T2, scanning the architectural image from left to right, obtaining the first column whose number of pixels with a pixel value of 255 is greater than the threshold value T1, which is the starting column of the architectural image Chinese character, denoted as S, and continuing to scan the architectural image until a column whose number of pixels with a pixel value of 255 is less than the threshold value T1 is identified, denoted as O, if OS>T2, then O is the ending column of the architectural image Chinese character, otherwise continue to scan the image until a column whose width is greater than T2 from the S column and whose number of pixels with a pixel value of 255 is less than the preset threshold is identified, then the column is the ending column of the architectural image Chinese character; Continue to scan the remaining characters, obtain the column with a pixel value of 255 and the number of pixels greater than the threshold T1 as the character start column, and the column with a pixel value of 255 and the number of pixels less than the threshold T1 as the character end column, and repeat this process to complete the character segmentation of the building image.

[0010] As a further solution of the present invention: the cloud platform also includes a character library, which is generated based on big data and stores standard characters of various architectural images. The standard characters are selected, the outer edge lines of the standard characters are drawn, a coordinate system is established with the geometric center point of the standard characters as the origin, and several pairs of feature points on the outer edge lines of the standard characters are selected as reference points, and the reference points are marked as (Xi, Yi) and (Mi, Ni), i=1…n; Select the building image character, draw the outer edge line of the building image character, establish a coordinate system with the geometric center point of the character as the origin, select several pairs of feature points on the outer edge line of the building image character as detection points, and mark the detection points as (Ui, Vi) and (Zi, Wi), i=1...n; By formula A similarity value XS between the building image character and the standard character is obtained. If the similarity value XS is lower than a preset threshold, the building image character is determined to be a corresponding standard character and the building image information is obtained. Otherwise, recognition is continued.

[0011] An unmanned aerial vehicle remote sensing mapping system, comprising: The information transmission module is used to collect the location information of the target building in this survey, communicate with the drone, and send the corresponding location information. The drone generates a flight route and takes pictures of the target building in turn; An image processing module is used to obtain a building image taken by a drone, pre-process the building image, and perform angle correction on the building image to generate a building image from a bird's-eye view. A sign recognition module is used to recognize building images, extract text signs in the building, and compare the clarity of the text signs based on standard text, and determine the fuzzy text signs with a clarity lower than the preset clarity as the text to be processed; The manual review module is used to send the pending text to the manual end for review, and mark the pending text that has passed the review with a pending flag.

[0012] Beneficial effects of the present invention: In order to quickly and accurately locate the position of a building image under complex backgrounds and uneven lighting conditions, the present invention solves the existing problems by using a character image positioning algorithm based on an improved Sobel edge detection operator, realizes texture feature extraction of the character image in the horizontal, vertical and diagonal directions, then uses a maximum inter-class variance algorithm for binarization, performs mathematical morphological operations on the binary image to obtain a candidate area for the character image, uses character image features to remove pseudo-character images, segments individual characters in the character image using a threshold method, and recognizes characters through feature comparison. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present invention will be further described below in conjunction with the accompanying drawings.

[0014] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] See also Figure 1 As shown, the present invention is a UAV remote sensing mapping method and system, comprising: Communicate with the drone and send the location information of the building image required by the current terminal; Acquire the building image of the building sent by the drone after receiving the positioning information, and perform angle correction on the building image to generate a building image from a bird's-eye view; Recognize the building image and extract text information from the building image information; The text information is sent to a manual end for review, and the reviewed text information is stored.

[0017] In the existing building image recognition system, due to the imperfections of the imaging system, transmission medium, recording equipment, etc., as well as changes in weather conditions, the building image is often polluted by various noises, making it difficult to accurately identify the characters in the building image. In order to quickly and accurately locate the position of the building image under complex backgrounds and uneven lighting conditions, the present invention is based on a building image positioning algorithm based on an improved Sobel edge detection operator to solve the existing problems, and realizes the extraction of texture features of the building image in the horizontal, vertical and diagonal directions, and then uses the maximum inter-class variance algorithm to binarize it, and then performs mathematical morphological operations on the binary image to obtain the candidate area of ​​the building image, uses the building image features to remove pseudo-building images, uses the threshold method to segment the building image characters, and recognizes the characters through feature comparison.

[0018] In a preferred embodiment of the present invention, the process of identifying the building image information includes: Splitting the building image into a plurality of image frames, performing grayscale processing on the image frames to obtain grayscale images, and performing noise reduction processing on the grayscale images; If the color image is processed directly, the execution speed of the system will be reduced and the storage space will become larger. Reasonable grayscale conversion will be of great help to the extraction and subsequent processing of image information, saving storage space and speeding up processing.

[0019] In a preferred embodiment of the present invention, the Sobel edge detection operator is used to perform edge detection on the grayscale image after denoising, and the maximum inter-class variance method is used to perform binarization processing on the grayscale image after edge detection to obtain a binarized image, the pixel value of the grayscale image after binarization processing is 0 or 255, and the binarized image is subjected to denoising processing again; The Sobel operator has different weights based on the distance between the neighboring pixels and the current pixel, emphasizing the influence of the neighboring pixels on the opposite sides of the central pixel, while weakening the effect of the four diagonal neighboring pixels; each pixel in the image is convolved with these two kernels, one convolution kernel responds most to the vertical edge of the image, and the other responds most to the horizontal edge, and the maximum value of the two convolutions is taken as the output value of the pixel; this makes the Sobel operator suppress noise, so there will not be many isolated edge pixels.

[0020] In a preferred case of the present embodiment, the outline of the white connected area in the binary image is selected by a rectangular bounding box to obtain a pending rectangular area, the aspect ratio of the rectangular bounding box is A±a:B±b, A and B are preset coefficients, and a and b are preset error thresholds, and the pending rectangular area is expanded and enlarged. If there are multiple pending rectangular areas, the pixel ratio of the grayscale value of 255 and the grayscale value of 0 in the pending rectangular area and the range of the number of grayscale jumps in the pending rectangular area are calculated respectively. If there is a pending rectangular area with a pixel ratio lower than 0.25 and a jump number range of [5,30], the pending rectangular area is determined to be a building image, otherwise the selection continues; Since mathematical morphological operations on building images will reduce building image information and building images obtained by shooting will be tilted, etc., a length and width threshold is set to eliminate candidate areas whose length and width ratios do not meet the requirements.

[0021] It is worth noting that the architectural image is segmented into architectural image characters, two thresholds T1 and T2 are set, and the architectural image is scanned from left to right to obtain the first column whose number of pixels with a pixel value of 255 is greater than the threshold T1, which is the starting column of the architectural image Chinese character, denoted as S, and the architectural image is continuously scanned until a column whose number of pixels with a pixel value of 255 is less than the threshold T1 is identified, denoted as O, if OS>T2, then O is the ending column of the architectural image Chinese character, otherwise the image is continuously scanned until a column whose width is greater than T2 from the S column and whose number of pixels with a pixel value of 255 is less than the preset threshold is identified, and this column is the ending column of the architectural image Chinese character; Continue to scan the remaining characters, obtain the column whose pixel value is 255 and the number of pixels is greater than the threshold T1 as the character start column, and the column whose pixel value is 255 and the number of pixels is less than the threshold T1 as the character end column, and repeat this process to complete the character segmentation of the building image; According to the characteristics of Chinese characters, two thresholds are set to segment the first Chinese character of the building image. When segmenting disconnected Chinese characters, this improved method works significantly. The remaining characters are all English letters and Arabic numerals, which do not have the problem of disconnection. Therefore, only one threshold is used to segment the remaining characters of the building image.

[0022] In another preferred situation of this embodiment, the cloud platform also includes a character library, which is generated based on big data. The cloud platform also includes a character library, which is generated based on big data and stores standard characters of various architectural images. The standard characters are selected, and the outer edge lines of the standard characters are drawn. A coordinate system is established with the geometric center point of the standard characters as the origin, and several pairs of feature points on the outer edge lines of the standard characters are selected as reference points. The reference points are marked as (Xi, Yi) and (Mi, Ni), where i=1…n; Select the building image character, draw the outer edge line of the building image character, establish a coordinate system with the geometric center point of the character as the origin, select several pairs of feature points on the outer edge line of the building image character as detection points, and mark the detection points as (Ui, Vi) and (Zi, Wi), i=1...n; By formula Obtaining a similarity value XS between the building image character and the standard character. If the similarity value XS is lower than a preset threshold, the building image character is determined to be the corresponding standard character and the building image information is obtained. Otherwise, the recognition is continued. The present application draws the outer edge lines of standard characters and architectural image characters, selects reference points and detection points respectively, and compares the similarity between the standard characters and architectural image characters through a formula, thereby identifying the architectural image characters.

[0023] An unmanned aerial vehicle remote sensing mapping system, comprising: The information transmission module is used to collect the location information of the target building in this survey, communicate with the drone, and send the corresponding location information. The drone generates a flight route and takes pictures of the target building in turn; An image processing module is used to obtain a building image taken by a drone, pre-process the building image, and perform angle correction on the building image to generate a building image from a bird's-eye view. A sign recognition module is used to recognize building images, extract text signs in the building, and compare the clarity of the text signs based on standard text, and determine the fuzzy text signs with a clarity lower than the preset clarity as the text to be processed; The manual review module is used to send the pending text to the manual end for review, and mark the pending text that has passed the review with a pending flag.

[0024] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A UAV remote sensing mapping method, It is characterized in that The following steps are involved: Collect the location information of the target building for this survey, establish communication connection with the drone, and send the corresponding location information. The drone generates a flight route and takes pictures of the target building in turn; Acquire a building image taken by a drone, preprocess the building image, and perform angle correction on the building image to generate a building image from a bird's-eye view; Recognize the building image, extract the text signs in the building, and compare the clarity of the text signs based on the standard text, and determine the fuzzy text signs with a clarity lower than the preset clarity as the text to be processed; The text to be processed is sent to a manual end for review, and the text to be processed that passes the review is marked with a pending flag.

2. A method for remote sensing mapping using an unmanned aerial vehicle according to claim 1, It is characterized in that The process of identifying the building image information includes: The building image is split into a plurality of image frames, the image frames are gray-processed to obtain gray-scale images, and the gray-scale images are subjected to noise reduction processing.

3. A method for remote sensing mapping using an unmanned aerial vehicle according to claim 2, It is characterized in that The Sobel edge detection operator is used to perform edge detection on the grayscale image after denoising, and the maximum inter-class variance method is used to perform binarization processing on the grayscale image after edge detection to obtain a binarized image. After the binarization processing, the pixel value of the grayscale image is 0 or 255, and the binarization image is subjected to denoising again.

4. The method for remote sensing mapping using an unmanned aerial vehicle according to claim 3, It is characterized in that The outline of the white connected area in the binary image is selected by a rectangular bounding box to obtain a pending rectangular area, wherein the aspect ratio of the rectangular bounding box is A±a:B±b, A and B are preset coefficients, and a and b are preset error thresholds. The pending rectangular area is expanded and enlarged. If there are multiple pending rectangular areas, the pixel ratio of the grayscale value of 255 and the grayscale value of 0 in the pending rectangular area and the range of the number of grayscale jumps in the pending rectangular area are calculated respectively. If there is a pending rectangular area with a pixel ratio lower than 0.25 and a jump number range of [5,30], the pending rectangular area is determined to be a building image, otherwise the selection continues.

5. The method for remote sensing mapping using an unmanned aerial vehicle according to claim 4, It is characterized in that Perform architectural image character segmentation on the architectural image, set two thresholds T1 and T2, scan the architectural image from left to right, obtain the first column whose number of pixels with a pixel value of 255 is greater than the threshold value T1, which is the starting column of the architectural image Chinese character, denoted as S, continue to scan the architectural image until a column whose number of pixels with a pixel value of 255 is less than the threshold value T1 is identified, denoted as O, if OS>T2, then O is the ending column of the architectural image Chinese character, otherwise continue to scan the image until a column whose width is greater than T2 from the S column and whose number of pixels with a pixel value of 255 is less than the preset threshold is identified, then this column is the ending column of the architectural image Chinese character; Continue to scan the remaining characters, obtain the column with a pixel value of 255 and the number of pixels greater than the threshold T1 as the character start column, and the column with a pixel value of 255 and the number of pixels less than the threshold T1 as the character end column, and repeat this process to complete the character segmentation of the building image.

6. The method for remote sensing mapping using an unmanned aerial vehicle according to claim 5, It is characterized in that The cloud platform also includes a character library, which is generated based on big data and stores standard characters of various architectural images. The standard characters are selected, the outer edge lines of the standard characters are drawn, a coordinate system is established with the geometric center point of the standard characters as the origin, and several pairs of feature points on the outer edge lines of the standard characters are selected as reference points, and the reference points are marked as (Xi, Yi) and (Mi, Ni), i=1...n; Select the building image character, draw the outer edge line of the building image character, establish a coordinate system with the geometric center point of the character as the origin, select several pairs of feature points on the outer edge line of the building image character as detection points, and mark the detection points as (Ui, Vi) and (Zi, Wi), i=1...n; By formula A similarity value XS between the building image character and the standard character is obtained. If the similarity value XS is lower than a preset threshold, the building image character is determined to be the corresponding standard character, and the building image text information is obtained. Otherwise, recognition continues.

7. A UAV remote sensing mapping system, It is characterized in that include: The information transmission module is used to collect the location information of the target building in this survey, communicate with the drone, and send the corresponding location information. The drone generates a flight route and takes pictures of the target building in turn; An image processing module is used to obtain a building image taken by a drone, pre-process the building image, and perform angle correction on the building image to generate a building image from a bird's-eye view. A sign recognition module is used to recognize building images, extract text signs in the building, and compare the clarity of the text signs based on standard text, and determine the fuzzy text signs with a clarity lower than the preset clarity as the text to be processed; The manual review module is used to send the pending text to the manual end for review, and mark the pending text that passes the review with a pending flag.

Images