A Remote Sensing Image Target Recognition Method and System Based on Deep Learning

Abstract

The present invention discloses a method for remote sensing image target recognition based on deep learning, which relates to the technical field of remote sensing image target recognition and includes the following steps: Based on a common object image library, an attention mechanism is introduced to locate object targets in SAR images, and the SAR images are fully segmented into high-definition images, transition images, and blurred images according to the resolution values of the geometric centers of the object targets; The overall scattering characteristics of the high-definition images are calculated by the Deeplab v3+ algorithm to obtain scattering characteristic data, and the transition images are sliced based on the scattering characteristic data and the time series lengths are extracted to obtain scattering characteristic vectors; The EM algorithm is used to iteratively optimize the scattering characteristic vectors to obtain a number of feature mapping matrices, and the YOLOv5 network is trained using the feature mapping matrices. Finally, the SAR images are input into the trained YOLOv5 network to obtain high-resolution SAR images; Based on the target picture database, the positions of the target objects in the high-resolution SAR images are recognized.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image target recognition, and particularly to a remote sensing image target recognition method and system based on deep learning. Background Art

[0002] Synthetic Aperture Radar (SAR) forms images by actively emitting beams. It is not affected by illumination, weather, etc., and can work all day and all weather in various environments, providing comprehensive and real-time information for relevant workers to make judgments in a timely manner based on the image information. However, since SAR images are extremely sensitive to the azimuth angle of imaging, there are certain differences in SAR images of the same target at different azimuth angles. The correlation between multi-view SAR images is not stable enough, resulting in low fusion degree and poor integration effect.

[0003] The already authorized invention patent, with the publication number CN105373809B and the patent name "SAR target recognition method based on non-negative least squares sparse representation", uses the spectral features of SAR images as recognition features. By projecting test samples onto the training set and adding non-negative constraints during the sparse projection process, it avoids the interference caused by the positive and negative sparse coefficients in the sparse representation, which makes the mathematical description of the sparse representation not conform to the actual situation in radar target recognition. At the same time, it enables the sparse solution to more effectively reflect the low-dimensional structure of the target in the high-dimensional space. By determining the category of the test sample through the sparse reconstruction process, it realizes the recognition of radar targets, thereby improving the recognition rate and avoiding the interference caused by factors such as azimuth angle estimation, defocusing, or signal-to-noise ratio of SAR image targets, having good noise robustness, and being able to effectively improve the accuracy of radar target recognition.

[0004] However, in the above method of projecting onto the training set, the sparse reconstruction process adopted during the projection mainly encodes the image in a complete reconstruction manner, which will introduce unnecessary information during the encoding process, and will also make the correlation between the low-resolution part and the high-resolution part on the same SAR image unstable, with low fusion degree and poor integration effect, affecting the final target recognition result. Summary of the Invention

[0005] In order to overcome the above defects, the present invention provides a remote sensing image target recognition method and system based on deep learning. The present invention makes full use of the existence of the transitional image between the low-resolution and high-resolution parts in the SAR image, and by using the transitional image to assist in calculating the scattering feature vector of the high-definition image scattering feature, the solution process is softer, so that the obtained scattering feature vector contains the correlation between different-resolution images in the image, which is beneficial to improving the material recognition accuracy and detection efficiency of roads in high-resolution remote sensing images.

[0006] On the one hand, a remote sensing image target recognition method based on deep learning is provided, including the following steps:

[0007] Based on a common object image library, an attention mechanism is introduced to locate object targets in SAR images, and the SAR image is fully segmented into high-definition images, transition images, and blurred images according to the resolution values of the geometric centers of the object targets;

[0008] The overall scattering characteristics of the high-definition image are calculated by the Deeplab v3+ algorithm to obtain scattering characteristic data. Based on the scattering characteristic data, the transition image is sliced and the time series length is extracted to obtain a scattering characteristic vector;

[0009] The EM algorithm is used to iteratively optimize the scattering characteristic vector to obtain a number of characteristic mapping matrices. The YOLOv5 network is trained using the characteristic mapping matrices. Finally, the SAR image is input into the trained YOLOv5 network to obtain a high-resolution SAR image;

[0010] Based on the target picture database, the positions of the target objects in the high-resolution SAR image are recognized.

[0011] Preferably, when the SAR image is fully segmented into high-definition images, transition images, and blurred images according to the resolution values between adjacent object targets, the following steps are specifically included:

[0012] After the object targets in the SAR image are completely located, the adjacent environment calculation is performed on the remaining irregular images that do not contain object targets to obtain an adjacent environment result, and the adjacent environment result is the adjacent ratio of the irregular image to the transition image and the blurred image;

[0013] If the adjacent ratio of the transition image in the adjacent environment result of the irregular image exceeds 70%, then the irregular image is defined as a transition image; if not, the irregular image is defined as a blurred image.

[0014] Preferably, an object target with a geometric center resolution not exceeding 1m is a high-definition image, an object target with a geometric center resolution not exceeding 5m is a transition image, and an object target with a geometric center resolution greater than 5m is a blurred image.

[0015] Preferably, the following formula is used to calculate the overall amplitude mean of the SAR image:

[0016]

[0017] Where P is the amplitude mean, q is the central gray level, k is the edge gray level, e is the Euclidean distance, n is the total number of pixels, and l is the distance from the ground;

[0018] Among them, the Euclidean distance is calculated using the following formula:

[0019]

[0020] Among them, p i is the scattering coefficient of the i-th column pixel block in the high-definition image, is the scattering coefficient of the j-th pixel block in the i-th column pixel block of the high-definition image.

[0021] Preferably, the scattering characteristics include surface and volume scattering, double echo, combined scattering, penetrating scattering, and dielectric property scattering.

[0022] Preferably, when calculating the spectral-texture feature set of the high-definition image according to the scattering feature vector, it is calculated through an identification network model. The construction process of the identification network model specifically includes the following steps:

[0023] Construct and train a first identification model based on the autoencoder structure. The first identification model is used to map the input quantity into the feature space to obtain encoded data, and then map the encoded data back to the feature space to make the encoded data in the feature space as close as possible to the input data;

[0024] Construct a second identification model. The second identification model includes the trained first identification model and the gray-level co-occurrence matrix;

[0025] Construct a third identification model. The third identification model includes the second identification model and the texture model. The texture model includes several types of spectral indices.

[0026] In a second aspect, a remote sensing image target recognition system based on deep learning is provided, including the following:

[0027] Image library: including a common object database and a target picture database. The common object database is used to store the image information of common object objects, and the target picture database is used to store the image information of target objects;

[0028] Object object positioning module: used to perform object object positioning operations on the SAR image according to the common object image library;

[0029] Image segmentation module: used to segment the located SAR image into high-definition images, transition images, and blurred images;

[0030] Scattering feature calculation module: used to calculate the scattering feature vector based on the segmented high-definition image and blurred image;

[0031] Mapping matrix generation module: used to construct an identification network model according to the scattering feature vector, and calculate the feature mapping matrix based on the segmented high-definition image;

[0032] The YOLOv5 network training module: It is used to train the YOLOv5 network according to the feature mapping matrix to obtain high-resolution SAR images;

[0033] The target position recognition module: It is used to determine the position of the target object in the high-resolution SAR image according to the target picture database.

[0034] Preferably,

[0035] The fuzzy environment algorithm establishment module: It is used to establish a fuzzy environment algorithm, and the fuzzy environment algorithm is used to fully segment the SAR image to obtain irregular images and the adjacent environment results of the irregular images;

[0036] The adjacent result comparison module: It is used to judge the segmentation result of the irregular image according to the adjacent environment result.

[0037] In a third aspect, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the remote sensing image target recognition method based on deep learning is implemented.

[0038] In a fourth aspect, a non-transitory computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image target recognition method based on deep learning is implemented.

[0039] The beneficial effects of the present invention are reflected in:

[0040] The present invention makes full use of the existence of the transition images between low resolution and high resolution in the SAR image. By using the transition images to assist in calculating the scattering feature vectors of the high-definition image scattering features, the solution process is softer, so that the obtained scattering feature vectors contain the correlation between different resolution images in the image, which is beneficial to improving the material recognition accuracy and detection efficiency of roads in high-resolution remote sensing images. Description of the Drawings

[0041] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts do not necessarily draw according to the actual scale.

[0042] Figure 1 It is a flowchart of a remote sensing image target recognition method based on deep learning provided by the present invention;

[0043] Figure 2Flowchart of the Deeplab v3+ algorithm in a remote sensing image target recognition method based on deep learning provided by the present invention;

[0044] Figure 3a Ground distance imaging result diagram described in the embodiment of a remote sensing image target recognition method based on deep learning provided by the present invention;

[0045] Figure 3b Ground slant range imaging result diagram described in the embodiment of a remote sensing image target recognition method based on deep learning provided by the present invention. Detailed implementation manners

[0046] The embodiments of the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings. The following embodiments are only used to illustrate the technical solutions of the present invention more clearly, so they are only examples and cannot be used to limit the protection scope of the present invention.

[0047] It should be noted that unless otherwise specified, the technical terms or scientific terms used in this application should be of the ordinary meaning understood by those skilled in the art to which the present invention belongs.

[0048] In Embodiment 1, as Figure 1 、 Figure 2 shown, a remote sensing image target recognition method based on deep learning includes the following steps:

[0049] Based on a common object image library, an attention mechanism is introduced to locate object targets in the SAR image, and the SAR image is fully segmented into high-definition images, transition images, and blurred images according to the resolution values of the geometric centers of the object targets;

[0050] Calculate the overall scattering characteristics of the high-definition image through the Deeplab v3+ algorithm to obtain scattering characteristic data, slice the transition image based on the scattering characteristic data and extract the time series length to obtain a scattering characteristic vector;

[0051] Use the EM algorithm to iteratively optimize the scattering characteristic vector to obtain several feature mapping matrices, train the YOLOv5 network with the feature mapping matrices, and finally input the SAR image into the trained YOLOv5 network to obtain a high-resolution SAR image;

[0052] Based on the target picture database, identify the positions of the target objects in the high-resolution SAR image.

[0053] In this solution, the existence of transitional images between low-resolution and high-resolution SAR images is fully utilized. By using transitional images to assist in calculating the scattering feature vector of high-definition images, the solution process is smoother, and the obtained scattering feature vector contains the correlation between different-resolution images in the image, which is beneficial to improving the material recognition accuracy and detection efficiency of roads in high-resolution remote sensing images.

[0054] More specifically, when the SAR image is fully segmented into high-definition images, transitional images, and blurred images according to the resolution values between adjacent object objects, the following steps are specifically included:

[0055] After the object objects in the SAR image are completely located, by performing a neighboring environment calculation on the remaining irregular images that do not contain object objects, a neighboring environment result is obtained. The neighboring environment result is the neighboring ratio of the irregular image to the transitional image and the blurred image.

[0056] If the neighboring ratio of the transitional image in the neighboring environment result of the irregular image exceeds 70%, then the irregular image is defined as a transitional image; if not, the irregular image is defined as a blurred image.

[0057] The calculation method of the neighboring ratio between the edge of the irregular image and the transitional image here is to first calculate the total perimeter of the irregular image, then calculate the first coincidence length between the edge of the irregular image and the edge of the transitional image, and obtain the ratio of the first coincidence length to the total perimeter.

[0058] The calculation method of the neighboring ratio between the edge of the irregular image and the blurred image here is to first calculate the total perimeter of the irregular image, then calculate the second coincidence length between the edge of the irregular image and the edge of the blurred image, and obtain the ratio of the second coincidence length to the total perimeter.

[0059] More specifically, object objects with a geometric center resolution not exceeding 1m are high-definition images, object objects with a geometric center resolution not exceeding 5m are transitional images, and object objects with a geometric center resolution greater than 5m are blurred images.

[0060] Since there may be cases where the resolution of the image edge or part of the content of the object object is different, but the object object can be located, it indicates that the significant features of the object image are relatively easy to identify. At the same time, the integrity of general object objects is relatively strong, and the blurring of the identified object image edge or part of the content does not affect the overall recognition of the object image. Considering the issue of data processing efficiency, here the geometric center resolution of the object object image data is directly used to represent the overall resolution of the object object, reducing the complexity of data processing.

[0061] Nowadays, the resolution of spaceborne optical SAR images has been able to reach 0.5m - 0.3m. In the case of the best acquisition effect of general SAR images, the resolution is still lower than that of spaceborne optical SAR. Therefore, it is best to set the resolution of high-definition images to 1m here. Poor SAR imaging devices can all reach a resolution of 5m. Here, 5m resolution is used as the watershed between transitional images and blurred images to make the fusion of high-definition images and blurred images softer in the follow-up.

[0062] More specifically, the following formula is used to calculate the overall amplitude mean of the SAR image:

[0063]

[0064] Where P is the amplitude mean, q is the central gray level, k is the edge gray level, e is the Euclidean distance, n is the total number of pixels, and l is the distance from the ground;

[0065] Among them, the following formula is used to calculate the Euclidean distance:

[0066]

[0067] Where p i is the scattering coefficient of the i-th column pixel block in the high-definition image, is the scattering coefficient of the j-th pixel block in the i-th column pixel block of the high-definition image.

[0068] More specifically, the scattering characteristics include surface and volume scattering, double echo, combined scattering, penetration scattering, and dielectric property scattering.

[0069] Since SAR is an active side-looking radar system and the imaging geometry belongs to the slant range imaging type, there are significant differences between the schematic diagrams of slant range imaging and ground range imaging of ground images of the same size. Taking the same piece of ground as an example, as Figure 3a shown, it is the ground range imaging result of this ground. As Figure 3b shown, it is the slant range imaging result of this ground. Therefore, there are significant differences between SAR images and optical images in terms of imaging mechanism, geometric characteristics, radiation characteristics, etc. When the same object is imaged using any one of the scattering methods such as volume scattering, double echo, combined scattering, penetration scattering, or dielectric property scattering, the results are different from those of other scattering methods. Therefore, it is necessary to distinguish in terms of scattering characteristics. Among them, the scattering characteristic data mainly includes the working wavelength, incident angle, polarization mode, etc. of the radar sensor.

[0070] More specifically, when calculating the spectral-texture feature set of the high-definition image according to the scattering feature vector, it is calculated through an identification network model. The construction process of the identification network model specifically includes the following steps:

[0071] Construct and train a first recognition model based on the autoencoder structure. The first recognition model is used to map the input quantity into the feature space to obtain encoded data, and then map the encoded data back into the feature space to make the encoded data in the feature space as close as possible to the input data.

[0072] Construct a second recognition model, which includes the trained first recognition model and the gray-level co-occurrence matrix.

[0073] Construct a third recognition model, which includes the second recognition model and the texture model. The texture model includes several types of spectral indices.

[0074] Among them, the first recognition model adopts a deep autoencoder (DAE) formed by connecting in series an encoding module with 8 hidden layers and a decoding module with 8 hidden layers to extract the deep features of the data and reduce the dimension of the data.

[0075] In Embodiment 2, a remote sensing image target recognition system based on deep learning includes the following:

[0076] Image library: It includes a common object database and a target picture database. The common object database is used to store the image information of common object targets, and the target picture database is used to store the image information of target objects.

[0077] Object target positioning module: It is used to perform object target positioning operations on the SAR image according to the common object image library.

[0078] Image segmentation module: It is used to segment the located SAR image into high-definition images, transitional images, and blurred images.

[0079] Scattering feature calculation module: It is used to calculate the scattering feature vector based on the segmented high-definition image and blurred image.

[0080] Mapping matrix generation module: It is used to construct a recognition network model according to the scattering feature vector and calculate the feature mapping matrix based on the segmented high-definition image.

[0081] YOLOv5 network training module: It is used to train the YOLOv5 network according to the feature mapping matrix to obtain a high-resolution SAR image.

[0082] Target position recognition module: It is used to determine the position of the target object in the high-resolution SAR image according to the target picture database.

[0083] More specifically,

[0084] Fuzzy environment algorithm establishment module: used to establish a fuzzy environment algorithm, which is used to fully segment the SAR image to obtain an irregular image and the adjacent environment result of the irregular image;

[0085] Adjacent result comparison module: used to judge the segmentation result of the irregular image according to the adjacent environment result.

[0086] It can be understood that a remote sensing image target recognition system based on deep learning provided by the present invention corresponds to a remote sensing image target recognition method based on deep learning provided in each of the foregoing embodiments. The relevant technical features of a remote sensing image target recognition system based on deep learning can refer to the relevant technical features of a remote sensing image target recognition method based on deep learning, which will not be elaborated here.

[0087] In Embodiment 3, an electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the remote sensing image target recognition method based on deep learning is implemented.

[0088] Specifically, the above-mentioned processor may include a central processing unit, or a specific integrated circuit, or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0089] Among them, the memory may include a mass storage for data or instructions. By way of example and not limitation, the memory may include a hard disk drive, a floppy disk drive, a solid state drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus drive, or a combination of two or more of these. In a suitable case, the memory may include a removable or non-removable (or fixed) medium. In a suitable case, the memory may be internal or external to the data processing device. In a specific embodiment, the memory is a non-volatile memory. In a specific embodiment, the memory includes a read-only memory (abbreviated as ROM) and a random access memory. In a suitable case, the ROM may be a mask-programmed ROM, a programmable ROM (abbreviated as PROM), an erasable PROM, an electrically erasable PROM, an electrically rewritable ROM, or a flash memory (FLASH), or a combination of two or more of these. In a suitable case, the RAM may be a static random access memory or a dynamic random access memory (abbreviated as DRAM), where the DRAM may be a fast page mode dynamic random access memory, an extended data output dynamic random access memory, a synchronous dynamic random access memory, etc.

[0090] The memory can be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor.

[0091] The processor reads and executes the computer program instructions stored in the memory to implement any one of the above-described deep learning-based remote sensing image target recognition methods in the embodiments.

[0092] In Embodiment 4, a non-transitory computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the deep learning-based remote sensing image target recognition method is implemented.

[0093] The readable storage medium includes flash memory, mobile hard disks, multimedia cards, card-type memories, magnetic memories, magnetic disks, optical discs, etc. The readable storage medium may be an internal storage unit of an electronic device in some embodiments, such as the mobile hard disk of the electronic device. The readable storage medium may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a smart memory card, a secure digital card, a flash card, etc. equipped on the electronic device. The readable storage medium may also include both the internal storage unit and the external storage device of the electronic device. The readable storage medium can be used not only to store application software and various types of data installed in the electronic device, but also to temporarily store data that has been output or will be output.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered by the scope of the claims and the description of the present invention.

Claims

1. A method for target recognition of remote sensing images based on deep learning, characterized in that, it includes the following steps: Based on a common object image library, an attention mechanism is introduced to locate object targets in SAR images, and the SAR image is fully segmented into high-definition images, transition images, and blurred images according to the resolution value of the geometric center of the object target; among them, the object target with a geometric center resolution not exceeding 1m is a high-definition image, the object target with a geometric center resolution exceeding 1m and not exceeding 5m is a transition image, and the object target with a geometric center resolution greater than 5m is a blurred image; The step of fully segmenting the SAR image into high-definition images, transition images, and blurred images according to the resolution value of the geometric center of the object target includes: When the object target in the SAR image is completely located, by performing a neighboring environment calculation on the remaining irregular image that does not contain the object target, a neighboring environment result is obtained, and the neighboring environment result is the neighboring ratio of the irregular image to the transition image and the blurred image; If the neighboring ratio of the irregular image to the transition image in the neighboring environment result exceeds 70%, the irregular image in the SAR image is defined as a transition image; if the neighboring ratio of the irregular image to the transition image in the neighboring environment result does not exceed 70%, the irregular image in the SAR image is defined as a blurred image; among them, the calculation method of the neighboring ratio of the irregular image to the transition image is to first calculate the total perimeter of the irregular image, then calculate the first coincidence length of the edge of the irregular image and the edge of the transition image, and obtain the ratio of the first coincidence length to the total perimeter; the calculation method of the neighboring ratio of the irregular image to the blurred image is to first calculate the total perimeter of the irregular image, then calculate the second coincidence length of the edge of the irregular image and the edge of the blurred image, and obtain the ratio of the second coincidence length to the total perimeter; The step of fully segmenting the SAR image into high-definition images, transition images, and blurred images according to the resolution value of the geometric center of the object target further includes: The following formula is used to calculate the overall amplitude mean of the SAR image: Among them, P is the mean amplitude, q is the central gray level, k is the edge gray level, and E d is the Euclidean distance, m is the total number of pixels, and l is the distance from the ground; Among them, the following formula is used to calculate the Euclidean distance: where p i is the scattering coefficient of the i-th column pixel block in the high-definition image, and is the scattering coefficient of the j-th pixel block in the i-th column pixel block of the high-definition image; The overall scattering characteristics of the high-definition image are calculated by the Deeplab v3+ algorithm to obtain scattering characteristic data, and based on the scattering characteristic data, the transition image is sliced and the time series length is extracted to obtain a scattering characteristic vector; The EM algorithm is used to iteratively optimize the scattering characteristic vector to obtain several feature mapping matrices, the YOLOv5 network is trained using the feature mapping matrices, and finally the SAR image is input into the trained YOLOv5 network to obtain a high-resolution SAR image; Based on the target picture database, the position of the target object in the high-resolution SAR image is recognized.

2. The method for target recognition of remote sensing images based on deep learning according to claim 1, characterized in that, the scattering characteristics include surface and volume scattering, double echo, combined scattering, penetration scattering, and dielectric property scattering.

3. An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for identifying remote sensing image targets based on deep learning according to any one of claims 1 to 2.

4. A non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that when the computer program is executed by a processor, it implements the method for identifying remote sensing image targets based on deep learning according to any one of claims 1 to 2.

Images