A method for generating semantic intelligence from remote sensing images based on deep learning

Abstract

The present invention proposes a method for generating semantic intelligence of remote sensing images based on deep learning, comprising: obtaining geographical location information of a target through remote sensing image target recognition and target positioning; performing feature classification on the remote sensing image by using remote sensing image semantic segmentation; encoding the geographical location information of the target and the semantic segmentation result; and completing semantic intelligence generation by using relational triples and multi-task semantic description for the geographical location information, feature classification and encoding of the target.

Description

Technical Field

[0001] The present invention belongs to the field of satellite intelligent information processing and relates to a remote sensing image semantic intelligence generation method based on deep learning. Background Art

[0002] With the rapid development of remote sensing technology, the value of remote sensing data has gradually been reflected. Whether it is national defense and military or commercial applications, there is an urgent need for automatic extraction of remote sensing image information. Facing the military application needs of intelligent and autonomous operation of remote sensing satellite systems, networked and systematic collaboration, efficient and intelligent cognition of battlefield situations in the future intelligent warfare form, a new space-based remote sensing system with autonomous intelligence, efficient collaboration, and collective intelligence emergence is constructed. Using the new generation of artificial intelligence technologies represented by deep learning and knowledge reasoning, and focusing on human intelligence elements such as perception, cognition, communication, reasoning, and decision-making, a remote sensing satellite system that can achieve similar human intelligent behavior is designed - an intelligent remote sensing satellite with the ability to "see, speak, listen, and think". Based on the new generation of artificial intelligence methods, autonomous intelligence is realized in remote sensing satellite data acquisition and analysis, intelligence generation and push tasks.

[0003] Situational awareness theory and technology have been widely used in many fields such as combat command and control, network security, and intelligent transportation. With the development of science and technology, the amount of information obtained by remote sensing satellites has exploded. How to quickly and accurately obtain effective information from massive data has become a new research topic. In many organizational systems, humans are increasingly playing the role of understanding cognition and making command and control. However, in the face of massive data that is updated all the time, human understanding ability can neither complete the perception of a large amount of data, nor can its understanding speed match the speed of information update, which can easily cause a series of key targets and situations. The perception is wrong or the perception is not timely. Therefore, it is of great significance to develop remote sensing image semantic intelligence generation technology based on deep learning to improve the system's situational awareness ability. Summary of the invention

[0004] Aiming at the application requirements of intelligent remote sensing satellite systems, efficient perception of battlefield situations, and intelligent cognition in future intelligent warfare forms, the present invention proposes a method for generating semantic intelligence of remote sensing images based on deep learning.

[0005] The present invention is achieved through the following technical solutions.

[0006] A method for generating semantic intelligence of remote sensing images based on deep learning, comprising:

[0007] Obtain the target's geographical location information through remote sensing image target recognition and target positioning;

[0008] The remote sensing image semantic segmentation method is used to perform feature classification on the remote sensing image;

[0009] Encoding the geographic location information and semantic segmentation results of the target;

[0010] The target's geographic location information, feature classification, and coding are used to generate semantic intelligence using relational triples and multi-task semantic descriptions.

[0011] Beneficial effects of the present invention:

[0012] The present invention uses One-Hot encoding technology to encode the image after semantic segmentation, and can obtain slice data containing the target image according to the encoded information. Subsequently, only the slice data is analyzed and processed, which greatly reduces the data volume and effectively improves the computing efficiency. The relational triple algorithm involved can realize the rapid generation of semantic intelligence information. By designing a fixed semantic form, the results obtained by other processes in the present invention are nested into the fixed semantic model to obtain image semantic intelligence information. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flow chart of the method for generating semantic intelligence from remote sensing images based on deep learning of the present invention. DETAILED DESCRIPTION

[0014] The present invention will be further described below in conjunction with the accompanying drawings.

[0015] A method for generating remote sensing image semantic intelligence based on deep learning in this specific implementation manner includes:

[0016] Step 1: Obtain the target’s geographical location information through remote sensing image target recognition and target positioning;

[0017] In specific implementation, the existing deep learning-based target detection and recognition algorithm can be used to perform target detection and recognition tests on optical images and SAR images, which can effectively realize the detection and recognition of aircraft and ship targets. After the target detection and recognition, only the pixel position information of the target in the image coordinate system can be obtained. Therefore, the range-Doppler model is used in this embodiment to obtain the target geographical location information;

[0018] The range-Doppler model, referred to as the RD model, is expressed as follows:

[0019]

[0020] In formula (1), R st is the satellite-to-earth position vector, R t The position vector of the ground target, R s Satellite position vector; at a certain imaging time t, if the satellite position vector is known, the SAR receives a pulse with a delay of τ, indicating that the distance between the SAR satellite and the ground reflection point is a certain value |Rst |=cτ / 2, c is the speed of light; therefore, equation (1) is expressed as: The desired point R t =(x t ,y t ,z t ) must be distributed at a distance of |R from the satellite st | = cτ / 2;

[0021] Formula (2) is the expression of Doppler frequency shift, where f DC is the Doppler frequency shift, R=|R s -R t | is the distance from the satellite to the ground, V s is the satellite velocity vector, V t is the velocity vector of the ground target, λ is the wavelength of the radar beam; if f DC is a constant, then equation (2) is expressed as a cluster of hyperbolas, i.e., iso-Doppler lines. In specific implementation, due to the rotation of the earth, these curves are asymmetrically distributed relative to the subsatellite point D.

[0022] Formula (3) is the earth data model, h is the distance from a point P on the ground to The normal distance of the represented oblate spheroid, R e ,R p are the major and minor axes of the oblate ellipsoid; the solutions of equations (1) and (2) must be based on the coordinate frame based on this ellipsoid. The coordinates of the point R t =(x t ,y t ,z t ) is also expressed in this coordinate frame;

[0023] The RD model is solved by least squares, the satellite information of the image is obtained according to the satellite attitude and orbit information, and the geographical location information of the target is finally obtained by combining the location information of the target pixel in the image.

[0024] Step 2: Using remote sensing image semantic segmentation to perform feature classification on remote sensing images;

[0025] In this embodiment, the following method is specifically adopted:

[0026] 2.1 Construct an encoder based on the feature pyramid model to extract multi-scale features;

[0027] 2.2 Build a decoder to complete the segmentation of low-order and high-order features;

[0028] 2.3 The encoder and decoder are used to obtain high-order and low-order features. The high-level features extract semantic information, and the low-level features provide boundary information. More abstract features are extracted to complete high-accuracy classification, and denser predictions are generated to achieve fine semantic segmentation.

[0029] Step 3: Encode the geographic location information and semantic segmentation results of the target;

[0030] In the specific implementation, the remote sensing image is divided into 64 regions. The number of targets obtained in each region corresponding to step one is different, and the semantic segmentation results obtained in each region corresponding to step two are different. One-Hot encoding is used to complete the encoding of the 64 regions.

[0031] Combined with the target detection and recognition results achieved in step 1, the number of targets in different areas can be obtained. According to the number of targets in each area, the areas are marked with different colors. The darker the color, the more targets there are in the area.

[0032] Count the number of targets in different areas, coded as follows:

[0033] No target is defined as category one, coded as [1,0,0,0];

[0034] The number of targets is 1-2, which is category 2 and is coded as [0,1,0,0];

[0035] The number of targets is 3 to 5, which is category 3 and is coded as [0, 0, 1, 0];

[0036] Targets with more than 5 are of category 4 and are encoded as [0,0,0,1].

[0037] The codes of different regions can be obtained according to the number of targets. Different codes represent different numbers of regional targets, that is, different regional importance. Therefore, the regional importance can be converted into binary code.

[0038] Combined with the image semantic segmentation results achieved in step 2, the codes corresponding to different scenes such as different roads, rivers, farmlands and buildings can be counted.

[0039] The road is coded as [1,0,0,0], the river is coded as [0,1,0,0], the farmland is coded as [0,0,1,0], and the building is coded as [0,0,0,1]. Different scenes have different codes, so the scenes can be converted into binary code values.

[0040] Step 4: Generate semantic intelligence based on the target's geographic location information, feature classification, and coding using relational triples and multi-task semantic description;

[0041] In the specific implementation, the relationship triples are designed with a fixed semantic description model to form a description sentence such as "×× position (×× scene) contains ×× (target) ×× frame / ship", where the ×× information can be obtained through the remote sensing image target detection, recognition and positioning, remote sensing image semantic segmentation and semantic segmentation image coding and other technologies introduced in steps one to three. In the technology of relationship triples, multi-task semantic description technology is added to divide the generated structured text <object, attribute, activity, scene> into three tasks, each of which has a corresponding association relationship. At the same time, this relationship between tasks can improve the correctness of the generated structured text. Multi-task learning focuses on the connection between tasks. Through joint learning, different regression functions are learned for three tasks at the same time, taking into account both the differences between tasks and the connection between tasks. This is also one of the most important ideas of multi-task learning.

[0042] For example, if we identify "ship" and "port" in remote sensing images and divide them into two separate classification tasks, "port" may be misidentified, but if "ship" and "port" are two related tasks, the misidentification rate will be greatly reduced because the matching between "port" and "ship" is better. In addition, semantic intelligence can be further improved based on location information to increase situation information in key areas.

[0043] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating semantic intelligence of remote sensing images based on deep learning, characterized in that: include: Obtain the target's geographical location information through remote sensing image target recognition and target positioning; The remote sensing image semantic segmentation method is used to perform feature classification on the remote sensing image; Encoding the geographic location information and semantic segmentation results of the target; The geographic location information, feature classification, and coding of the target are used to complete the generation of semantic intelligence using relational triples and multi-task semantic description; The distance-Doppler model is used to obtain the target geographical location information. The distance-Doppler model, referred to as the RD model, is expressed as follows: In formula (1), R st is the satellite-to-earth position vector, R t is the position vector of the ground target, R s is the satellite position vector; at a certain imaging time t, if the satellite position vector is known, the SAR receives a pulse with a delay of τ, indicating that the distance between the SAR satellite and the ground reflection point is a certain value |R st |=cτ / 2, c is the speed of light; therefore, equation (1) is expressed as: The desired point R t =(x t ,y t ,z t ) must be distributed at a distance of |R from the satellite st | = cτ / 2; Formula (2) is the expression of Doppler frequency shift, where f dc is the Doppler frequency shift, R=|R s -R t | is the distance from the satellite to the ground, V s is the satellite velocity vector, V t is the velocity vector of the ground target, λ is the wavelength of the radar beam; if f dc is a constant, then equation (2) is expressed as a cluster of hyperbolas, i.e., equal Doppler lines; Formula (3) is the earth data model, h is the distance from a point P on the ground to The normal distance of the represented oblate spheroid, R e ,R p are the major and minor axes of the oblate ellipsoid; the solutions of equations (1) and (2) must be based on the coordinate frame based on this ellipsoid. The coordinates of the point R t =(x t ,y t ,z t ) is also expressed in this coordinate frame; The RD model is solved by least squares, the satellite information of the image is obtained according to the satellite attitude and orbit information, and the geographical location information of the target is finally obtained by combining the location information of the target pixel in the image; The method of using remote sensing image semantic segmentation to perform feature classification on remote sensing images specifically adopts the following method: 2.1 Construct an encoder based on the feature pyramid model to extract multi-scale features; 2.2 Build a decoder to complete the segmentation of low-order and high-order features; 2.3 The encoder and decoder are used to obtain high-order and low-order features, wherein the high-order features extract semantic information and the low-order features provide boundary information, thereby extracting more abstract features to complete high-accuracy classification and generate denser predictions to achieve fine semantic segmentation; When encoding the geographic location information and semantic segmentation results of the target, the remote sensing image is divided into 64 areas, each area has a different number of targets obtained in step one, and each area has a different semantic segmentation result obtained in step two, and One-Hot encoding is used to complete the encoding of the 64 areas.

2. The method for generating remote sensing image semantic intelligence based on deep learning according to claim 1, characterized in that: According to the number of targets in each area, the areas are marked with different colors. The darker the color, the more targets there are in the area.

Images