Target on-orbit intelligent identification method and device for rotating swing-scan super-wide satellite

Abstract

The target on-orbit intelligent identification method, device and storage medium for the rotating swing-scan super-wide satellite provided in the application first constructs a lightweight target identification model, then tunes the lightweight target identification model using a historical remote sensing image dataset to obtain an optimal lightweight target identification model, inputs the obtained to-be-identified remote sensing image into the optimal lightweight target identification model, obtains a shallow feature map through a shallow feature extraction module in the optimal lightweight target identification model, inputs the shallow feature map into a plurality of serially connected network sub-modules to obtain corresponding feature maps, and then inputs the feature maps into a feature fusion module to obtain a target identification result of the to-be-identified remote sensing image and output the target identification result. As can be seen, the application provides a method for identifying targets in remote sensing images on a super-wide satellite, thereby meeting the requirement of improving the on-orbit target identification efficiency of the satellite under the condition of limited resources on the satellite, and realizing the rapid identification of targets by the satellite.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of rotating swing-scan super-wide satellite target identification, and particularly relates to a target in-orbit intelligent identification method and device for a rotating swing-scan super-wide satellite and a storage medium. BACKGROUND

[0002] With the rapid development of earth observation technology, different amplitude widths and resolutions of high, medium and low orbit satellites jointly constitute an earth observation network. Among them, a super-wide remote sensing satellite adopts a new imaging system of vertical orbit scanning and along-orbit splicing, has an orbit height of 500 km, a space resolution of 1 meter at a subsatellite point, and an imaging width of 3000 km, and can realize global coverage of the earth's surface in a single day. However, the data volume of remote sensing images obtained by the super-wide satellite in unit time is as high as 420 Gbps, far exceeding the bandwidth of data transmission and the storage space of the super-wide satellite. Therefore, how to identify targets in remote sensing images on the super-wide satellite is a problem to be solved. SUMMARY

[0003] The application provides a target in-orbit intelligent identification method and device for a rotating swing-scan super-wide satellite and a storage medium to propose a method for identifying targets in remote sensing images on a super-wide satellite.

[0004] The first aspect embodiment of the application provides a target in-orbit intelligent identification method for a rotating swing-scan super-wide satellite, which comprises the following steps:

[0005] Constructing a lightweight target identification model, wherein the lightweight target identification model comprises a shallow feature extraction module, a network sub-module and a feature fusion module;

[0006] Optimizing the lightweight target identification model using a historical remote sensing image dataset to obtain an optimal lightweight target identification model;

[0007] Inputting the obtained to-be-identified remote sensing image into the optimal lightweight target identification model to obtain a shallow feature map through the shallow feature extraction module in the optimal lightweight target identification model;

[0008] Inputting the shallow feature map into a plurality of serially connected network sub-modules to obtain corresponding feature maps;

[0009] Inputting the feature maps into a feature fusion module to obtain a target identification result of the to-be-identified remote sensing image and outputting the target identification result.

[0010] The second aspect embodiment of the application provides a target in-orbit intelligent detection device for a rotating swing-scan super-wide satellite, which comprises the following steps:

[0011] The constructing module is configured to construct a lightweight target recognition model, wherein the lightweight target recognition model comprises a shallow feature extraction module, a network sub-module, and a feature fusion module.

[0012] The tuning module is configured to tune the lightweight target recognition model using a historical remote sensing image dataset to obtain an optimal lightweight target recognition model.

[0013] The first processing module is configured to input the obtained remote sensing image to be recognized into the optimal lightweight target recognition model, and obtain a shallow feature map through the shallow feature extraction module in the optimal lightweight target recognition model.

[0014] The second processing module is configured to input the shallow feature map into a plurality of network sub-modules connected in series to obtain corresponding feature maps.

[0015] The output module is configured to input the feature maps into the feature fusion module to obtain a target recognition result of the remote sensing image to be recognized and output the target recognition result.

[0016] The computer storage medium provided in the third aspect of the present application, wherein the computer storage medium stores computer schedulable instructions; the computer schedulable instructions can implement the method in the first aspect of the present application when scheduled by a processor.

[0017] The computer device provided in the fourth aspect of the present application, wherein the computer device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor; the processor can implement the method in the first aspect of the present application when executing the program.

[0018] The technical scheme provided by the embodiments of the present application at least brings the following beneficial effects:

[0019] In the target on-orbit intelligent recognition method, device and storage medium for a rotating scanning super-wide satellite provided in the present application, a lightweight target recognition model is first constructed, wherein the lightweight target recognition model comprises a shallow feature extraction module, a network sub-module, and a feature fusion module; then the lightweight target recognition model is tuned using a historical remote sensing image dataset to obtain an optimal lightweight target recognition model; the obtained remote sensing image to be recognized is input into the optimal lightweight target recognition model, a shallow feature map is obtained through the shallow feature extraction module in the optimal lightweight target recognition model, the shallow feature map is input into a plurality of network sub-modules connected in series to obtain corresponding feature maps, and the feature maps are input into the feature fusion module to obtain a target recognition result of the remote sensing image to be recognized and output the target recognition result. Therefore, the present application provides a method for target recognition on a super-wide satellite, thereby improving the on-orbit target recognition efficiency of the satellite under the condition of limited resources on the satellite and realizing fast target recognition of the satellite.

[0020] Additional aspects and advantages of the present application will be made apparent by the following description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and be more readily understood through consideration of the following description, taken in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a target on-orbit intelligent identification method for a rotating swing-scan super-wide satellite according to an embodiment of the present application is shown in FIG. 1.

[0023] Figure 2 A structural diagram of a network sub-module according to an embodiment of the present application is shown in FIG. 2.

[0024] Figure 3 A flowchart of a deep separable mixed hollow convolution according to an embodiment of the present application is shown in FIG. 3.

[0025] Figure 4 A flowchart of a target on-orbit intelligent identification method based on an optimal lightweight target identification model according to an embodiment of the present application is shown in FIG. 4.

[0026] Figure 5 A structural diagram of a target on-orbit intelligent detection device for a rotating swing-scan super-wide satellite according to an embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0027] Embodiments of the present application are described in detail below with reference to examples illustrated in the accompanying drawings, in which the same or similar components have the same or similar designations throughout. The embodiments described below are examples intended to provide an explanation of the present application and are not intended to restrict the present application.

[0028] A target on-orbit intelligent identification method and device for a rotating swing-scan super-wide satellite according to embodiments of the present application are described below with reference to the accompanying drawings.

[0029] Embodiment One

[0030] Figure 1 A flowchart of a target on-orbit intelligent identification method for a rotating swing-scan super-wide satellite according to an embodiment of the present application is shown in FIG. 1, which can include: Figure 1

[0031] Step 101, constructing a lightweight target identification model.

[0032] In the present application, the lightweight target identification model includes a shallow feature extraction module, a network sub-module, and a feature fusion module.​

[0033] Step 102, using a historical remote sensing image dataset to optimize a lightweight target recognition model to obtain an optimal lightweight target recognition model.

[0034] In the present application, the optimization of the lightweight target recognition model by the historical remote sensing image dataset can include the following steps:

[0035] Step a, preprocessing the historical remote sensing image dataset.

[0036] In the present application, the preprocessing of the historical remote sensing image dataset can include data enhancement (e.g., rotating the image angle, adding white noise, etc.) of the data in the historical remote sensing image dataset.

[0037] Step b, dividing the preprocessed remote sensing image dataset into a training set and a validation set.

[0038] Step c, inputting the training set into the lightweight target recognition model to calculate the loss function of the lightweight target recognition model.

[0039] Step d, according to the loss value of the loss function, the validation set is used to optimize the lightweight target recognition model to obtain the optimal lightweight target recognition model.

[0040] In addition, it should be noted that the present application optimizes the lightweight target recognition model using the historical remote sensing image dataset in the ground data center, which can optimize the lightweight target recognition model using ground resources to improve the efficiency of optimizing the lightweight target recognition model. After obtaining the optimal lightweight target recognition model, the obtained optimal lightweight target recognition model is transmitted to the satellite, and the optimal lightweight target recognition model is loaded and run on the on-board board card, so that the super wide swath satellite can realize intelligent target recognition of the acquired remote sensing image in orbit, solving the problem of "super wide swath satellite cannot transmit remote sensing image to the ground for processing".

[0041] Step 103, inputting the acquired remote sensing image to be identified into the optimal lightweight target recognition model to obtain a shallow feature map through a shallow feature extraction module in the optimal lightweight target recognition model.

[0042] In the present application, the above-mentioned shallow feature extraction module adopts a focus module. The focus module realizes 2 times down-sampling of features without information loss through slicing operation and one layer of convolution operation, and accelerates the inference speed of the network.

[0043] Step 104, inputting the shallow feature map into a plurality of serially connected network sub-modules to obtain corresponding feature maps.

[0044] In the present application, the optimal lightweight target recognition model can include N network sub-modules connected in series, wherein N is a positive integer.

[0045] In the present application, the method of inputting the shallow feature map into a plurality of network sub-modules connected in series to obtain the corresponding feature map can include:

[0046] The obtained shallow feature map is input into the optimal lightweight target recognition model, the input shallow feature map is a scale one feature, then the shallow feature map is input into the first network sub-module to obtain a scale two feature, the scale two feature is input into the second network sub-module to obtain a scale three feature, and so on, the obtained scale features are sequentially input into the remaining N-2 lightweight computing units to obtain N-2 corresponding scale features, and the obtained N scale features are stacked to obtain the corresponding feature map.

[0047] Further, in the present application, Figure 2 A structural diagram of a network sub-module is provided in the present application. As shown in Figure 2 The network sub-module includes branch 1 and branch 2. Among them, branch 1 extracts local fine-grained features through convolution and standardization operations, and branch 2 extracts large-scale features through lightweight computing units.

[0048] In the present application, the method of inputting the shallow feature map into the first network sub-module to obtain the scale two feature map can include the following steps:

[0049] Step one, the input shallow feature map is evenly divided into two sub-feature maps through channel splitting in the first network sub-module.

[0050] Step two, input the two sub-feature maps into branch 1 and branch 2 in the first network sub-module respectively to obtain the first scale feature map corresponding to branch 1 and the second scale feature map corresponding to branch 2.

[0051] In the present application, the two sub-feature maps can be randomly input into branch 1 and branch 2 in the first network sub-module.

[0052] In the present application, the two sub-feature maps are simultaneously input into branch 1 and branch 2 in the first network sub-module.

[0053] Step three, the first scale feature map and the second scale feature map are fused by feature channel mixing to obtain the scale two feature map.

[0054] In the present application, the operation of the remaining network sub-modules is similar to the above steps one to three, which can be referred to.

[0055] Further, the lightweight computing unit in the present application is composed of depth separable hybrid atrous convolution. Figure 3A flowchart of a deep separable mixed hollow convolution provided in the present application is shown.

[0056] In step 105, the feature map is input into the feature fusion module to obtain a target recognition result of the to-be-recognized remote sensing image and output.

[0057] In the present application, the feature fusion module is processed by global average pooling on the input feature map.

[0058] In the present application, the target recognition result can include multiple target recognition types in the to-be-recognized remote sensing image. For example, in the present application, the target recognition of the to-be-recognized remote sensing image is ground object classification, and the target recognition result can include houses, airports, ports, farmland, and mountains.

[0059] In the present application, after the super-wide satellite obtains the target recognition result, the obtained target recognition result is returned to the ground. At this time, the data amount of the target recognition result is less than the data transmission bandwidth of the super-wide satellite, so that the target recognition result can be returned to the ground through the super-wide satellite. In the present application, if the target recognition result is related to the target of interest, the recognition result and the image containing the target are returned to the ground at the same time.

[0060] Based on the above content, the steps 103-105 are illustrated based on the obtained optimal lightweight target detection model in the present application.

[0061] Figure 4 A flowchart of a target on-orbit intelligent recognition method based on an optimal lightweight target recognition model according to an embodiment of the present application is shown. As shown in Figure 4 The optimal lightweight target recognition model includes three network sub-modules connected in series.

[0062] Specifically, as shown in Figure 4 In the present application, the to-be-recognized remote sensing image obtained is input into the shallow feature extraction module in the optimal lightweight target recognition model to obtain a shallow feature map. Then, the obtained shallow feature map is a scale one feature, the shallow feature map is input into the first network sub-module to obtain a scale two feature map, the obtained scale two feature is input into the second network sub-module to obtain a scale three feature, and the scale one feature, the scale two feature, and the scale three feature are stacked to obtain a corresponding feature map.

[0063] Further, in the present application, the obtained feature map is input into the feature fusion module to obtain target type 1 to target type M of the to-be-recognized remote sensing image, and the obtained target recognition result is output to the ground.

[0064] The target on-orbit intelligent identification method, device and storage medium for a rotating swing-scan ultra-wide satellite provided in the application first constructs a lightweight target identification model, wherein the lightweight target identification model comprises a shallow feature extraction module, a network submodule and a feature fusion module, then the lightweight target identification model is fine-tuned using a historical remote sensing image dataset to obtain an optimal lightweight target identification model, the obtained to-be-identified remote sensing image is input into the optimal lightweight target identification model, the shallow feature map is obtained through the shallow feature extraction module in the optimal lightweight target identification model, the shallow feature map is input into a plurality of serially connected network submodules to obtain corresponding feature maps, and then the feature maps are input into the feature fusion module to obtain the target identification result of the to-be-identified remote sensing image and output. As can be seen, the application provides a method for identifying targets in remote sensing images on an ultra-wide satellite, thereby meeting the requirement of improving the on-orbit target identification efficiency of the satellite under the condition of limited resources on the satellite and realizing the rapid identification of targets by the satellite.

[0065] Embodiment two

[0066] Further, Figure 5 The structure diagram of the target on-orbit intelligent detection device for a rotating swing-scan ultra-wide satellite provided according to an embodiment of the application is shown in Figure 5 , which can include:

[0067] The construction module 501 is configured to construct a lightweight target identification model, wherein the lightweight target identification model comprises a shallow feature extraction module, a network submodule and a feature fusion module.

[0068] The fine-tuning module 502 is configured to fine-tune the lightweight target identification model using a historical remote sensing image dataset to obtain an optimal lightweight target identification model.

[0069] The first processing module 503 is configured to input the obtained to-be-identified remote sensing image into the optimal lightweight target identification model to obtain a shallow feature map through the shallow feature extraction module in the optimal lightweight target identification model.

[0070] The second processing module 504 is configured to input the shallow feature map into a plurality of serially connected network submodules to obtain corresponding feature maps.

[0071] The output module 505 is configured to input the feature maps into the feature fusion module to obtain the target identification result of the to-be-identified remote sensing image and output.

[0072] In order to realize the above-mentioned embodiments, the disclosure further provides a computer storage medium.

[0073] The computer storage medium provided in the embodiments of the disclosure stores an executable program; after the executable program is executed by a processor, the method shown in Figure 1 can be realized.

[0074] To achieve the above-mentioned embodiments, the present disclosure further proposes a computer device.

[0075] The computer device provided by the embodiments of the present disclosure comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor; when the processor executes the program, the method as shown in Figure 1 can be implemented.

[0076] To achieve the above-mentioned embodiments, the present disclosure further proposes a mobile terminal device.

[0077] The mobile terminal device provided by the embodiments of the present application comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and when the processor executes the program, the method as shown in Figure 1 can be implemented.

[0078] The target on-orbit intelligent identification method and device for the rotating swing-scan super-wide satellite and the storage medium provided by the present application first construct a lightweight target identification model, wherein the lightweight target identification model comprises a shallow feature extraction module, a network submodule and a feature fusion module, then the lightweight target identification model is fine-tuned using a historical remote sensing image dataset to obtain an optimal lightweight target identification model, the obtained to-be-identified remote sensing image is input into the optimal lightweight target identification model, the shallow feature map is obtained through the shallow feature extraction module in the optimal lightweight target identification model, the shallow feature map is input into a plurality of serially connected network submodules to obtain corresponding feature maps, and then the feature maps are input into the feature fusion module to obtain the target identification result of the to-be-identified remote sensing image and output. As can be seen, the present application provides a method for identifying targets in remote sensing images on a super-wide satellite, thereby meeting the requirement of improving the on-orbit target identification efficiency of the satellite under the condition of limited resources on the satellite, and realizing the rapid identification of targets by the satellite.

[0079] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0080] Any processes or methods described in the flow charts or otherwise described herein can be understood as representing modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions (or steps) and / or can be implemented entirely or substantially in hardware, and / or a combination of hardware and software. The embodiments of the present application can broadly include a number of hardware components and / or machine-executable instructions to implement processes or methods described in the flow charts or otherwise described herein, and the scope of the present application should not be limited by the scope of the embodiments described herein, which are exemplary in nature. It should be understood that the functions of the components and / or the functions of the machine-executable instructions can be combined into a single function or spread over several functions.

[0081] Although the embodiments of the present application have been shown and described above, it should be understood that the above-described embodiments are exemplary, and should not be understood as limiting the present application, and those ordinarily skilled in the art can make changes, modifications, replacements, and variations to the above-described embodiments within the scope of the present application.

Claims

1. A target on-orbit intelligent identification method for a rotating swing-scan ultra-wide swath satellite, characterized in that, The method comprises: constructing a lightweight target recognition model, wherein the lightweight target recognition model comprises a shallow feature extraction module, a network submodule and a feature fusion module, the shallow feature extraction module adopts a focus module, wherein the focus module is through a slicing operation and a one-layer convolution operation; optimizing the lightweight target recognition model using a historical remote sensing image dataset to obtain an optimal lightweight target recognition model; inputting the obtained to-be-recognized remote sensing image into the optimal lightweight target recognition model to obtain a shallow feature map through the shallow feature extraction module in the optimal lightweight target recognition model; inputting the shallow feature map into a plurality of serially connected network submodules to obtain a corresponding feature map; inputting the feature map into a feature fusion module to obtain a target recognition result of the to-be-recognized remote sensing image and outputting the target recognition result; the optimal lightweight target recognition model comprises N serially connected network submodules, wherein N is a positive integer, the inputting of the shallow feature map into the plurality of serially connected network submodules to obtain the corresponding feature map comprises: inputting the obtained shallow feature map into the optimal lightweight target recognition model, the input shallow feature map being a scale-1 feature, then inputting the shallow feature map into a first network submodule to obtain a scale-2 feature, inputting the scale-2 feature into a second network submodule to obtain a scale-3 feature, and so on, sequentially inputting the obtained scale features through the remaining N-2 lightweight calculation units to obtain N-2 scale features, and stacking the obtained N scale features to obtain a corresponding feature map; the network submodule comprises a branch 1 and a branch 2, the inputting of the shallow feature map into the first network submodule to obtain a scale-2 feature map comprises: averaging the input shallow feature map into two sub-feature maps through channel splitting in the first network submodule; inputting the two sub-feature maps into the branch 1 and the branch 2 in the first network submodule respectively to obtain a first scale feature map corresponding to the branch 1 and a second scale feature map corresponding to the branch 2; performing feature channel aliasing fusion on the first scale feature map and the second scale feature map to obtain a scale-2 feature map; the branch 2 comprises a lightweight calculation unit, wherein the lightweight calculation unit is composed of a depth separable mixed hollow convolution.

2. The method of claim 1, wherein, the optimization of the lightweight target recognition model through the historical remote sensing image dataset comprises: preprocessing the remote sensing image dataset; dividing the preprocessed remote sensing image dataset into a training set and a validation set; inputting the training set into the lightweight target recognition model to calculate a loss function of the lightweight target recognition model; optimizing the lightweight target recognition model using the validation set according to the loss value of the loss function to obtain an optimal lightweight recognition detection model.

3. A device for on-orbit intelligent detection of targets for a rotating-swing-scan ultra-wide swath satellite, characterized in that, The device comprises: a construction module configured to construct a lightweight target recognition model, wherein the lightweight target recognition model comprises a shallow feature extraction module, a network submodule and a feature fusion module, the shallow feature extraction module adopts a focus module, wherein the focus module is through a slicing operation and a one-layer convolution operation; The tuning module is configured to tune the lightweight target recognition model using a historical remote sensing image dataset to obtain an optimal lightweight target recognition model. The first processing module is configured to input the obtained remote sensing image to be recognized into the optimal lightweight target recognition model, and obtain a shallow feature map through a shallow feature extraction module in the optimal lightweight target recognition model. The second processing module is configured to input the shallow feature map into a plurality of network sub-modules connected in series to obtain a corresponding feature map. The output module is configured to input the feature map into a feature fusion module to obtain a target recognition result of the remote sensing image to be recognized and output the target recognition result. The optimal lightweight target recognition model comprises N network sub-modules connected in series, where N is a positive integer, The second processing module is configured to input the obtained shallow feature map into the optimal lightweight target recognition model, the input shallow feature map is a scale one feature, then the shallow feature map is input into a first network sub-module to obtain a scale two feature, the scale two feature is input into a second network sub-module to obtain a scale three feature, and the obtained scale features are sequentially input into remaining N-2 lightweight calculation units to obtain corresponding N-2 scale features, and the obtained N scale features are stacked to obtain a corresponding feature map. The network sub-module comprises a branch 1 and a branch 2, The input shallow feature map is input into the first network sub-module to obtain a scale two feature map, comprising: The input shallow feature map is evenly divided into two sub-feature maps through channel splitting in the first network sub-module; The two sub-feature maps are input into the branch 1 and the branch 2 in the first network sub-module respectively to obtain a first scale feature map corresponding to the branch 1 and a second scale feature map corresponding to the branch 2; The first scale feature map and the second scale feature map are fused through feature channel mixing to obtain a scale two feature map; the branch 2 comprises a lightweight calculation unit, and the lightweight calculation unit is composed of a depth separable mixed hollow convolution.

4. A computer storage medium, wherein, The computer storage medium stores computer executable instructions; the computer executable instructions are executed by the processor to implement the method in any of claims 1-2.

5. A computer device, comprising: The computer program is stored in the memory and executable on the processor, and the processor executes the program to implement the method in any of claims 1-2.

6. A mobile terminal, characterized in that The computer program is stored in the memory and executable on the processor, and the processor executes the program to implement the method in any of claims 1-2.

Images