SAR image description method, related device and computer storage medium

Abstract

The invention provides an SAR image description method, a related device and a computer storage medium, and the method comprises the steps: carrying out the preprocessing of a scene for each scene in an SAR image after receiving the SAR image, and obtaining a target image; then, inputting the target image into the semantic segmentation model, outputting description information of the target image, inputting the target image into the image recognition model, and outputting a detection result of an object in the target image; finally, based on the description information of the target image and the detection result of the object in the target image, a description result of the SAR image is generated, the description result of the SAR image is finally generated by combining the output of the semantic segmentation model and the output of the image recognition model, the accuracy and applicability of content description of the SAR image are effectively improved, and the method is suitable for popularization and application. And more reliable image understanding support is provided for key applications such as reconnaissance and disaster response.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular relates to a SAR image description method, related device and computer storage medium. BACKGROUND

[0002] SAR, which stands for Synthetic Aperture Radar, is an advanced radar imaging technology that transmits microwave pulses and receives the echoes reflected by the ground surface. By utilizing the interference and processing of radar signals, it forms high-resolution two-dimensional or three-dimensional images.

[0003] SAR is widely used in key fields such as earth observation, environmental monitoring, and reconnaissance due to its unique advantages such as high resolution, all-weather, and all-day imaging.

[0004] Since SAR is based on electromagnetic scattering principle imaging, the resulting images have significant differences in visual appearance from ordinary optical images, such as the presence of speckle noise, lack of true color, and complex background clutter. When traditional image processing and analysis methods are used to process SAR images, they face multiple challenges, including sensitivity to noise and insufficient semantic description capabilities, which significantly reduce the accuracy and applicability of algorithms in key tasks such as reconnaissance or disaster monitoring. SUMMARY

[0005] Therefore, the present application provides a SAR image description method, related device and computer storage medium to overcome the core problems of speckle noise interference, lack of true color, and complex background clutter in the prior art, and to provide more reliable image understanding support for key applications such as reconnaissance and disaster response.

[0006] The first aspect of the present application provides a SAR image description method, comprising:

[0007] receiving a SAR image;

[0008] for each scene image in the SAR image, pre-processing the scene image to obtain a target image;

[0009] inputting the target image into a semantic segmentation model to output description information of the target image, and inputting the target image into an image recognition model to output a detection result of an object in the target image; the semantic segmentation model is obtained by training a deep learning model based on a self-attention mechanism by using a first training sample set; the first training sample set includes a plurality of first training sample data, and the first training sample data includes a first training sample image and real description information of the first training sample image; the image recognition model is obtained by training a target detection network by using a second training sample set; the second training sample set includes a plurality of second training sample data, and the second training sample data includes a second training sample image and a real class label of each object in the second training sample image;

[0010] generating a description result of the SAR image based on the description information of the target image and the detection result of the object in the target image.

[0011] Optionally, the target image is obtained by preprocessing each one of the SAR images, including:

[0012] The target image is obtained by inputting each one of the SAR images into a data preprocessing module; the data preprocessing module includes a denoising submodule, a compression submodule, an instance normalization and residual connection submodule; the denoising submodule performs denoising processing on the one image to obtain a feature map, the compression submodule performs compression processing on the feature map to obtain a compressed feature map, and the instance normalization and residual connection submodule performs instance normalization and residual connection processing on the compressed feature map to obtain the target image.

[0013] Optionally, the image recognition model further includes an edge detection module and a semantic structure framework.

[0014] The edge detection module segments the target image to obtain a segmented image; the target detection network identifies an object in each segmented image to obtain an identification result of each object; and the detection result of the object is generated based on the identification result of the object and the semantic structure framework.

[0015] Optionally, the backbone network in the target detection network includes a residual connection, a dense connection, a deformable convolution, a feature pyramid and an attention module; the feature pyramid adopts a weighted bidirectional feature pyramid network.

[0016] Optionally, the description result of the SAR image is generated based on the description information of the target image and the detection result of the object in the target image, including:

[0017] The detection result of the object in the target image is supplemented to the description information of the target image as supplementary information of the description information of the target image, and a description result of the SAR image is obtained.

[0018] The second aspect of the present application provides a SAR image description device, comprising:

[0019] A receiving unit is configured to receive a SAR image.

[0020] A preprocessing unit is configured to preprocess each scene in the SAR image to obtain a target image.

[0021] A first input unit is configured to input the target image into a semantic segmentation model to output description information of the target image, and input the target image into an image recognition model to output a detection result of an object in the target image. The semantic segmentation model is obtained by training a deep learning model based on a self-attention mechanism using a first training sample set. The first training sample set includes a plurality of first training sample data, and the first training sample data includes a first training sample image and true description information of the first training sample image. The image recognition model is obtained by training a target detection network using a second training sample set. The second training sample set includes a plurality of second training sample data, and the second training sample data includes a second training sample image and a true class label of each object in the second training sample image.

[0022] A generating unit is configured to generate a description result of the SAR image based on the description information of the target image and the detection result of the object in the target image.

[0023] Optionally, the preprocessing unit comprises:

[0024] A second input unit is configured to input each scene in the SAR image into a data preprocessing module to output a target image. The data preprocessing module includes a denoising submodule, a compression submodule, an instance normalization and residual connection submodule. The denoising submodule performs denoising processing on the scene to obtain a feature map. The compression submodule performs compression processing on the feature map to obtain a compressed feature map. The instance normalization and residual connection submodule performs instance normalization and residual connection processing on the compressed feature map to obtain a target image.

[0025] Optionally, the image recognition model further comprises an edge detection module and a semantic structure framework.

[0026] The edge detection module segments the target image to obtain a segmented image; the target detection network identifies an object in each segmented image to obtain an identification result of each object; and the detection result of the object is generated based on the identification result of the object and a semantic structure framework.

[0027] Optionally, the backbone network in the target detection network comprises residual connection, dense connection, deformable convolution, a feature pyramid and an attention module; and the feature pyramid adopts a weighted bidirectional feature pyramid network.

[0028] Optionally, the generating unit comprises:

[0029] The generating subunit supplements the description information of the target image by taking the detection result of the object in the target image as supplementary information of the description information of the target image, to obtain a description result of the SAR image.

[0030] The third aspect of the present application provides an electronic device comprising:

[0031] one or more processors;

[0032] a storage device having one or more programs stored thereon;

[0033] When the one or more programs are executed by the one or more processors, the one or more processors implement the SAR image description method according to any one of the first aspect.

[0034] The fourth aspect of the present application provides a computer storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the SAR image description method according to any one of the first aspect.

[0035] As can be seen from the above scheme, the present application provides a SAR image description method, related device and computer storage medium, which comprises receiving a SAR image, pre-processing each scene image in the SAR image to obtain a target image, inputting the target image into a semantic segmentation model to output description information of the target image, inputting the target image into an image recognition model to output a detection result of an object in the target image, and finally generating a description result of the SAR image based on the description information of the target image and the detection result of the object in the target image. The present application combines the outputs of the semantic segmentation model and the image recognition model to finally generate the description result of the SAR image, effectively improves the accuracy and applicability of the content description of the SAR image, and provides more reliable image understanding support for key applications such as reconnaissance and disaster response. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below only constitute a part of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0037] Figure 1 A specific flowchart of a SAR image description method provided by an embodiment of the present application is shown in the figure.

[0038] Figure 2 A schematic diagram of a SAR image description device provided by another embodiment of the present application is shown in the figure.

[0039] Figure 3 A schematic diagram of an electronic device for implementing a SAR image description method provided by another embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0041] The term "comprising" and its variants as used herein are open-ended, that is "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions of other terms will be given in the following description.

[0042] It should be noted that the terms "first", "second", and the like mentioned in the present application are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0043] It should be noted that the terms "one", "multiple" in the present application are illustrative and not restrictive, and those skilled in the art should understand that unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0044] An embodiment of the present application provides a SAR image description method, as shown in the figure, which specifically includes the following steps: Figure 1

[0045] ​S101, receive a SAR image.

[0046] In the practical application process of the present application, the SAR image can be obtained by carrying a synthetic aperture radar (SAR) system on an unmanned aerial vehicle, but is not limited thereto.

[0047] S102, for each scene in the SAR image, pre-processing the scene to obtain a target image.

[0048] Since the SAR image is affected by the coherent speckle noise and has a high dynamic range, directly using the original image for detection will reduce the performance.

[0049] Therefore, the present application designs an end-to-end learnable data preprocessing module, which can suppress noise and compress dynamic range while maintaining the unique scattering information and edge features of the SAR image. The data preprocessing module includes a denoising submodule, a compression submodule, an instance normalization and residual connection submodule.

[0050] The denoising submodule performs denoising processing on the scene to obtain a feature map. The compression submodule performs compression processing on the feature map to obtain a compressed feature map. The instance normalization and residual connection submodule performs instance normalization and residual connection processing on the compressed feature map to obtain a target image.

[0051] In the practical application process of the present application, the denoising submodule can adaptively process noise in different regions by combining multi-scale convolution and attention mechanism.

[0052] Specifically, the multi-scale convolution in the present application extracts multi-scale features of the image by using different sizes of convolution kernels (such as 1x1, 3x3, 5x5) in parallel, effectively capturing hierarchical information from texture to semantics, thereby enhancing the robustness to noise. And through the attention mechanism, dynamically weight the key area, suppress noise interference, improve the attention of important features of the model, the combination of the two can significantly improve the noise suppression effect.

[0053] In the practical application process of the present application, residual learning can also be used to avoid detail loss, which is not limited herein.

[0054] In the practical application process of the present application, the denoising submodule can compress the SAR image with a high dynamic range to a range suitable for network processing by using learnable parameters and multi-stage processing, while maintaining the contrast between the target and the background.

[0055] Wherein, the learnable parameters in the application can be but are not limited to scaling factors, adjusting the brightness range of the image using the learnable parameters, mapping the high dynamic range values to a processable range (such as 0-1), and in the practical application of the application, the compression process can also be smoothed by piecewise linear mapping (such as Log transformation) to avoid information loss.

[0056] Wherein, the multi-stage processing in the application includes but is not limited to applying a global scaling parameter to compress the overall dynamic range, preserving large-scale structures, focusing on target areas through a learnable attention mechanism, enhancing contrast, using residual connections to preserve original details, and preventing blurring caused by excessive compression, which is not limited here.

[0057] In the practical application of the application, after compression, a learnable contrast stretching (such as CLahe) can be but is not limited to applied to dynamically adjust the contrast of the target and the background, and the target boundary is automatically detected through an adaptive threshold (such as the Otsu method) to achieve precise contrast adjustment.

[0058] In the practical application of the application, instance normalization in the instance normalization and residual connection sub-module stabilizes training by standardizing the activation value distribution, and residual connection alleviates gradient vanishing through cross-layer direct connection signals, both of which cooperate to improve model convergence speed and generalization ability, so that the preprocessed target image is suitable for network processing and also retains necessary original information.

[0059] In the practical application of the application, the above-mentioned learnable data preprocessing module can be embedded into a target detection network such as YOLO for joint training, which is not limited here.

[0060] Based on the above, in another embodiment of the application, one implementation of step S102 specifically includes:

[0061] For each scene image in the SAR image, the scene image is input into the data preprocessing module, and a target image is output.

[0062] S103, inputting the target image into a semantic segmentation model to output description information of the target image.

[0063] Wherein, the semantic segmentation model is obtained by training a deep learning model based on a self-attention mechanism using a first training sample set; the first training sample set includes a plurality of first training sample data, and the first training sample data includes a first training sample image and true description information of the first training sample image.

[0064] It can be understood that data set construction is the cornerstone of image understanding and description tasks, and neural network-based methods require a large amount of data for learning, so it is crucial to construct a rich SAR image understanding and description data set.

[0065] The implementation of cross-modal recognition through semantic segmentation requires a language description of the main recognition target of each picture in the first training sample set, and the content of the language description needs to be accurate and appropriate. Too long sentences require higher capabilities of the deep learning network, which may easily cause redundant input information. Too short sentences may result in a small number of training sets for deep learning, and poor learning effect.

[0066] In the practical application of the present application, the semantic segmentation model can use, but is not limited to, a Transformer deep learning model based on a regional attention mechanism, which is not limited here. After receiving the SAR image, the semantic segmentation model encodes the image vector using an encoder, then decodes using a decoder, and generates natural language based on the decoding result, i.e., description information.

[0067] The key of the Transformer is the self-attention mechanism (Self-Attention), which gives the model the ability to dynamically focus on the correlation between different positions in the sequence data when processing sequence data, without relying on fixed window size or processing data in order. In the image description generation task, the self-attention mechanism can help the model grasp the relationship between different regions in the image and different words in the text description, and then generate more accurate and image content consistent description text. In the present application, the model considers the information of all other elements in the sequence when calculating the representation of an element. For example, when processing a sentence, the model will consider the influence of other words such as "on" and "grass" on "cat" when calculating the representation of the word "cat". In this way, the Transformer can capture the global dependencies within the sequence.

[0068] In the practical application of the present application, the training method of the semantic segmentation model can be, but is not limited to, inputting the first training sample data into the deep learning model based on the self-attention mechanism, outputting the predicted description information of the first training sample image, then adjusting the parameters in the deep learning model based on the self-attention mechanism based on the error between the true description information of the first training sample image and the predicted description information of the first training sample image, until the preset convergence condition (such as the preset number of iterations, the error is less than the preset threshold, etc.) is reached, which is not limited here.

[0069] S104, input the target image into the image recognition model, and output the detection result of the object in the target image.

[0070] The image recognition model is obtained by training a target detection network by using a second training sample set.

[0071] The real class label of each object in the second training sample image focuses more on the detailed description of the object, such as target class, model, state and the like.

[0072] It can be understood that, Figure 1 It is only an example of the embodiment of the present application, and steps S103 and S104 have no sequence in actual application, and step S103 can be performed before step S104, or step S104 can be performed before step S103, or steps S103 and S104 can be performed simultaneously, which is not limited here.

[0073] Optionally, in another embodiment of the present application, the image recognition model further comprises an edge detection module and a semantic structure framework.

[0074] In actual application, the edge detection module segments the target image to obtain a segmented image; the target detection network identifies the object in the segmented image to obtain the identification result of each object; and the detection result of the object is generated based on the identification result of the object and the semantic structure framework.

[0075] The edge detection module is used for feature extraction and feature detection, and aims to detect the edge with obvious change or the discontinuous region in a digital image. In one-dimensional space, the similar operation is called step detection. The edge is the boundary line between different regions in an image, and generally an edge image is a binary image. The purpose of edge detection is to capture the region with sharp change in brightness.

[0076] Specifically, but not limited to, an edge filter can be used to realize edge detection, which is not limited here. The edge filter extracts the edge by finding the boundary pixels of the brighter and darker regions. The filter finds the part with obvious gradient change in the image, and the gradient is generally described as the amplitude and direction of the edge. Then, all the pixels with high edge amplitude are selected to complete the extraction of the edge contour of the region.

[0077] In the specific implementation process of the present application, the image information obtained by recognition can also be expressed in words by pre-constructing a language structure framework, for example: "There are abc. here." a is the target number, b is the target class, and c is the target state. "There are 7 ships parked on the shore", wherein a is "7", b is "ship", and c is "parked on the shore".

[0078] It should be noted that a, b, c are all coding information.

[0079] Optionally, in another embodiment of the application, the backbone network in the target detection network comprises residual connection, dense connection, deformable convolution, feature pyramid and attention module; the feature pyramid adopts a weighted bidirectional feature pyramid network.

[0080] It should be noted that the target detection network can be but is not limited to a YOLO deep learning network, which is not limited here.

[0081] In the actual application process of the application, the YOLO structure is adjusted and optimized according to the SAR image characteristics, so that the optimization algorithm is more suitable for SAR images, and the image recognition effect is better.

[0082] The backbone network in the target detection network in the application is used to solve the problem that the traditional convolution is easily disturbed by noise due to the influence of the coherent speckle noise of the SAR image, residual connection and dense connection are used in the backbone network to strengthen feature propagation, alleviate gradient vanishing, and different levels of information are fused by using multi-level feature fusion, which brings significant performance improvement and model robustness enhancement. Deformable convolution (Deformable Convolution) is introduced in the backbone network to adapt to the geometric deformation of the target. Attention modules, such as SE (Squeeze-and-Excitation) modules or CBAM (Convolutional Block Attention Module), are added to the backbone network, which makes the network pay attention to important feature channels and spatial positions, which are not limited here. By embedding attention modules in the backbone network and the detection head, the network can suppress noise and pay more attention to target areas.

[0083] Due to the large difference in target scale in the SAR image, such as small boats near the coast and large ships in the open sea, the feature pyramid in the target detection network in the application adopts a more dense feature pyramid structure, such as BiFPN (weighted bidirectional feature pyramid network), to strengthen the fusion of multi-scale features, which is not limited here.

[0084] In the actual application process of the application, the backbone network in the target detection network also uses loss function optimization (Loss Function Optimization), and Focal Loss is used for classification loss to solve the problem of unbalanced positive and negative samples in the SAR image, which is not limited here.

[0085] In the practical application of this invention, the training method of the semantic segmentation model can be, but is not limited to, inputting the second training sample data into the object detection network, outputting the predicted category label of each object in the second training sample image, and then adjusting the parameters in the object detection network based on the error between the true category label of each object in the second training sample image and the predicted category label of each object in the second training sample image until the preset convergence condition is reached (such as the preset number of iterations, the error being less than the preset threshold, etc.), which is not limited here.

[0086] It is understandable that the accuracy of image recognition determines the credibility of subsequent image descriptions. To quantitatively measure the image recognition results, the evaluation metrics for object recognition tasks can include, but are not limited to, accuracy, recall, F1 score, and confusion matrix. No specific criteria are set here.

[0087] Accuracy: Represents the proportion of samples correctly predicted by the model out of the total number of samples; it is the most intuitive performance metric. It is usually represented by P.

[0088] Where TP (True Positive): The number of samples correctly predicted as positive. FP (False Positive): The number of samples incorrectly predicted as positive.

[0089] Recall: Recall R is the proportion of all samples that are actually positive that are predicted as positive by the model.

[0090] Where FN (False Negative) is the number of false negatives, which are incorrectly predicted as negative.

[0091] F1 Score: The F1 score is the harmonic mean of precision and recall, used to comprehensively consider the performance of precision and recall.

[0092] ;

[0093] Confusion Matrix: A matrix plotted with the number of predicted categories on the horizontal axis and the number of true labels on the vertical axis. The diagonal represents the number of cases where the model predictions match the data labels.

[0094] Evaluation metrics for image description tasks can include, but are not limited to, BLEU (Bilingual Evaluation Understudy), METEOR (Metric for Evaluation of Translation with Explicit Ordering), ROUGE (Recall-Oriented Understudy for Gisting Evaluation), etc., and are not limited here.

[0095] BLEU is an evaluation metric based on exact n-gram matching, initially used in machine translation tasks and later widely applied to image captioning tasks. It calculates a score by comparing the degree of n-gram overlap between the generated and reference captions. BLEU is simple to compute and can reflect the degree of lexical matching between the generated and reference captions relatively well. However, it relies too heavily on lexical matching, fails to capture semantic similarity, and is insensitive to the fluency of the generated captions.

[0096] Calculation method: When calculating BLEU scores, different lengths of n-grams (such as single words, word pairs, triples, etc.) are considered, and a penalty term is added to avoid generating descriptions that are too short.

[0097] METEOR is an evaluation metric that comprehensively considers multiple factors such as word matching, word order, and word meaning. It not only calculates exact word matching but also considers semantic matching (via a dictionary) and word order matching. METEOR is more comprehensive than BLEU, better capturing semantic similarity and word order information. However, its calculation is relatively complex and relies on an external dictionary, which may be affected by the dictionary's quality.

[0098] Calculation method: METEOR calculates the word matching score, word meaning matching score, word order matching score, etc., and combines them with a harmonic mean to obtain the final score.

[0099] ROUGE is primarily used to evaluate the lexical overlap between generated and reference text, typically for text summarization tasks, but it can also be used for image captioning tasks. It focuses more on the lexical recall of the generated description relative to the reference description. ROUGE is simple to use and can reflect the lexical coverage of the generated description relative to the reference description quite well. However, it relies too heavily on lexical matching and fails to capture semantic similarity.

[0100] Calculation method: There are several variants of ROUGE, such as ROUGE-N (recall based on n-gram) and ROUGE-L (recall based on the longest common subsequence).

[0101] S105. Based on the description information of the target image and the detection results of objects in the target image, generate the description results of the SAR image.

[0102] Optionally, in another embodiment of the present invention, one implementation of step S105 specifically includes:

[0103] The detection results of objects in the target image are used as supplementary information to the description information of the target image, thus obtaining the description result of the SAR image.

[0104] It should be noted that before supplementing the descriptive information of the target image, it is necessary to first associate and match the objects in the descriptive information of the target image with the detection results of the objects in the target image. Specifically, this can be achieved by using, but not limited to, the spatial location of the objects in the target image (such as image coordinates, region boundaries) to ensure that the objects in the target image output by the semantic segmentation model correspond precisely with the detection results of the objects in the target image output by the image recognition model, thus ensuring that the information is fused from the same object. No restrictions are imposed here.

[0105] Specifically, the detection results of objects in the target image generated by the image recognition model are directly added as qualifiers to the description information of the target image generated by the semantic segmentation model.

[0106] Suppose the semantic segmentation model outputs the following: the description of the target image is a boat moored on the shore. It can be seen that there are two objects in this description: object A is the boat and object B is the shore.

[0107] Suppose the image recognition model outputs the following result: there are two objects in the target image, object C is a ship of model XXX, and object D is port YYY.

[0108] First, we perform association matching based on the spatial location of objects in the target image. After matching, we determine that object A and object C are the same object, and object B and object D are the same object. We then add the description of object C output by the image recognition model to the beginning of object A as a qualifier, i.e., a ship of type XXX. We add the description of object D output by the image recognition model to the beginning of object B as a qualifier, i.e., the shore of port YYY. Therefore, the final description of the SAR image is: a ship of type XXX is moored on the shore of port YYY.

[0109] As can be seen from the above scheme, the present invention provides a method for describing SAR images. After receiving a SAR image, each scene in the SAR image is preprocessed to obtain a target image. Then, the target image is input into a semantic segmentation model, which outputs descriptive information of the target image, and the target image is input into an image recognition model, which outputs the detection results of objects in the target image. Finally, based on the descriptive information of the target image and the detection results of objects in the target image, a description result of the SAR image is generated. By combining the outputs of the semantic segmentation model and the image recognition model, the present invention ultimately generates a description result of the SAR image, effectively improving the accuracy and applicability of SAR image content description, and providing more reliable image understanding support for key applications such as reconnaissance and disaster response.

[0110] Another embodiment of the present invention provides a SAR image description device, such as... Figure 2 As shown, it specifically includes:

[0111] The receiving unit 201 is used to receive SAR images.

[0112] The preprocessing unit 202 is used to preprocess each scene in the SAR image to obtain the target image.

[0113] Optionally, in another embodiment of the present invention, one implementation of the preprocessing unit 202 includes:

[0114] The second input unit takes each scene in the SAR image, inputs it into the data preprocessing module, and outputs the target image.

[0115] The data preprocessing module includes a denoising submodule, a compression submodule, an instance normalization submodule, and a residual connection submodule. The denoising submodule denoises a scene image to obtain a feature map, the compression submodule compresses the feature map to obtain a compressed feature map, and the instance normalization and residual connection submodule performs instance normalization and residual connection processing on the compressed feature map to obtain the target image.

[0116] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0117] The first input unit 203 is used to input the target image into the semantic segmentation model and output the descriptive information of the target image, and to input the target image into the image recognition model and output the detection result of the object in the target image.

[0118] The semantic segmentation model is obtained by training a deep learning model based on a self-attention mechanism using a first training sample set. The first training sample set includes multiple first training sample data, which includes first training sample images and their true descriptive information. The image recognition model is obtained by training an object detection network using a second training sample set. The second training sample set includes multiple second training sample data, which includes second training sample images and their true category labels for each object in the second training sample images.

[0119] Optionally, in another embodiment of the present invention, the image recognition model further includes an edge detection module and a semantic structure framework;

[0120] The edge detection module segments the target image to obtain segmented images; the object detection network identifies objects in each segmented image to obtain the recognition result for each object; based on the object recognition result and semantic structure framework, the object detection result is generated.

[0121] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0122] Optionally, in another embodiment of the present invention, the backbone network in the target detection network includes residual connections, dense connections, deformable convolutions, feature pyramids, and attention modules; the feature pyramid adopts a weighted bidirectional feature pyramid network.

[0123] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0124] The generation unit 204 is used to generate a description result of the SAR image based on the description information of the target image and the detection results of objects in the target image.

[0125] For details on the specific operation of the units disclosed in the above embodiments of the present invention, please refer to the corresponding method embodiments, such as... Figure 1 As shown, it will not be elaborated further here.

[0126] Optionally, in another embodiment of the present invention, one implementation of the generation unit 204 specifically includes:

[0127] The generation subunit is used to supplement the description information of the target image by using the detection results of objects in the target image as supplementary information, and thus obtain the description result of the SAR image.

[0128] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0129] As can be seen from the above scheme, the present invention provides a SAR image description device. After the receiving unit 201 receives the SAR image, the preprocessing unit 202 preprocesses each scene in the SAR image to obtain the target image. Then, the first input unit 203 inputs the target image into the semantic segmentation model and outputs the description information of the target image, and inputs the target image into the image recognition model and outputs the detection results of objects in the target image. Finally, the generation unit 204 generates the description result of the SAR image based on the description information of the target image and the detection results of objects in the target image. The present invention, by combining the outputs of the semantic segmentation model and the image recognition model, finally generates the description result of the SAR image, effectively improving the accuracy and applicability of the SAR image content description, and providing more reliable image understanding support for key applications such as reconnaissance and disaster response.

[0130] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0131] Another embodiment of the present invention provides an electronic device, such as... Figure 3 As shown, it includes:

[0132] One or more processors 301.

[0133] Storage device 302, on which one or more programs are stored.

[0134] When the one or more programs are executed by the one or more processors 301, the one or more processors 301 implement the SAR image description method as described in the above embodiments.

[0135] Another embodiment of the present invention provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the SAR image description method as described in the above embodiments.

[0136] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0137] It should be noted that the computer-readable medium described above in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0138] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0139] Another embodiment of the present invention provides a computer program product, which, when executed, is used to perform the above-described SAR image description method.

[0140] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined in the methods of the embodiments of the present invention.

[0141] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in this invention is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely exemplary forms for implementing the invention.

[0142] While several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of the invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0143] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the specific combination of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with technical features of the present invention (but not limited to) that have similar functions.

Claims

1. A method for describing SAR images, characterized in that, include: Receive SAR images; For each scene in the SAR image, the scene is preprocessed to obtain the target image; The target image is input into a semantic segmentation model, which outputs descriptive information of the target image; and the target image is input into an image recognition model, which outputs detection results of objects in the target image. The semantic segmentation model is trained using a first training sample set on a deep learning model based on a self-attention mechanism. The first training sample set includes multiple first training sample data, each including first training sample images and their true descriptive information. The image recognition model is trained using a second training sample set on an object detection network. The second training sample set includes multiple second training sample data, each including second training sample images and their true category labels for each object in the second training sample images. Based on the description information of the target image and the detection results of the objects in the target image, a description result of the SAR image is generated.

2. The SAR image description method according to claim 1, characterized in that, The step of preprocessing each scene in the SAR image to obtain the target image includes: For each scene in the SAR image, the scene is input to the data preprocessing module, and the target image is output. The data preprocessing module includes a denoising submodule, a compression submodule, and an instance normalization and residual connection submodule. The denoising submodule denoises the scene to obtain a feature map, the compression submodule compresses the feature map to obtain a compressed feature map, and the instance normalization and residual connection submodule performs instance normalization and residual connection processing on the compressed feature map to obtain the target image.

3. The SAR image description method according to claim 1, characterized in that, The image recognition model also includes an edge detection module and a semantic structure framework; The edge detection module segments the target image to obtain segmented images; the target detection network identifies objects in each segmented image to obtain an identification result for each object; and based on the object identification result and semantic structure framework, generates the object detection result.

4. The SAR image description method according to claim 1, characterized in that, The backbone network of the target detection network includes residual connections, dense connections, deformable convolutions, feature pyramids, and attention modules; the feature pyramid adopts a weighted bidirectional feature pyramid network.

5. The SAR image description method according to claim 1, characterized in that, The step of generating a description result for the SAR image based on the description information of the target image and the detection results of objects in the target image includes: The detection results of objects in the target image are used as supplementary information to the description information of the target image to supplement the description information of the target image, thereby obtaining the description result of the SAR image.

6. A SAR image description device, characterized in that, include: The receiving unit is used to receive SAR images; The preprocessing unit is used to preprocess each scene in the SAR image to obtain the target image; The first input unit is used to input the target image into a semantic segmentation model and output descriptive information of the target image, and to input the target image into an image recognition model and output the detection result of the object in the target image; wherein, the semantic segmentation model is trained by a deep learning model based on a self-attention mechanism using a first training sample set; the first training sample set includes multiple first training sample data, and the first training sample data includes first training sample images and real descriptive information of the first training sample images; the image recognition model is trained by a target detection network using a second training sample set; the second training sample set includes multiple second training sample data, and the second training sample data includes second training sample images and real category labels of each object in the second training sample images; The generation unit is used to generate a description result of the SAR image based on the description information of the target image and the detection results of objects in the target image.

7. The SAR image description apparatus according to claim 6, characterized in that, The preprocessing unit includes: The second input unit, for each scene in the SAR image, inputs the scene to the data preprocessing module and outputs the target image. The data preprocessing module includes a denoising submodule, a compression submodule, and an instance normalization and residual connection submodule. The denoising submodule denoises the scene to obtain a feature map; the compression submodule compresses the feature map to obtain a compressed feature map; and the instance normalization and residual connection submodule performs instance normalization and residual connection processing on the compressed feature map to obtain the target image.

8. The SAR image description apparatus according to claim 6, characterized in that, The image recognition model also includes an edge detection module and a semantic structure framework; The edge detection module segments the target image to obtain segmented images; the target detection network identifies objects in each segmented image to obtain an identification result for each object; and based on the object identification result and semantic structure framework, generates the object detection result.

9. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the SAR image description method as described in any one of claims 1 to 5.

10. A computer storage medium, characterized in that, It stores a computer program, wherein the computer program, when executed by a processor, implements the SAR image description method as described in any one of claims 1 to 5.

Images