A method and device for evaluating the quality of a SAR image and a storage medium

Abstract

The application discloses a quality evaluation method and device of a SAR image and a storage medium, and comprises the following steps: collecting a SAR image corresponding to a target region by using a SAR satellite; inputting the SAR image into a pre-trained quality evaluation model; determining a plurality of first feature maps corresponding to the SAR image by using each time convolution model in the quality evaluation model, wherein the time convolution model comprises a plurality of time convolution networks, and the receptive fields of the time convolution networks are different; and performing quality evaluation on the SAR image corresponding to the target region based on the plurality of first feature maps and by using the quality evaluation model. The plurality of time convolution models are used for feature extraction on the SAR image, so that the accuracy of feature extraction can be maximally ensured.

Description

Technical Field

[0001] This application relates to the field of SAR image processing technology, and in particular to a method, apparatus and storage medium for quality assessment of SAR images. Background Technology

[0002] SAR imagery is a high-resolution radar image acquired through active microwave remote sensing technology. Unlike optical sensors, SAR satellites do not rely on sunlight; instead, they image by emitting electromagnetic waves and receiving backscattered signals from the target area. Because SAR images are not based on optical imaging but rather on the emission of electromagnetic waves, SAR satellites can operate in all weather conditions and are not limited by lighting conditions. Furthermore, the microwave signals emitted by SAR satellites can penetrate clouds and dust, making them suitable for most areas.

[0003] In SAR images, the horizontal axis typically represents the azimuth, and the vertical axis represents the slant range. The pixel value of each pixel in a SAR image reflects the backscattering intensity of the target at the corresponding location. Furthermore, the azimuth of a SAR image refers to the direction of the SAR satellite's flight trajectory, which is directly related to time series. Therefore, the spatial information of a SAR image is essentially derived from a time-series signal.

[0004] Furthermore, in the current quality assessment of SAR images, convolutional neural networks (CNNs) or Vit are commonly used to process SAR images. While CNNs and Vit perform well in processing spatially distributed data (e.g., optical remote sensing images), they have certain limitations when processing SAR images, which are essentially time-series signals. For example, since the pixel values ​​of optical remote sensing images directly reflect the spatial radiometric characteristics of ground features, CNNs can effectively extract features from neighboring pixels belonging to the same ground feature, or Vit's self-attention mechanism can effectively extract ground feature distribution features across the entire image.

[0005] However, since the spatial information of SAR images is essentially derived from time-series signals, directly using convolutional neural networks (CNNs) or Vit to process SAR images and evaluate their quality may result in inaccuracies.

[0006] The publication number is CN118967632A, and the title is "A Method and System for Constructing a SAR Image Quality Assessment Model." The method includes: constructing a training set and a test set; constructing a machine learning model; based on the training set, iteratively optimizing the hyperparameters of the machine learning model using an improved Harris Eagle optimization algorithm to obtain the optimal hyperparameter combination; based on the optimal hyperparameter combination and the test set, iteratively optimizing the weight coefficients of the machine learning model to obtain the optimal weight coefficient combination; and obtaining an evaluation model based on the optimal hyperparameter combination and the optimal weight coefficient combination.

[0007] The publication number is CN117291873A, and the title is "A SAR Image Quality Assessment Method Based on Correlation and Discrimination." It includes the following steps: S1. At two different times—one when the SAR is not disturbed by noise from a civilian radiation source or active interference source of the same frequency, and the other when it is disturbed by noise—two single-look complex images of the same ground feature are acquired by the SAR. The correlation coefficient of the two images after registration is calculated, and the corresponding interference-to-signal ratio is obtained by establishing a quantitative relationship between the correlation coefficient and the interference-to-signal ratio. S2. The forms of interference-to-signal ratio and signal-to-interference-to-clutter ratio are established, and the interference-to-signal ratio is converted into the signal-to-interference-to-clutter ratio of the SAR image disturbed by noise. S3. Based on the signal-to-interference-to-clutter ratio, a national image interpretability rating table based on the SAR image is calculated, and it is converted into the corresponding NIIRS of the infrared detector. Then, the discrimination probability of the target to be identified in the image is calculated.

[0008] The existing technologies mentioned above have limitations. Since the spatial information of SAR images is essentially derived from time-series signals, if models suitable for processing spatial analysis data are used to evaluate the quality of SAR images, there may be technical problems of inaccurate evaluation. Currently, no effective solution has been proposed. Summary of the Invention

[0009] The embodiments of this disclosure provide a method, apparatus, and storage medium for evaluating the quality of SAR images, in order to at least solve the technical problem in the prior art that, since the spatial information of SAR images is essentially derived from time-series signals, if a model suitable for processing spatial analysis data is used to evaluate the quality of SAR images, inaccurate evaluation may occur.

[0010] According to one aspect of the present disclosure, a method for quality assessment of a SAR image is provided, comprising: acquiring a SAR image corresponding to a target region using a SAR satellite, wherein the horizontal axis of the SAR image represents the time-related azimuth direction, the vertical axis of the SAR image represents the slant range, and the pixel value of the SAR image represents the backscattering intensity corresponding to the azimuth direction and the slant range; inputting the SAR image into a pre-trained quality assessment model, and using various temporal convolutional models in the quality assessment model to determine multiple first feature maps corresponding to the SAR image, wherein the temporal convolutional model includes a multi-layer temporal convolutional network, and each layer of the temporal convolutional network has a different receptive field; and performing quality assessment on the SAR image corresponding to the target region based on the multiple first feature maps and using the quality assessment model.

[0011] According to another aspect of the present disclosure, a storage medium is also provided, the storage medium including a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0012] According to another aspect of the present disclosure, a SAR image quality assessment apparatus is also provided, comprising: a SAR image acquisition module, configured to acquire a SAR image corresponding to a target area using a SAR satellite, wherein the horizontal axis of the SAR image represents the time-related azimuth direction, the vertical axis of the SAR image represents the slant range, and the pixel value of the SAR image represents the backscattering intensity corresponding to the azimuth direction and the slant range; a first feature map determination module, configured to input the SAR image into a pre-trained quality assessment model and determine multiple first feature maps corresponding to the SAR image using various temporal convolutional models in the quality assessment model, wherein the temporal convolutional model includes a multi-layer temporal convolutional network, and the receptive field of each layer of the temporal convolutional network is different; and a quality assessment module, configured to perform quality assessment on the SAR image corresponding to the target area based on the multiple first feature maps and using the quality assessment model.

[0013] According to another aspect of the present disclosure, a SAR image quality assessment apparatus is also provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions to perform the following processing steps: acquiring a SAR image corresponding to a target area using a SAR satellite, wherein the horizontal axis of the SAR image represents the time-related azimuth direction, the vertical axis of the SAR image represents the slant range, and the pixel value of the SAR image represents the backscattering intensity corresponding to the azimuth direction and slant range; inputting the SAR image into a pre-trained quality assessment model, and using various temporal convolutional models in the quality assessment model to determine multiple first feature maps corresponding to the SAR image, wherein the temporal convolutional model includes a multi-layer temporal convolutional network, and each layer of the temporal convolutional network has a different receptive field; and performing quality assessment on the SAR image corresponding to the target area based on the multiple first feature maps and using the quality assessment model.

[0014] This application discloses a method for quality assessment of SAR images. First, a SAR satellite acquires SAR images corresponding to a target region. Then, the SAR satellite inputs the acquired SAR images into a pre-trained quality assessment model and uses various temporal convolutional models in the quality assessment model to determine multiple first feature maps corresponding to the SAR images. Finally, the SAR satellite performs quality assessment on the SAR images corresponding to the target region based on the multiple first feature maps and the quality assessment model.

[0015] This application discloses a quality assessment model, which includes multiple temporal convolutional models. These temporal convolutional models are primarily used for modeling and analyzing time-dimensional signals or sequence data, rather than traditional spatial images. Furthermore, since the quality assessment in this application focuses on SAR images, and the spatial information of SAR images is essentially derived from time-series signals, compared to existing technologies that utilize convolutional neural networks and Vit models suitable for processing spatial information, this application employs multiple temporal convolutional models for feature extraction from SAR images, thereby maximizing the accuracy of feature extraction.

[0016] This solves the technical problem in existing technologies where, since the spatial information of SAR images is essentially derived from time-series signals, using models suitable for processing spatial analysis data to evaluate the quality of SAR images may result in inaccurate evaluations. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this application, illustrate exemplary embodiments of this disclosure and are used to explain this disclosure, but do not constitute an undue limitation of this disclosure. In the drawings:

[0018] Figure 1This is a schematic diagram of the communication connection system between a SAR satellite and a terminal device according to Embodiment 1 of this application;

[0019] Figure 2A This is a schematic diagram of the hardware architecture of the SAR satellite according to Embodiment 1 of this application;

[0020] Figure 2B This is a schematic diagram of the hardware architecture of the terminal device according to Embodiment 1 of this application;

[0021] Figure 3 This is a flowchart of the SAR image quality assessment method according to Embodiment 1 of this application;

[0022] Figure 4 This is a schematic diagram of the quality assessment model according to Embodiment 1 of this application;

[0023] Figure 5A This is a schematic diagram of the convolution process of the temporal convolution model TCN-1 according to Embodiment 1 of this application;

[0024] Figure 5B This is a schematic diagram of the convolution process of the temporal convolution model TCN-2 according to Embodiment 1 of this application;

[0025] Figure 6 This is a schematic diagram of the SAR image quality assessment device according to Embodiment 2 of this application; and

[0026] Figure 7 This is a schematic diagram of the SAR image quality assessment device according to Embodiment 3 of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] According to this embodiment, a method embodiment for quality assessment of SAR images is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] Figure 1 A schematic diagram of a communication connection system between a SAR satellite and a terminal device according to this embodiment is shown. The system includes a terminal device 10, a ground system 20, and a SAR satellite 30. The terminal device 10 sends a command to the SAR satellite 30 via the ground system 20 to perform a quality assessment of a SAR image corresponding to a target area. The processor of the SAR satellite 30 receives and responds to the command, acquires a SAR image corresponding to the target area, and inputs the SAR image into a pre-trained quality assessment model. Finally, the quality assessment model outputs a quality assessment result corresponding to the SAR image.

[0032] Figure 2A Further shown Figure 1 A schematic diagram of the hardware architecture of the ZhongSAR-30 satellite. (Reference) Figure 2A As shown, SAR satellite 30 includes an integrated electronic system, which comprises a processor, a memory, a bus management module, and a communication interface. The memory is connected to the processor, allowing the processor to access the memory, read program instructions stored in the memory, read data from the memory, or write data to the memory. The bus management module is connected to the processor and also to a bus such as a CAN bus. Thus, the processor can communicate with onboard peripherals connected to the bus through the bus managed by the bus management module. Furthermore, the processor also communicates with devices such as cameras, star sensors, telemetry and command transponders, and data transmission equipment via the communication interface. Those skilled in the art will understand that… Figure 2A The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, a satellite system may also include... Figure 2A The more or fewer components shown, or having the same Figure 2A The different configurations shown.

[0033] Figure 2B Further shown Figure 1 A schematic diagram of the hardware architecture of the middle terminal device 10. (Reference) Figure 2B As shown, the terminal device 10 may include one or more processors (processors may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, transmission device, and input / output interface are connected to the processor via a bus. In addition, it may also include a display, keyboard, and cursor control device connected to the input / output interface. Those skilled in the art will understand that... Figure 2B The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, the ground system may also include... Figure 2B The more or fewer components shown, or having the same Figure 2B The different configurations shown.

[0034] It should be noted that, Figure 2A and Figure 2B One or more processors and / or other data processing circuits shown herein may generally be referred to as "data processing circuitry". This data processing circuitry may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be integrated, in whole or in part, into any other element in a computing device. As involved in embodiments of this disclosure, the data processing circuitry serves as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0035] Figure 2A and Figure 2B The memory shown can be used to store software programs and modules for application software, such as the program instructions / data storage device corresponding to the SAR image quality assessment method in this embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the SAR image quality assessment method of the aforementioned application. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.

[0036] It should be noted here that, in some optional embodiments, the above... Figure 2A and Figure 2B The device shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 2A and Figure 2B This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned devices.

[0037] Under the aforementioned operating environment, according to the first aspect of this embodiment, a SAR image quality assessment method is provided, which consists of... Figure 2A The SAR satellite 30 shown is implemented. Figure 3 A flowchart illustrating the method is shown below. (Refer to...) Figure 3 As shown, the method includes:

[0038] S302: SAR images corresponding to the target area are acquired using SAR satellites. The horizontal axis of the SAR image represents the azimuth direction related to the time series, the vertical axis of the SAR image represents the slant range, and the pixel value of the SAR image represents the backscatter intensity corresponding to the azimuth direction and slant range.

[0039] S304: Input the SAR image into a pre-trained quality assessment model, and use the various temporal convolutional models in the quality assessment model to determine multiple first feature maps corresponding to the SAR image. The temporal convolutional models include multi-layer temporal convolutional networks, and each layer has a different receptive field.

[0040] S306: Based on multiple first feature maps and using a quality assessment model, perform quality assessment on the SAR image corresponding to the target area.

[0041] Specifically, firstly, the operator uses terminal device 10 and ground system 20 to send a command to SAR satellite 30 to evaluate the quality of the SAR image corresponding to the target area. The processor of SAR satellite 30 receives and responds to this command, acquiring the SAR image corresponding to the target area (S302). Here, the azimuth direction is taken as the flight direction of SAR satellite 30, and each row in the SAR image represents the signal strength reflected back by SAR satellite 30 at the same slant range from different azimuth directions. Furthermore, all pixels in that row correspond to the same slant range between the SAR satellite and the target area. Similarly, the slant range direction is taken as the side-looking direction of SAR satellite 30, and each column in the SAR image represents the signal strength reflected back at different slant ranges in the same azimuth direction. Furthermore, all pixels in that column correspond to the same azimuth range between the SAR satellite and the target area.

[0042] Furthermore, given the x-coordinate and y-coordinate corresponding to the SAR image, the signal intensity of the electromagnetic wave backscattered along a unique azimuth and slant range is determined. In practice, SAR images are grayscale images, and the pixel values ​​in the SAR image correspond to the signal intensity of the electromagnetic wave backscattered along a unique azimuth and slant range. That is, the greater the backscattering intensity, the larger the pixel value; the smaller the backscattering intensity, the smaller the pixel value.

[0043] For example, determining the azimuth in a SAR image slant distance Furthermore, each combination of azimuth and diagonal distance determines a unique pixel value. With pixel value correspond, With pixel value correspond, With pixel value correspond,..., With pixel value correspond. With pixel value correspond, With pixel value correspond, With pixel value correspond,..., With pixel value Corresponding. And so on. With pixel value correspond, With pixel value correspond, With pixel value correspond,..., With pixel value correspond.

[0044] And due to the orientation With time Correspondingly, therefore at this moment With time series vectors Corresponding, and Orientation With time Correspondingly, therefore at this moment With time series vectors Corresponding, and And so on. Orientation With time Correspondingly, therefore at this moment With time series vectors Corresponding, and .

[0045] Therefore, a SAR image can actually be represented by multiple time-series vectors. These time-series vectors indicate the distribution of pixel values ​​at different slant distances at a given time. Each element in the time-series vector indicates the pixel value corresponding to a different slant distance.

[0046] In other words, the spatial information of SAR images is essentially derived from time-series signals, and SAR images can be represented by multiple time-series vector sequences.

[0047] Thus, the SAR satellite 30 inputs the SAR image into the pre-trained quality assessment model and uses the various temporal convolutional models in the quality assessment model to determine multiple first feature maps corresponding to the SAR image (S304). Figure 4 This is a schematic diagram of the quality assessment model according to an embodiment of this application. (Reference) Figure 4 As shown, the quality assessment model includes multiple temporal convolutional models TCN-1 to TCN-U, and each temporal convolutional model contains multiple layers of temporal convolutional networks. For example, the temporal convolutional model TCN-1 includes multiple layers of temporal convolutional networks, and the kernel size of each layer is the same, but the stride of sliding on the SAR image is different, resulting in different receptive fields formed by dilated convolution. That is, the receptive field is expanded by using the interval sampling method of each layer of temporal convolutional networks to capture long-term dependencies (i.e., dilated convolution).

[0048] Figure 5A This is a schematic diagram of the convolution process of the temporal convolution model TCN-1 according to an embodiment of this application. (Reference) Figure 5A As shown, the first temporal convolutional network (input layer) of the TCN-1 temporal convolutional model has a kernel size of 3. The second temporal convolutional network (hidden layer) of the TCN-1 temporal convolutional model has a kernel size of 3 and a stride of 1. The third temporal convolutional network (hidden layer) of the TCN-1 temporal convolutional model has a kernel size of 3 and a stride of 2. That is, the receptive field of the third temporal convolutional network (hidden layer) is 7, the receptive field of the second temporal convolutional network (hidden layer) is 5, and the receptive field of the first temporal convolutional network (input layer) is 3. The receptive field of the third temporal convolutional network is larger than that of the second temporal convolutional network, and the receptive field of the second temporal convolutional network is larger than that of the first temporal convolutional network.

[0049] Figure 5B This is a schematic diagram of the convolution process of the temporal convolution model TCN-2 according to an embodiment of this application. (Reference) Figure 5BAs shown, the first temporal convolutional network (input layer) of the TCN-2 model has a kernel size of 3 and a stride of 1. The second temporal convolutional network (hidden layer) of the TCN-2 model has a kernel size of 3 and a stride of 2. The second temporal convolutional network (hidden layer) of the TCN-3 model has a kernel size of 3 and a stride of 3. That is, the receptive field of the third temporal convolutional network (hidden layer) is 9, the receptive field of the second temporal convolutional network (hidden layer) is 7, and the receptive field of the first temporal convolutional network (input layer) is 5. The receptive field of the third temporal convolutional network is larger than that of the second temporal convolutional network, and the receptive field of the second temporal convolutional network is larger than that of the first temporal convolutional network.

[0050] contrast Figure 5A and Figure 5B It can be seen that the receptive fields of each layer of the temporal convolutional network in the first-time convolutional model and the second-time convolutional model are not the same.

[0051] Therefore, unlike existing technologies that use convolutional neural networks or ViT to extract features from SAR images, this application uses a temporal convolutional model suitable for processing time-series signals to process SAR images, which is more accurate than traditional methods.

[0052] Finally, given the multiple first feature maps output by the convolutional models at each time step, a quality assessment is performed on the SAR image corresponding to the target region based on these multiple first feature maps and using a quality assessment model (S306). Thus, the operator can determine the quality analysis result corresponding to the SAR image through the terminal device 10 and perform a quality assessment on the SAR image based on the quality analysis result.

[0053] As described in the background section, SAR image quality assessment currently typically utilizes Convolutional Neural Networks (CNNs) or Vit to process SAR images. While CNNs and Vit perform well in processing spatially distributed data (e.g., optical remote sensing images), they have limitations when processing SAR images, which are essentially time-series signals. For example, since pixel values ​​in optical remote sensing images directly reflect the spatial radiometric characteristics of ground features, CNNs can effectively extract features from neighboring pixels belonging to the same ground feature, or Vit's self-attention mechanism can effectively extract ground feature distribution features across the entire image.

[0054] However, since the spatial information of SAR images is essentially derived from time-series signals, directly using convolutional neural networks (CNNs) or Vit to process SAR images and evaluate their quality may result in inaccuracies.

[0055] In view of this, this application discloses a quality assessment model, which includes multiple temporal convolutional models. These temporal convolutional models are primarily used for modeling and analyzing time-dimensional signals or sequence data, rather than traditional spatial images. Furthermore, since the quality assessment in this application involves SAR images, and the spatial information of SAR images is essentially derived from time-series signals, compared to existing technologies that utilize convolutional neural networks and Vit models suitable for processing spatial information, this application employs multiple temporal convolutional models for feature extraction from SAR images, thereby maximizing the accuracy of feature extraction.

[0056] This solves the technical problem in existing technologies where, since the spatial information of SAR images is essentially derived from time-series signals, using models suitable for processing spatial analysis data to evaluate the quality of SAR images may result in inaccurate evaluations.

[0057] Optionally, the operation of determining multiple first feature maps corresponding to SAR images using each temporal convolutional model in the quality assessment model includes: inputting the SAR image into each temporal convolutional model respectively; and using the convolution kernels corresponding to each layer of the temporal convolutional network in the temporal convolutional model to perform dilated convolution on the SAR image in the column direction with different receptive fields, and generating first feature maps corresponding to each temporal convolutional model.

[0058] Specifically, refer to Figure 5A As shown, taking the first temporal convolutional model TCN-1 as an example, the SAR image is input into the first temporal convolutional model TCN-1, and the convolution kernels of each layer of the temporal convolutional network in the first temporal convolutional model TCN-1 are used to perform dilated convolution on the SAR image in the column direction with different receptive fields. This generates a first feature map corresponding to the temporal convolutional model TCN-1.

[0059] For example, the first temporal convolutional network (input layer) of the first temporal convolutional model TCN-1 has a convolution kernel size of 3. The convolution kernel in the first temporal convolutional network convolves each pixel block on the SAR image in the column direction.

[0060] The second-layer temporal convolutional network (hidden layer) of the first-time convolutional model TCN-1 has a kernel size of 3, a stride of 1, and a receptive field of 5. The convolution kernel in the second-layer temporal convolutional network convolves each pixel block on the SAR image in the column direction.

[0061] The third temporal convolutional network (hidden layer) of the first temporal convolutional model TCN-1 has a kernel size of 3, a stride of 1, and a receptive field of 7. The convolutional kernel in the second temporal convolutional network convolves each pixel block on the SAR image in the column direction.

[0062] Therefore, the receptive field of the third temporal convolutional network increases compared to the receptive field of the second temporal convolutional network, and the receptive field of the second temporal convolutional network increases compared to the receptive field of the first temporal convolutional network.

[0063] Similarly, the receptive field of the last layer of the temporal convolutional network in the first temporal convolutional model TCN-1 is increased compared to the receptive field of the temporal convolutional network preceding the last layer.

[0064] Thus, with different receptive fields in each layer of the temporal convolutional network and the receptive field constantly increasing, shallow temporal convolutional networks with small receptive fields can capture local temporal patterns, while deep temporal convolutional networks with large receptive fields can identify global periodic change trends. Therefore, compared with traditional convolutional neural networks, they can extract features more accurately and finally output the first feature map.

[0065] Similarly, other temporal convolutional models TCN-2 to TCN-U can output the corresponding first feature map.

[0066] Therefore, multiple temporal convolutional models TCN-1 to TCN-U can output multiple first feature maps corresponding to SAR images.

[0067] Optionally, the quality assessment model includes multiple transformer models corresponding to each temporal convolutional model, convolutional neural networks, MLP models, fully connected layers, and softmax classifiers corresponding to each transformer model. Based on multiple first feature maps, the model performs quality assessment on SAR images corresponding to the target region, including: inputting the feature maps corresponding to each temporal convolutional model into the corresponding transformer model and outputting second feature maps corresponding to each first feature map; inputting each second feature map into the convolutional neural network and outputting third feature maps corresponding to each second feature map; inputting multiple third feature maps into the MLP model to determine the feature vectors corresponding to each third feature map, concatenating the feature vectors to generate a fused feature vector; and inputting the fused feature vector into the fully connected layer and the softmax classifier, and performing quality assessment on the SAR image based on the output results.

[0068] Specifically, refer to Figure 4As shown, firstly, after using the outputs of the first feature maps corresponding to the SAR image from each of the temporal convolutional models TCN-1 to TCN-U, each first feature map is input into its corresponding transformer model, thereby using the transformer model to output a second feature map corresponding to each first feature map. Specifically, when the first feature map is input into the transformer model, it is flattened into a corresponding temporal vector sequence. Furthermore, when the transformer model uses a self-attention mechanism to process the feature vectors and generate a feature vector sequence, the feature vector sequence is reshaped into the second feature map.

[0069] Then, the quality assessment model inputs each second feature map into a convolutional neural network and outputs a third feature map corresponding to each second feature map. Further, the quality assessment model inputs multiple third feature maps into an MLP model and determines the feature vector corresponding to each third feature map. These feature vectors are then concatenated to generate a fused feature vector. Finally, the quality assessment model inputs the fused feature vector into a fully connected layer and a softmax classifier to output the probability corresponding to each quality category.

[0070] Optionally, the operation of quality assessment of SAR images based on the output results includes: determining the quality assessment result corresponding to the SAR image based on the probability of the SAR image in each quality category output by the softmax classifier, wherein the quality categories include high quality, general quality and low quality.

[0071] Specifically, the quality categories include high quality, average quality, and low quality. Given the probabilities of the SAR image in each quality category output by the softmax classifier, the highest probability value among those probabilities for each quality category is determined. The quality category corresponding to the highest probability value is then identified as the quality assessment result for the SAR image. For example, the probability value corresponding to high quality is 70%, the probability value corresponding to average quality is 20%, and the probability value corresponding to low quality is 10%.

[0072] Therefore, through the above operations, accurate quality analysis of SAR images can be performed.

[0073] Furthermore, before using the quality assessment model to determine the quality of the SAR image corresponding to the target area, SAR Satellite 30 needs to pre-train the quality assessment model. The specific training steps include:

[0074] First, SAR satellite 30 acquires SAR image samples corresponding to each sample region. Each SAR image sample is labeled with a corresponding quality category, including high quality, general quality, and low quality.

[0075] Then, the SAR satellite constructs a quality assessment model. This quality assessment model includes multiple temporal convolutional models, multiple transformer models, multiple convolutional neural networks, an MLP model, fully connected layers, and a softmax classifier.

[0076] Then, multiple labeled SAR image samples are input into the quality assessment model, and the quality assessment model is trained.

[0077] Thus, according to the first aspect of this embodiment, the technical effect of maximizing the accuracy of feature extraction from SAR images and thereby ensuring accurate quality assessment of SAR images is achieved.

[0078] In addition, refer to Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0079] Thus, according to this embodiment, the technical effect of maximizing the accuracy of feature extraction from SAR images and thereby ensuring accurate quality assessment of SAR images is achieved.

[0080] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0082] Example 2

[0083] Figure 6A SAR image quality assessment device 600 according to this embodiment is shown, which corresponds to the method described in Embodiment 1. Referring to Figure 6, the device 600 includes: a SAR image acquisition module 610, used to acquire SAR images corresponding to a target area using SAR satellites, wherein the horizontal axis of the SAR image represents the time-related azimuth, the vertical axis of the SAR image represents the slant range, and the pixel value of the SAR image represents the backscattering intensity corresponding to the azimuth and slant range; a first feature map determination module 620, used to input the SAR image into a pre-trained quality assessment model, and use various temporal convolutional models in the quality assessment model to determine multiple first feature maps corresponding to the SAR image, wherein the temporal convolutional model includes a multi-layer temporal convolutional network, and the receptive field of each layer of the temporal convolutional network is different; and a quality assessment module 630, used to perform quality assessment on the SAR image corresponding to the target area based on the multiple first feature maps and using the quality assessment model.

[0084] Optionally, the first feature map determination module 620 includes: a SAR image input module for inputting SAR images into each temporal convolution model; and a first feature map output module for using the convolution kernels corresponding to each layer of the temporal convolution network in the temporal convolution model to perform dilated convolution on the SAR image in the column direction with different receptive fields, and generating a first feature map corresponding to each temporal convolution model.

[0085] Optionally, the quality assessment model includes multiple transformer models corresponding to each temporal convolutional model, convolutional neural networks, MLP models, fully connected layers, and softmax classifiers corresponding to each transformer model. The quality assessment module 630 includes: a second feature map output module, used to input the feature maps corresponding to each temporal convolutional model into the corresponding transformer model and output second feature maps corresponding to each first feature map; a third feature map output module, used to input each second feature map into the convolutional neural network and output third feature maps corresponding to each second feature map; a fused feature vector generation module, used to input multiple third feature maps into the MLP model, determine the feature vectors corresponding to each third feature map, concatenate the feature vectors, and generate a fused feature vector; and a quality assessment submodule, used to input the fused feature vector into the fully connected layer and the softmax classifier, and perform quality assessment on the SAR image based on the output results.

[0086] Optionally, the quality assessment submodule includes: a quality assessment result output module, used to determine the quality assessment result corresponding to the SAR image based on the probability of the SAR image in each quality category output by the softmax classifier, wherein the quality categories include high quality, general quality and low quality.

[0087] Thus, according to this embodiment, the technical effect of maximizing the accuracy of feature extraction from SAR images and thereby ensuring accurate quality assessment of SAR images is achieved.

[0088] Example 3

[0089] Figure 7 A SAR image quality assessment apparatus 700 according to this embodiment is shown, which corresponds to the method described according to Embodiment 1. Reference Figure 7 As shown, the device 700 includes: a processor 710; and a memory 720 connected to the processor 710, used to provide the processor 710 with instructions to process the following steps: acquiring a SAR image corresponding to a target area using a SAR satellite, wherein the horizontal axis of the SAR image represents the time-related azimuth, the vertical axis of the SAR image represents the slant range, and the pixel value of the SAR image represents the backscattering intensity corresponding to the azimuth and slant range; inputting the SAR image into a pre-trained quality assessment model, and using various temporal convolutional models in the quality assessment model to determine multiple first feature maps corresponding to the SAR image, wherein the temporal convolutional model includes a multi-layer temporal convolutional network, and the receptive field of each layer of the temporal convolutional network is different; and performing quality assessment on the SAR image corresponding to the target area based on the multiple first feature maps and using the quality assessment model.

[0090] Optionally, the operation of determining multiple first feature maps corresponding to SAR images using each temporal convolutional model in the quality assessment model includes: inputting the SAR image into each temporal convolutional model respectively; and using the convolution kernels corresponding to each layer of the temporal convolutional network in the temporal convolutional model to perform dilated convolution on the SAR image in the column direction with different receptive fields, and generating first feature maps corresponding to each temporal convolutional model.

[0091] Optionally, the quality assessment model includes multiple transformer models corresponding to each temporal convolutional model, convolutional neural networks, MLP models, fully connected layers, and softmax classifiers corresponding to each transformer model. Based on multiple first feature maps, the model performs quality assessment on SAR images corresponding to the target region, including: inputting the feature maps corresponding to each temporal convolutional model into the corresponding transformer model and outputting second feature maps corresponding to each first feature map; inputting each second feature map into the convolutional neural network and outputting third feature maps corresponding to each second feature map; inputting multiple third feature maps into the MLP model to determine the feature vectors corresponding to each third feature map, concatenating the feature vectors to generate a fused feature vector; and inputting the fused feature vector into the fully connected layer and the softmax classifier, and performing quality assessment on the SAR image based on the output results.

[0092] Optionally, the operation of quality assessment of SAR images based on the output results includes: determining the quality assessment result corresponding to the SAR image based on the probability of the SAR image in each quality category output by the softmax classifier, wherein the quality categories include high quality, general quality and low quality.

[0093] Thus, according to this embodiment, the technical effect of maximizing the accuracy of feature extraction from SAR images and thereby ensuring accurate quality assessment of SAR images is achieved.

[0094] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0095] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0097] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0098] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0099] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0100] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method of quality assessment of a SAR image, characterized in that, The method comprises: collecting a SAR image corresponding to a target area by using a SAR satellite, wherein an abscissa of the SAR image represents a time sequence related azimuth direction, an ordinate of the SAR image represents slant range, and a pixel value of the SAR image represents a backscattering intensity corresponding to the azimuth direction and the slant range; inputting the SAR image into a pre-trained quality assessment model, and determining a plurality of first feature maps corresponding to the SAR image by using each time convolution model in the quality assessment model, wherein the time convolution model comprises a plurality of time convolution networks, and a receptive field of each time convolution network is different; and based on the plurality of first feature maps and by using the quality assessment model, performing quality assessment on the SAR image corresponding to the target area, wherein the operation of determining the plurality of first feature maps corresponding to the SAR image by using each time convolution model in the quality assessment model comprises: inputting the SAR image into each time convolution model respectively; and performing dilated convolution on the SAR image in a column direction of the SAR image by using a convolution kernel corresponding to each time convolution network in the time convolution model with different receptive fields, and generating a first feature map corresponding to each time convolution model, wherein the quality assessment model comprises a plurality of transformer models corresponding to each time convolution model respectively, a convolutional neural network corresponding to each transformer model respectively, an MLP, a fully connected layer, and a softmax classifier, and the operation of performing quality assessment on the SAR image corresponding to the target area based on the plurality of first feature maps comprises: inputting the first feature map corresponding to each time convolution model into the corresponding transformer model respectively, and outputting a second feature map corresponding to each first feature map; inputting each second feature map into the convolutional neural network respectively, and outputting a third feature map corresponding to each second feature map; inputting a plurality of third feature maps into the MLP, determining a feature vector corresponding to each third feature map, splicing each feature vector, and generating a fused feature vector; and inputting the fused feature vector into the fully connected layer and the softmax classifier, and performing quality assessment on the SAR image based on an output result.

2. The method of claim 1, wherein, The operation of performing quality assessment on the SAR image based on the output result comprises: determining a quality assessment result corresponding to the SAR image based on a probability of the SAR image in each quality category output by the softmax classifier, wherein the quality category comprises high quality, general quality, and low quality.

3. A storage medium, characterized by The storage medium comprises a stored program, wherein the program is executed by a processor when the program is running to perform the method of any one of claims 1 to 2.

4. A quality assessment apparatus of a SAR image, characterized by, The method comprises: The SAR image acquisition module is configured to acquire a SAR image corresponding to a target area by using a SAR satellite, wherein an abscissa of the SAR image represents a time sequence related azimuth, an ordinate of the SAR image represents a slant range, and a pixel value of the SAR image represents a backscattering intensity corresponding to the azimuth and the slant range; The first feature map determination module is configured to input the SAR image into a pre-trained quality assessment model, and determine a plurality of first feature maps corresponding to the SAR image by using each time convolution model in the quality assessment model, wherein the time convolution model comprises a plurality of time convolution networks, and a receptive field of each layer of the time convolution networks is different; and The quality assessment module is configured to perform quality assessment on the SAR image corresponding to the target area based on the plurality of first feature maps and by using a quality assessment model, wherein the first feature map determination module comprises: The SAR image input module is configured to input the SAR image into each time convolution model respectively; and The first feature map output module is configured to perform a dilated convolution on the SAR image in a column direction of the SAR image by using a convolution kernel corresponding to each layer of the time convolution networks in the time convolution model with different receptive fields, and generate a first feature map corresponding to each time convolution model, wherein the quality assessment model comprises a plurality of transformer models corresponding to the plurality of time convolution models respectively, a convolutional neural network corresponding to each transformer model respectively, an MLP, a fully connected layer, and a softmax classifier, and the quality assessment module comprises: The second feature map output module is configured to input the feature map corresponding to each time convolution model into the corresponding transformer model respectively, and output a second feature map corresponding to each first feature map; The third feature map output module is configured to input each second feature map into the convolutional neural network respectively, and output a third feature map corresponding to each second feature map; The fusion feature vector generation module is configured to input a plurality of third feature maps into the MLP, determine a feature vector corresponding to each third feature map, splice each feature vector, and generate a fusion feature vector; and The quality assessment submodule is configured to input the fusion feature vector into the fully connected layer and the softmax classifier, and perform quality assessment on the SAR image based on an output result.

5. The apparatus of claim 4, wherein, The quality assessment submodule comprises: The quality assessment result output module is configured to determine a quality assessment result corresponding to the SAR image based on a probability of the SAR image in each quality category output by the softmax classifier, wherein the quality categories comprise high quality, general quality, and low quality.

6. A quality assessment apparatus of a SAR image, characterized by, The quality assessment submodule comprises: A processor; and A memory connected with the processor, configured to provide the processor with instructions for processing the following processing steps: ​ acquiring a SAR image corresponding to a target area by using a SAR satellite, wherein an abscissa of the SAR image represents a time sequence related azimuth direction, an ordinate of the SAR image represents slant range, and a pixel value of the SAR image represents a backscattering intensity corresponding to the azimuth direction and the slant range; inputting the SAR image into a pre-trained quality assessment model, and determining a plurality of first feature maps corresponding to the SAR image by using each time convolution model in the quality assessment model, wherein the time convolution model comprises a plurality of layers of time convolution networks, and a receptive field of each layer of the time convolution networks is different; and performing quality assessment on the SAR image corresponding to the target area based on the plurality of first feature maps and by using the quality assessment model, wherein the operation of determining the plurality of first feature maps corresponding to the SAR image by using each time convolution model in the quality assessment model comprises: inputting the SAR image into each time convolution model respectively; and performing dilated convolution on the SAR image in a column direction of the SAR image by using a convolution kernel corresponding to each layer of the time convolution networks in the time convolution model with different receptive fields, and generating a first feature map corresponding to each time convolution model, wherein the quality assessment model comprises a plurality of transformer models corresponding to each time convolution model respectively, a convolutional neural network corresponding to each transformer model respectively, an MLP, a fully connected layer, and a softmax classifier, and the operation of performing quality assessment on the SAR image corresponding to the target area based on the plurality of first feature maps comprises: inputting the first feature map corresponding to each time convolution model into the corresponding transformer model respectively, and outputting a second feature map corresponding to each first feature map; inputting each second feature map into the convolutional neural network respectively, and outputting a third feature map corresponding to each second feature map; inputting a plurality of third feature maps into the MLP, determining a feature vector corresponding to each third feature map, splicing each feature vector, and generating a fused feature vector; and inputting the fused feature vector into the fully connected layer and the softmax classifier, and performing quality assessment on the SAR image based on an output result.

Images