A task-driven method for evaluating image quality of space targets

Abstract

The application discloses a space target image quality evaluation method based on task driving, which comprises the following steps: obtaining an image to be evaluated; using a trained quality evaluation model to evaluate the quality of the image to be evaluated, and obtaining a quality category representing the advantages and disadvantages of the quality of the image to be evaluated; the trained quality evaluation model is obtained by training an initial quality evaluation model in a manner of metric learning using a target data set with a quality label; the target data set is constructed according to the Euclidean distance between the target recognition result of the image category of a first data set by a trained target recognition model and the real target category of the first data set; the first data set is a homologous data set of a data set used for training the target recognition model; and the image with an evaluation result reaching a preset quality category is used as a training sample of the target recognition model or an initial target recognition model. The application can reduce the cost of manual image labeling and improve the matching degree with a downstream task.

Description

Technical Field

[0001] This invention belongs to the field of machine vision technology, specifically relating to a task-driven method for evaluating the quality of spatial target images. Background Technology

[0002] Image Quality Assessment (IQA) is a technique that uses computer vision to classify the quality of image data used in downstream tasks. With the increasing number of space targets, space target image data is experiencing explosive growth, making intelligent storage of space target images a bottleneck. Image quality is a crucial indicator of the overall performance of a space target system, and IQA plays a vital role in performance evaluation and comparative analysis of target recognition and detection algorithms. Current real-world space target images suffer from overexposure, excessive ghosting, stellar noise, and motion blur, resulting in image quality inferior to natural images and making them unsuitable for direct application to downstream task algorithms. Furthermore, in tasks such as target detection and recognition, simply having a clear image often fails to meet all the requirements. Expert-led quality assessment of space targets for downstream tasks is currently the most accurate known method for space target image quality assessment. However, due to the inherent drawbacks of manual assessment—low ​​efficiency and high cost—deep neural networks are used for learning and modeling to simulate human vision, enabling deep neural network models to replace manual subjective simulation of human vision in radar image quality assessment.

[0003] Existing computer vision-based image quality assessment methods still require manually labeled training image datasets, which is costly. Furthermore, while using trainable classifiers in image quality classification models can achieve good results for tasks with ample data, spatial object images are diverse, but the number of images in each category is limited. Therefore, classification modules trained with sufficient data often perform poorly. Summary of the Invention

[0004] To address the aforementioned problems in related technologies, this invention provides a task-driven spatial target image quality assessment method. The technical problem to be solved by this invention is achieved through the following technical solution:

[0005] This invention provides a task-driven spatial target image quality assessment method, comprising:

[0006] Obtain the image to be evaluated;

[0007] A well-trained quality assessment model is used to evaluate the quality of the image to be evaluated, thereby obtaining the quality category of the image; the quality category represents the degree of quality of the image.

[0008] The fully trained quality assessment model is obtained by training the initial quality assessment model using a target dataset with quality labels through metric learning. The target dataset is constructed based on the target recognition results of the trained target recognition model on the image categories of the first dataset and the Euclidean distance between the true target categories of the first dataset. The first dataset is a homologous dataset to the dataset used to train the target recognition model. Images whose evaluation results reach the preset quality category are used as training samples for the target recognition model or the initial target recognition model.

[0009] In some embodiments, the fully trained quality assessment model has multiple convex combined covariance matrices and multiple feature representations corresponding one-to-one with multiple quality categories; the step of using the fully trained quality assessment model to perform quality assessment on the image to be evaluated, and obtaining the quality category of the image to be evaluated, includes:

[0010] Extract deep features from the image to be evaluated;

[0011] Based on the deep features of the image to be evaluated, the multiple convex combination covariance matrices, and the multiple feature representations, the Mahalanobis distance between the deep features and each of the multiple quality categories is calculated to obtain multiple Mahalanobis distances that correspond one-to-one with the multiple quality categories.

[0012] The distance vector is determined based on the multiple Mahalanobis distances, and the vector is normalized to obtain the normalized vector.

[0013] The quality category corresponding to the maximum value in the normalized vector is taken as the quality category of the image to be evaluated.

[0014] In some embodiments, extracting deep features from the image to be evaluated includes:

[0015] Local features of the image to be evaluated are extracted using a convolutional layer to obtain a first feature map;

[0016] The first feature map is normalized and activated using a batch normalization layer to obtain the second feature map;

[0017] The second feature map is downsampled using a max pooling layer to obtain the third feature map;

[0018] The deep semantic features of the third feature map are extracted by using multiple residual modules connected in series to obtain the fourth feature map;

[0019] The fourth feature map is pooled using an average pooling layer to obtain the deep features.

[0020] In some embodiments, the step of calculating the Mahalanobis distance between the deep features and each of the multiple quality categories based on the deep features of the image to be evaluated, the multiple convex combination covariance matrices, and the multiple feature representations, to obtain multiple Mahalanobis distances corresponding one-to-one with the multiple quality categories, includes:

[0021] For each quality category, determine the feature difference between the deep features of the image to be evaluated and the feature representation corresponding to that quality category;

[0022] The feature difference is transposed to obtain the transposed result;

[0023] Based on the feature difference, the transpose result, and the convex combination covariance matrix corresponding to the quality category, the Mahalanobis distance between the image to be evaluated and the quality category is determined, resulting in multiple Mahalanobis distances corresponding one-to-one with the multiple quality categories.

[0024] In some embodiments, before performing quality evaluation on the image to be evaluated using a fully trained quality evaluation model to obtain the quality category of the image to be evaluated, the method further includes:

[0025] Obtain the second dataset, the first dataset which is a dataset from the same source as the second dataset, and the initial target recognition model;

[0026] The initial target recognition model is trained using the second dataset to obtain the fully trained target recognition model;

[0027] The fully trained recognition model is used to predict the category of the target in each image in the first dataset, and the target recognition result of each image is obtained.

[0028] Determine the Euclidean distance between the target recognition result of each image in the first dataset and the true target category of the image, and construct the target dataset based on the Euclidean distance; the target dataset includes images of multiple quality categories;

[0029] The initial quality assessment model is obtained, and the initial quality assessment model is iteratively trained using the target dataset, the metric loss function, and the cross-entropy loss function to obtain the fully trained quality assessment model.

[0030] In some embodiments, determining the Euclidean distance between the target recognition result of each image in the first dataset and the true target category of the image, and constructing the target dataset based on the Euclidean distance, includes:

[0031] Determine the Euclidean distance between the target recognition result of each image in the first dataset and the true target category of that image;

[0032] Based on the Euclidean distance and multiple preset thresholds, the images in the first dataset are divided into the multiple quality categories; the quality category to which each image belongs is the quality label of that image.

[0033] In some embodiments, the step of iteratively training the initial quality assessment model using the target dataset, the metric loss function, and the cross-entropy loss function to obtain the fully trained quality assessment model includes:

[0034] Deep features are extracted from each image in the target dataset, and feature representations of each quality category in the multiple quality categories are obtained based on the deep features, as well as the full image feature representation corresponding to the multiple quality categories.

[0035] Based on the deep features, feature representations, and full image feature representations of the images in the target dataset, the convex combination covariance matrix of each quality category is determined;

[0036] During each training session, multiple images are selected from the target dataset and input into the previously trained quality assessment model. The previously trained quality assessment model is based on the convex combination covariance matrix of each quality category to obtain the output of the model for the current training session.

[0037] Based on the model output of the current training iteration, the deep features of the multiple images, the feature representation, and the quality labels of the multiple images, the metric loss and cross-entropy loss for the current iteration are determined respectively.

[0038] The network parameters of the quality evaluation model obtained in the previous training are adjusted according to the current metric loss and cross-entropy loss to obtain the quality evaluation model obtained in the current training. This process is repeated until the fully trained quality evaluation model is obtained.

[0039] In some embodiments, obtaining the feature representations of each of the plurality of quality categories based on the deep features, and the full image feature representation commonly corresponding to the plurality of quality categories, includes:

[0040] Average pooling is performed on the deep features of the image for each of the multiple quality categories to obtain the feature representation of that quality category;

[0041] The feature representations of the multiple quality categories are averaged and pooled to obtain the full image feature representation.

[0042] In some embodiments, determining the convex combination covariance matrix of each quality category based on the deep features, feature representations, and full image feature representations of the images in the target dataset includes:

[0043] Based on the feature representation of each of the plurality of quality categories and the deep features of all images belonging to that quality category, determine the intra-class feature covariance matrix of that quality category;

[0044] Based on the deep features of all images in the target dataset and the full image feature representation, determine the full image feature covariance matrix that corresponds to the multiple quality categories.

[0045] Based on the intra-class feature covariance matrix of each quality category, the full image feature covariance matrix, and the preset weighting coefficients of each quality category, the convex combination covariance matrix of each quality category is determined.

[0046] In some embodiments, the expression for the current loss metric is:

[0047]

[0048] Among them, L metric To measure the loss, d k (f θ (x i ),μ k ) represents f θ (x i ) and μ k The Mahalanobis distance between them, f θ (x i ) represents the deep feature of the i-th image among m images, μ k Let n represent the feature representation of the k-th quality category, n represent the total number of quality categories, and m represent the total number of images used in the current training iteration.

[0049] The present invention has the following beneficial technical effects:

[0050] The present invention uses a quality evaluation model for upstream tasks to test images, and evaluates the image quality based on the test results, which can achieve the quality evaluation of images for specific recognition tasks, thereby reducing the cost of manual image annotation for training the target recognition model for downstream tasks; moreover, by classifying the image quality in a metric learning manner, the correlation between features is considered, which can improve the algorithm robustness under the condition of insufficient training data, increase the accuracy of image quality evaluation, and thus improve the matching degree with downstream tasks.

[0051] The following will further elaborate on the present invention in conjunction with the accompanying drawings and embodiments. Brief Description of the Drawings

[0052] Figure 1 It is a flowchart of a method for task-driven spatial target image quality evaluation provided by an embodiment of the present invention;

[0053] Figure 2 It is a general framework diagram of an exemplary task-driven spatial target image quality evaluation method provided by an embodiment of the present invention;

[0054] Figure 3 It is an exemplary spatial target image in the BUAA-SID-Share1.0 dataset provided by an embodiment of the present invention;

[0055] Figure 4 It is provided by an embodiment of the present invention示例性 Figure 3 spatial target images formed by adding noise on the basis of;

[0056] Figure 5A It is a schematic diagram of an exemplary image belonging to the "excellent" category provided by an embodiment of the present invention;

[0057] Figure 5B It is a schematic diagram of an exemplary image belonging to the "good" category provided by an embodiment of the present invention;

[0058] Figure 5C It is a schematic diagram of an exemplary image belonging to the "medium" category provided by an embodiment of the present invention;

[0059] Figure 5D It is a schematic diagram of an exemplary image belonging to the "poor" category provided by an embodiment of the present invention. Detailed Embodiments

[0060] The following further describes the present invention in detail with specific embodiments, but the implementation manners of the present invention are not limited thereto.

[0061] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0062] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0063] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0064] Figure 1 This is a flowchart of a task-driven spatial target image quality assessment method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes the following steps:

[0065] S101. Obtain the image to be evaluated.

[0066] Here, the image to be evaluated can be an image of any target category, for example, it can be an image of a space target (e.g., a satellite).

[0067] S102. A fully trained quality assessment model is used to evaluate the quality of the image to be evaluated, and the quality category of the image to be evaluated is obtained. The quality category represents the degree of image quality. The fully trained quality assessment model is obtained by training the initial quality assessment model using a target dataset with quality labels through metric learning. The target dataset is constructed based on the target recognition results of the image categories in the first dataset by the fully trained target recognition model, and the Euclidean distance between the true target categories in the first dataset. The first dataset is a homologous dataset to the dataset used to train the target recognition model. Images that achieve the preset quality category are used as training samples for the target recognition model or the initial target recognition model.

[0068] Here, the fully trained target recognition model can be a model for recognizing any target, for example, a model for recognizing a specific space target (e.g., a satellite). The model structure of the fully trained target recognition model can be any existing target recognition model structure.

[0069] For example, a well-trained quality assessment model can classify the quality of each image into four categories: "excellent", "good", "medium", and "poor".

[0070] Here, the target categories of the images in the first dataset and the dataset used to train the target recognition model belong to the same category. For example, the first dataset contains images of some satellites, while the dataset used to train the target recognition model contains images of other satellites. Moreover, the types of satellites corresponding to the two datasets can be different.

[0071] Specifically, a fully trained quality assessment model has multiple convex combination covariance matrices Q that correspond one-to-one with multiple quality categories. k and multiple feature representations μ k Let k = 1, 2, ..., n, where n is the total number of quality categories. Based on this, the above S102 can be implemented through the following steps:

[0072] S1021. Extract deep features from the image to be evaluated.

[0073] Specifically, a convolutional layer with a kernel size of 7x7, a stride of 2, and padding of 3 is used to perform convolution operations on the image to be evaluated, processing different local regions of the image to extract local features. Through convolution, the image is converted from 3 channels (RGB) to 64 feature channels, and the size of the feature map is reduced to extract local features. After the convolution operation, the first feature map of the image is obtained. Then, a batch normalization layer is used to normalize each channel of the first feature map to have zero mean and unit variance, followed by processing with the ReLU activation function to obtain the second feature map, whose size remains unchanged. Finally, a max pooling layer is used to downsample the second feature map to obtain the third feature map, where the max pooling layer segments the feature map through the max pooling operation. The first feature map is divided into non-overlapping rectangular regions, and the maximum value in each region is selected as the output, reducing the feature map size by half. Then, the third feature map is passed through eight residual modules in sequence to extract deep semantic features, thus obtaining the fourth feature map. The first two residual modules consist of two 3x3 convolutional layers with a stride of 1, each followed by a batch normalization layer and a ReLU activation function. The input and output feature maps have the same size. The subsequent six residual modules also consist of two convolutional layers, each followed by a batch normalization layer and a ReLU activation function, but the stride of the second convolutional layer in the subsequent six residual modules is 2, which halves the size of the feature map. Then, an average pooling layer is used to pool the fourth feature map to reduce its spatial dimension to 1×1, thus obtaining deep features.

[0074] S1022. Based on the deep features of the image to be evaluated, multiple convex combination covariance matrices, and multiple feature representations, calculate the Mahalanobis distance between the deep features and each of the multiple quality categories to obtain multiple Mahalanobis distances that correspond one-to-one with the multiple quality categories.

[0075] Specifically, for each quality category, the feature difference between the deep features of the image to be evaluated and the feature representation corresponding to that quality category is determined; the feature difference is transposed to obtain the transposed result; based on the feature difference, the transposed result and the convex combination covariance matrix corresponding to that quality category, the Mahalanobis distance between the image to be evaluated and that quality category is determined, resulting in multiple Mahalanobis distances corresponding one-to-one with multiple quality categories.

[0076] For example, the Mahalanobis distance between the deep features of the image to be evaluated and each quality category can be calculated using the following formula:

[0077]

[0078] Where, d k (fθ (x),μ k f represents the deep features of the image to be evaluated. θ The Mahalanobis distance between (x) and the k-th quality category, μ k Q represents the feature representation corresponding to the k-th quality category. k Let T be the covariance matrix of the convex combination corresponding to the k-th quality category, and T be the transpose symbol.

[0079] S1023. Determine the distance vector based on multiple Mahalanobis distances, and normalize the vector to obtain the normalized vector.

[0080] For example, when n is 4, four Mahalanobis distances d1, d2, d3, and d4 can be calculated using S1022. Therefore, d1, d2, d3, and d4 can be concatenated to obtain the distance vector d = (d1, d2, d3, d4). Then, a SoftMax operation is performed on the distance vector to obtain the normalized vector. The expression for performing the SoftMax operation on the distance vector is as follows:

[0081] Where i = 1, 2, 3, 4.

[0082] S1024. The quality category corresponding to the maximum value in the normalized vector is taken as the quality category of the image to be evaluated.

[0083] In some embodiments, prior to S102, the method further includes:

[0084] S201. Obtain the second dataset, the first dataset which is a dataset from the same source as the second dataset, and the initial target recognition model.

[0085] Here, the initial target recognition model is the target recognition model obtained during initialization.

[0086] S202. The initial target recognition model is trained using the second dataset to obtain a fully trained target recognition model.

[0087] Specifically, the training method is the same as that of existing target recognition models, so it will not be elaborated upon here.

[0088] S203. Use a fully trained recognition model to predict the category of the target in each image in the first dataset, and obtain the target recognition result for each image.

[0089] Here, each image in the first dataset is input into the fully trained recognition model, which can predict the category of the target in the image, thus obtaining the target recognition category of each image.

[0090] S204. Determine the Euclidean distance between the target recognition result of each image in the first dataset and the true target category of the image, and construct the target dataset based on the Euclidean distance; the target dataset includes images of multiple quality categories.

[0091] For example, for each image in the first dataset, the Euclidean distance between the target recognition result and the true target category of the image can be calculated. This yields the Euclidean distances for all images in the first dataset. Then, multiple preset thresholds are used to classify the images in the first dataset into four quality categories. For instance, for each target category, based on multiple thresholds, the smallest 25% of the Euclidean distances are designated as "Excellent," the next 25% as "Good," and so on, thus dividing the images in the first dataset into four categories: "Excellent," "Good," "Average," and "Poor." This results in a target dataset with quality labels. These multiple thresholds can be manually modified according to different image tasks.

[0092] In some embodiments, after obtaining the target dataset with quality labels, the target dataset can be divided into a target training set and a target test set, and there are no identical images in the target training set and the target test set. Finally, the target training set with quality labels is used to train the quality evaluation model.

[0093] S205. Obtain the initial quality assessment model, and use the target dataset, metric loss function, and cross-entropy loss function to iteratively train the initial quality assessment model to obtain a fully trained quality assessment model.

[0094] Specifically, S205 can be implemented through S2051 to S2055:

[0095] S2051. Extract the deep features of each image in the target dataset, and obtain the feature representation of each quality category in multiple quality categories based on the deep features, as well as the full image feature representation corresponding to multiple quality categories.

[0096] Specifically, using the same principle as S1021 above, deep features of each image in the target dataset are extracted. Then, average pooling is performed on the deep features of all images for each quality category in the target dataset to obtain the feature representation μ of that quality category. k ; after obtaining the characteristic representation μ of each quality category k Then, average pooling is performed on the feature representations of all quality categories to obtain the full image feature representation μ.

[0097] S2052. Based on the deep features, feature representations, and full image feature representations of the images in the target dataset, determine the convex combination covariance matrix of each quality category.

[0098] Specifically, based on the characteristic representation μ of each of the multiple quality categories... k And the deep features of all images belonging to this quality category, to determine the intra-class feature covariance matrix ∑ of this quality category. k Based on the deep features of all images in the target dataset and the full image feature representation μ, determine the full image feature covariance matrix ∑ corresponding to multiple quality categories; based on the intra-class feature covariance matrix ∑ of each quality category... k The full image feature covariance matrix ∑ and the preset weighting coefficients λ for each quality category. k Determine the covariance matrix Q of the convex combination of each quality category. k .

[0099] For example, the intra-class feature covariance matrix ∑ k The expression is as follows:

[0100]

[0101] Among them, S k f represents the number of images belonging to the k-th quality category. θ (x i ) represents S k The deep features of the i-th image, μ k This represents the feature representation of the k-th quality category.

[0102] For example, the expression for the covariance matrix ∑ of all image features is as follows:

[0103]

[0104] Where S represents the number of images in the target dataset, f θ (x c ) represents the deep features of the c-th image in S, and μ represents the deep feature representation of the entire image.

[0105] For example, the covariance matrix Q of the convex combination k The expression is as follows:

[0106] Q k =λ k ∑ k +(1-λ k )∑;

[0107] Where, λ k This represents the preset weighting coefficient for the k-th quality category.

[0108] S2053. During each training session, multiple images are selected from the target dataset and input into the quality assessment model obtained from the previous training. The quality assessment model obtained from the previous training is based on the convex combination covariance matrix of each quality category to obtain the output of the model for the current training session.

[0109] Here, the output of the model in the current training can be the quality category of each of these multiple images predicted by the quality assessment model obtained in the previous training.

[0110] S2054. Based on the model output of the current training iteration, the deep features of multiple images, the feature representations, and the quality labels of multiple images, determine the metric loss and cross-entropy loss for the current iteration, respectively.

[0111] For example, the expression for the loss metric in the current iteration is:

[0112]

[0113] Among them, L metric To measure the loss, d k (f θ (x i ),μ k ) represents f θ (x i ) and μ k The Mahalanobis distance between them, f θ (x i ) represents the deep feature of the i-th image among m images, μ k Let represent the feature representation of the k-th quality category, n represent the total number of quality categories, and m represent the total number of images used in the current training iteration.

[0114] For example, the expression for the current cross-entropy loss is:

[0115]

[0116] Among them, L CrossEntropy For cross-entropy loss, y i Let x represent the quality label of the i-th image out of m images. i This represents the predicted quality category of the i-th image.

[0117] S2055. Adjust the network parameters of the quality evaluation model obtained from the previous training based on the current metric loss and cross-entropy loss to obtain the quality evaluation model obtained from the current training. Iterate in this way until a fully trained quality evaluation model is obtained.

[0118] Here, after obtaining the current metric loss and the current cross-entropy loss, the metric loss and cross-entropy loss can be weighted using weight coefficients, and the weighted result can be used for backpropagation to adjust the network parameters of the quality assessment model obtained in the previous training, thereby obtaining the quality assessment model obtained in the current training. Then, this training principle is used to train the quality assessment model obtained in the current training, and so on iteratively until the number of training times reaches the preset number, or the model converges, to obtain the trained quality assessment model, that is, the fully trained quality assessment model.

[0119] Here, the weighting coefficients can be determined based on different task characteristics, and this application does not impose any restrictions on this.

[0120] This invention provides a dataset for image quality classification through downstream tasks, obtains different image quality feature representations through a deep feature extraction module, and finally performs quality classification based on the metric between deep image features and each image quality feature representation. This improves the efficiency of data filtering by combining task characteristics and provides data quality assurance for downstream tasks.

[0121] For example, Figure 2 This is a general framework diagram of a task-driven spatial target image quality assessment method provided for an embodiment of the present invention. (See diagram below.) Figure 2 As shown, first construct an image library containing images with a quality category of "excellent" ( Figure 2 The image library with "excellent" quality, and the image library with "poor" quality. Figure 2The dataset is divided into two parts: a "poor" quality image library and a "low" quality image library. The deep features of each image in the dataset are extracted using an encoder. Based on the deep features of the images in the dataset, the feature representations of "high quality" images corresponding to the "good" quality image library and "low quality" images corresponding to the "poor" quality image library are determined. The full image feature representation is then determined based on the feature representations of "high quality" and "low quality" images. Based on the deep features, feature representations of "high quality" images, feature representations of "low quality" images, and full image feature representations of the images in the dataset, the convex combination covariance matrix corresponding to the quality category "good" and the convex combination covariance matrix corresponding to the quality category "poor" are determined. This dataset, along with the feature representations of "high-quality" and "low-quality" images, the convex combined covariance matrix corresponding to the "excellent" quality category, and the convex combined covariance matrix corresponding to the "poor" quality category, are used to train a quality assessment model, resulting in a well-trained model. In practical applications, when image quality assessment of a target image is required, the target image can be input into the trained model. The model first uses an encoder to extract deep features from the target image, obtaining its feature distribution. Then, based on the feature distribution, the feature representations of "high-quality" and "low-quality" images, and the convex combined covariance matrix corresponding to the "excellent" and "poor" quality categories, the model classifies the target image's quality category in the metric space. Finally, the classification result of the target image's quality category is output, which is either "poor" or "excellent" quality.

[0122] This invention employs a quality assessment model for upstream tasks to test images and evaluates image quality based on the test results. This enables quality assessment of images for specific recognition tasks, thereby reducing the cost of manual image annotation for training target recognition models for downstream tasks. Furthermore, this invention uses metric learning to classify image quality and determines the global and local covariance between features using Mahalanobis distance. By fully considering the correlation between features through global and local covariance, the robustness of the algorithm can be improved even with insufficient training data, increasing the accuracy of image quality assessment and thus enhancing its compatibility with downstream tasks.

[0123] The following simulation experiment verifies and illustrates the effectiveness of the task-driven spatial target image quality evaluation method provided by this invention.

[0124] Test conditions:

[0125] This embodiment uses spatial target recognition as an example to simulate and test the downstream task of this invention. The BUAA-SID-Share1.0 dataset is used for testing. This dataset contains 20 different types of spatial targets, with each type containing 197 training images and 33 test images. To simulate real spatial target images, this example adds 200 training images and 30 test images to each class by increasing noise to varying degrees. Target recognition is performed on the images in this dataset, and four image quality categories ("Excellent," "Good," "Medium," and "Poor") are constructed based on the soft classification results. The training set is then used to train the image quality evaluation algorithm model.

[0126] Please see Figure 3 , Figure 4 , Figures 5A to 5D . Figure 3 This is a schematic diagram of spatial targets in the BUAA-SID-Share1.0 dataset provided in this embodiment of the invention; Figure 4 Is Figure 3 A schematic diagram of a spatial target with added noise; Figures 5A to 5D This is a schematic diagram illustrating image quality assessment classification based on the soft classification results of a spatial target recognition algorithm model. Figure 5A , 5B 5C and 5D represent the four categories of "Excellent", "Good", "Average", and "Poor", respectively. According to... Figures 5A to 5D It can be seen that noise can cause significant differences in the results of spatial target recognition, and the impact on the results increases with the increase of noise.

[0127] Experiment content:

[0128] This experimental example uses the BUAA-SID-Share1.0 training set to train the spatial target recognition algorithm model (the target recognition model mentioned above) until convergence. Target recognition is then performed on all images in the dataset. Finally, based on the soft classification results, the dataset is divided into four image quality categories: "Excellent," "Good," "Medium," and "Poor," resulting in an image quality evaluation dataset. The image quality evaluation model is then trained using the training set from this dataset until convergence, achieving image quality classification based on the spatial target recognition task. The spatial target recognition algorithm model is then trained using the "Excellent" and "Good" image training sets until convergence, and finally tested using the "Excellent" and "Good" image test sets.

[0129] Experimental results and analysis:

[0130] Please refer to Tables 1 and 2. Table 1 shows the image quality classification results based on the spatial target recognition task; Table 2 shows the spatial target recognition results trained on "excellent" and "good" images provided in this embodiment of the invention.

[0131]

[0132] Table 1

[0133]

[0134] Table 2

[0135] Table 1 shows that unclassified spatial target images exhibit significant quality variations for spatial target recognition tasks. This leads to low accuracy in spatial target recognition models trained using the entire image. Furthermore, the models fail to recognize low-quality spatial targets and also compromise high-quality images. Table 2 shows that training the spatial target recognition model with high-quality images obtained from classification using a quality classification algorithm for spatial target recognition tasks can improve the model's accuracy, robustness, and convergence speed.

[0136] This embodiment obtains an image quality classification dataset for the spatial object recognition task by classifying the dataset according to the spatial object recognition results. The image quality classification dataset is then used to train an image quality assessment model. Finally, high-quality images from the classification results are used to train the spatial object recognition system, thus improving the accuracy of spatial object recognition.

[0137] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A task-driven spatial target image quality assessment method, characterized in that, include: Obtain the image to be evaluated; A well-trained quality assessment model is used to evaluate the quality of the image to be evaluated, and the quality category of the image to be evaluated is obtained. The quality category characterizes the degree of image quality. The fully trained quality assessment model is obtained by training an initial quality assessment model using a target dataset with quality labels through metric learning. The target dataset is constructed based on the target recognition results of the trained target recognition model on the image categories of the first dataset and the Euclidean distance between the true target categories of the first dataset. The first dataset is a homologous dataset to the dataset used to train the target recognition model. Images that achieve a preset quality category in the evaluation are used as training samples for the target recognition model or the initial target recognition model. The target dataset is constructed as follows: the fully trained target recognition model is used to predict the category of the target in each image in the first dataset to obtain the target recognition result of each image; the Euclidean distance between the target recognition result of each image in the first dataset and the true target category of the image is determined; based on the Euclidean distance and multiple preset thresholds, the images in the first dataset are divided into multiple quality categories; the quality category to which each image belongs is the quality label of the image.

2. The task-driven spatial target image quality assessment method according to claim 1, characterized in that, The fully trained quality evaluation model has multiple convex combination covariance matrices and multiple feature representations that correspond one-to-one with multiple quality categories; The quality assessment of the image to be evaluated is performed using a fully trained quality assessment model to obtain the quality category of the image to be evaluated, including: Extract deep features from the image to be evaluated; Based on the deep features of the image to be evaluated, the multiple convex combination covariance matrices, and the multiple feature representations, the Mahalanobis distance between the deep features and each of the multiple quality categories is calculated to obtain multiple Mahalanobis distances that correspond one-to-one with the multiple quality categories. The distance vector is determined based on the multiple Mahalanobis distances, and the vector is normalized to obtain the normalized vector. The quality category corresponding to the maximum value in the normalized vector is taken as the quality category of the image to be evaluated.

3. The task-driven spatial target image quality assessment method according to claim 2, characterized in that, The extraction of deep features from the image to be evaluated includes: Local features of the image to be evaluated are extracted using a convolutional layer to obtain a first feature map; The first feature map is normalized and activated using a batch normalization layer to obtain the second feature map; The second feature map is downsampled using a max pooling layer to obtain the third feature map; The deep semantic features of the third feature map are extracted by using multiple residual modules connected in series to obtain the fourth feature map; The fourth feature map is pooled using an average pooling layer to obtain the deep features.

4. The task-driven spatial target image quality assessment method according to claim 2, characterized in that, The step involves calculating the Mahalanobis distance between the deep features of the image to be evaluated and each of the multiple quality categories based on the deep features, the multiple convex combination covariance matrices, and the multiple feature representations, to obtain multiple Mahalanobis distances corresponding one-to-one with the multiple quality categories, including: For each quality category, determine the feature difference between the deep features of the image to be evaluated and the feature representation corresponding to that quality category; The feature difference is transposed to obtain the transposed result; Based on the feature difference, the transpose result, and the convex combination covariance matrix corresponding to the quality category, the Mahalanobis distance between the image to be evaluated and the quality category is determined, resulting in multiple Mahalanobis distances corresponding one-to-one with the multiple quality categories.

5. The task-driven spatial target image quality assessment method according to claim 1, characterized in that, Before using a fully trained quality assessment model to assess the quality of the image to be evaluated and obtain the quality category of the image to be evaluated, the method further includes: Obtain the second dataset, the first dataset which is a dataset from the same source as the second dataset, and the initial target recognition model; The initial target recognition model is trained using the second dataset to obtain the fully trained target recognition model; The initial quality assessment model is obtained, and the initial quality assessment model is iteratively trained using the target dataset, the metric loss function, and the cross-entropy loss function to obtain the fully trained quality assessment model.

6. The task-driven spatial target image quality assessment method according to claim 5, characterized in that, The step of iteratively training the initial quality assessment model using the target dataset, metric loss function, and cross-entropy loss function to obtain the fully trained quality assessment model includes: Deep features are extracted from each image in the target dataset, and feature representations of each quality category in the multiple quality categories are obtained based on the deep features, as well as the full image feature representation corresponding to the multiple quality categories. Based on the deep features, feature representations, and full image feature representations of the images in the target dataset, the convex combination covariance matrix of each quality category is determined; During each training session, multiple images are selected from the target dataset and input into the previously trained quality assessment model. The previously trained quality assessment model is based on the convex combination covariance matrix of each quality category to obtain the output of the model for the current training session. Based on the model output of the current training iteration, the deep features of the multiple images, the feature representation, and the quality labels of the multiple images, the metric loss and cross-entropy loss for the current iteration are determined respectively. The network parameters of the quality evaluation model obtained in the previous training are adjusted according to the current metric loss and cross-entropy loss to obtain the quality evaluation model obtained in the current training. This process is repeated until the fully trained quality evaluation model is obtained.

7. The task-driven spatial target image quality assessment method according to claim 6, characterized in that, The step of obtaining the feature representation of each quality category among the multiple quality categories based on the deep features, and the full image feature representation corresponding to the multiple quality categories, includes: Average pooling is performed on the deep features of the image for each of the multiple quality categories to obtain the feature representation of that quality category; The feature representations of the multiple quality categories are averaged and pooled to obtain the full image feature representation.

8. The task-driven spatial target image quality assessment method according to claim 6, characterized in that, The determination of the convex combination covariance matrix of each quality category based on the deep features, feature representations, and full image feature representations of the images in the target dataset includes: Based on the feature representation of each of the plurality of quality categories and the deep features of all images belonging to that quality category, determine the intra-class feature covariance matrix of that quality category; Based on the deep features of all images in the target dataset and the full image feature representation, determine the full image feature covariance matrix that corresponds to the multiple quality categories. Based on the intra-class feature covariance matrix of each quality category, the full image feature covariance matrix, and the preset weighting coefficients of each quality category, the convex combination covariance matrix of each quality category is determined.

9. The task-driven spatial target image quality assessment method according to claim 6, characterized in that, The expression for the current loss metric is: ； in, To measure the loss, express and Mahalanobis distance between them express The first image Deep features of an image Indicates the first The characteristic representation of each quality category This represents the total number of quality categories. This represents the total number of images used in the current training iteration.

Images