Artificial generated image sample quality evaluation method
By extracting the feature map of image samples using a pre-trained convolutional neural network model, calculating its scores and weights, generating a heat map, and finally evaluating the quality of the generated samples through the consistency quality evaluation formula, solving the problem of obtaining high-quality image samples and lacking quantitative evaluation methods, and improving the efficiency and accuracy of deep learning model training.
Patent Information
- Application Number
- CN202411687652.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-03-25
AI Technical Summary
In some scenarios, such as military target detection and industry-specific defect detection, it is difficult to obtain high-quality image samples for deep learning model training, and the prior art lacks a quantitative evaluation method for the quality of generated image samples, resulting in inefficient model training.
A method of artificially generated image sample quality evaluation is adopted. By selecting a pre-trained convolutional neural network classification model, the real samples and the feature map of the generated samples are extracted, and its scores and weights for the target category are calculated to generate heat maps. Finally, the quality of the generated samples is evaluated through the consistency quality evaluation formula.
This method can conduct instant quality inspection when generating single image samples, solve the problem of lack of quantitative indicator evaluation of simulated samples, and improve the efficiency and accuracy of model training.
Smart Images
Figure QLYQS_7
Abstract
Description
Technical Field
[0001] The invention belongs to the field of artificial intelligence image processing and relates to a method for evaluating the quality of artificially generated picture samples. Background Art
[0002] The deep learning model of object detection based on convolutional neural network usually requires a large number of well-labeled images for training. However, in some scenarios, such as non-cooperative targets such as military target detection, or defect detection in specific industries, it is difficult to obtain high-quality images that can be used for neural network model training. Therefore, some generative methods are usually used to generate some image samples. However, whether these generated samples are suitable for the training of deep learning models requires the evaluation of the quality of the generated image samples.
[0003] For the target detection task in image processing, the generated image samples are not directly sampled from the real physical world, but are artificially generated data through computer programs. The purpose is to solve the problem of insufficient training samples for deep learning models. Based on this, the generated image samples should not have side effects on model training. The model obtained by using the generated samples in simulation training should have an accuracy in the test set that is not lower than that of the real samples in simulation training.
[0004] Considering that the number of generated samples is often relatively large, it is very time-consuming and labor-intensive to evaluate the model accuracy after the samples are generated. Therefore, it is necessary to evaluate the quality of the generated image samples. Summary of the invention
[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method for evaluating the quality of artificially generated image samples.
[0006] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention to solve the technical problem is: a method for evaluating the quality of artificially generated image samples, comprising the following steps: S1, model preparation: select a pre-trained convolutional neural network classification model as the test network model for generating image sample quality scores; S2, feature map extraction of real sample images and generated sample images S21, input the real sample image into the test network model, and extract the feature map in the last convolution layer of the test network model, denoted as M ,in M is a three-dimensional tensor, the dimension is usually height h ,width w , number of channels c , each real sample feature map M CThe residual convolutional layer of the test network model is passed as input to calculate the k Score S C ,in S C is a vector containing all the categories k The score is calculated by the formula Calculate the real sample feature map M C For target category k Weight , where the increment Used to characterize the changes in the model feature map, constant Used to characterize numerical stability; S22, the generated sample image is input into the test network model, and the feature map extracted in the last convolutional layer of the test network model is recorded as M′ ,in M′ is a three-dimensional tensor, the dimension is usually height h ,width w , number of channels c , generate each sample feature map M C ′ The residual convolutional layer of the test network model is passed as input to calculate the k Score S C ′ ,in S C ′ is a vector containing all the categories k The score is calculated by the formula Calculate and generate sample feature map M C ′ For target category k Weight , where the increment Used to characterize the changes in the model feature map, constant Used to characterize numerical stability; S3, generate heat map S31, real sample feature map obtained for the real sample image M C Weighted processing: each real sample feature map M C The corresponding weight Multiply them to get the weighted feature map of the real sample, sum all the weighted feature maps, and then normalize the summation result and record it as the heat map h; S32, generating a sample feature map obtained by generating a sample image M C ′ Perform weighted processing: generate each sample feature map M C ′ The corresponding weight Multiply them together to get the generated sample weighted feature map, sum all the weighted feature maps, and then normalize the summation result and record it as the heat map h ′ ; S4, consistency quality evaluation The heat maps h and h obtained in steps S31 and S32 are calculated by the following formula ′ Calculate the consistency between the two: , where m and n are the width and height of the heat map respectively. Since normalization has been performed, they can be directly compared. The mean is obtained. When the mean is ≤ 0.2, it is considered that the generated sample is consistent with the real sample image; otherwise, the generated sample is considered to have poor quality.
[0007] Furthermore, the weights in steps S22 and S23 are calculated by passing the feature map to the test network model and calculating the category k This is achieved by the gradient of .
[0008] Furthermore, the values after normalization in steps S32 and S33 are 0 to 1.
[0009] The beneficial effects of the present invention are as follows: the evaluation method of the present invention can quantitatively evaluate the quality of artificially generated image samples, and can promptly check when a single image sample is generated, thereby solving the problem of lack of quantitative index evaluation of simulated samples. DETAILED DESCRIPTION
[0010] The present invention is further described below with reference to the embodiments.
[0011] The present invention discloses a method for evaluating the quality of artificially generated image samples, which comprises the following steps.
[0012] S1, Model preparation.
[0013] Select a convolutional neural network (CNN) classification model that has been pre-trained on related tasks as the test network model for generating image sample quality scores.
[0014] S2, feature map extraction of real sample images and generated sample images.
[0015] S21, input the real sample image into the test network model, and extract the feature map in the last convolution layer of the test network model, denoted as M ,in M is a three-dimensional tensor, the dimension is usually height h ,width w , number of channels c , for each feature map M C , pass it as input to the rest of the test network model, and calculate the k Score S C , here S C is a vector containing all the categories k For each category k , calculate the feature map M C For Category k Weight .
[0016] This is done by passing the feature map M C Pass it to the test network model and calculate the category k The weights are It can be calculated by the following formula: , in is a small increment used to change the model feature map. is a small constant used for numerical stability.
[0017] S22, the generated sample image is input into the test network model, and the feature map extracted in the last convolutional layer of the test network model is recorded as M′ ,in M′ is a three-dimensional tensor, the dimension is usually height h ,width w , number of channels c , for each feature map M C ′ , pass it as input to the rest of the test network model, and calculate the score for each category S C ′ , here S C ′ is a vector containing all the categories k For each category k , calculate the feature mapM C ′ For Category k Weight .
[0018] This is done by passing the feature map M C ′ Pass it to the test network model and calculate the category k The weights are It can be calculated by the following formula: ,
[0019] in is a small increment used to change the model feature map. is a small constant used for numerical stability.
[0020] S3, generate a heat map.
[0021] S31, real sample feature map obtained for the real sample image M C Weighted processing: each real sample feature map M C The corresponding weight Multiply them to get the weighted feature map of the real sample, sum all the weighted feature maps, and then normalize the sum result to make its value between 0 and 1, which is recorded as the heat map h.
[0022] S32, generating a sample feature map obtained by generating a sample image M C ′ Perform weighted processing: generate each sample feature map M C ′ The corresponding weight Multiply them together to get the generated sample weighted feature map, sum all the weighted feature maps, and then normalize the summed result to make its value between 0 and 1, which is recorded as the heat map h ′ .
[0023] S4, consistency quality evaluation.
[0024] The heat maps h and h obtained in steps S31 and S32 ′ , calculate the consistency between the two: ,
[0025] m and n are the width and height of the heat map, respectively. Since normalization has been performed, they can be compared directly.
[0026] right The mean is calculated. When the mean is ≤ 0.2, the generated sample is considered to be consistent with the real sample. Otherwise, the generated sample is considered to be of poor quality.
[0027] The above embodiments are only illustrative of the principles and effects of the present invention, as well as some embodiments of its application. For those skilled in the art, several modifications and improvements may be made without departing from the creative concept of the present invention, and all of these belong to the protection scope of the present invention.
Claims
1. A method for evaluating the quality of artificially generated image samples, characterized in that: The following steps are included S1, select a pre-trained convolutional neural network classification model as a test network model for generating image sample quality scores; S2, feature map extraction of real sample images and generated sample images S21, input the real sample image into the test network model, and extract a height of h , width is w , the number of channels is c The three-dimensional tensor of M , get the feature map of each real sample image M C , and then passed as input to the remaining convolutional layer of the test network model to calculate for each target category k Score S C , and by the formula Calculate the real sample feature map M C For target category k The weight of S22, the generated sample image is input into the test network model, and a height of h , width is w , the number of channels is c The three-dimensional tensor of M′ , get the feature map of each generated sample image M C ′ , passed as input to the residual convolutional layer of the test network model, and calculated for each target category k Score S C ′ , through the formula Calculate and generate sample feature map M C ′ For target category k The weight of Used to characterize the changes in the model feature map, constant Used to characterize numerical stability; S3, generate heat map S31, feature map of real samples M C Weighted processing: each real sample feature map M C The corresponding weight Multiply them to get the weighted feature map of the real sample, sum all the weighted feature maps and normalize the results to get the thermal map h; S32, generate sample feature map M C ′ Perform weighted processing: generate each sample feature map M C ′ The corresponding weight Multiply them together to get the generated sample weighted feature map, and then normalize the sum of all weighted feature maps to get the thermal map h ′ ; S4, consistency quality evaluation By formula For the heatmaps h and h ′ Calculate the consistency, where m and n are the width and height of the heat map, respectively. The mean is obtained. When the mean is ≤ 0.2, it is considered that the generated sample is consistent with the real sample image; otherwise, the generated sample is considered to have poor quality.
2. The method for evaluating the quality of artificially generated image samples according to claim 1, characterized in that: The weights in steps S22 and S23 are calculated by passing the feature map to the test network model and calculating the category k This is achieved by the gradient of .
3. A method for evaluating the quality of artificially generated image samples according to claim 1 or 2, characterized in that: The values after normalization processing in steps S32 and S33 are 0 to 1.