Image feature quality assessment method, device and storage medium

Abstract

The present application discloses a quality assessment method, device and storage medium for image features, and the quality assessment method for image features includes: obtaining an input original image, performing degradation processing on the original image to obtain a degraded image; inputting the degraded image into a feature extraction network to extract image features to obtain degraded image features; inputting the degraded image features into an image restoration network to perform image restoration processing to obtain a restored image; calculating the image similarity between the original image and the restored image, and obtaining a quality score of the image features extracted by the feature extraction network based on the image similarity. By degrading and restoring the image, the feature representation ability of the image features is amplified, and then according to the image similarity between the original image and the restored image, the quality score of the image features extracted by the feature extraction network can be derived, without the need to train a quality assessment model, thereby improving the accuracy of image quality assessment.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and storage medium for evaluating the quality of image features. Background Art

[0002] With the popularization of image recognition technology, the demand for image recognition in unconstrained environments has increased sharply. Due to image acquisition conditions in unconstrained environments (such as lighting, background, blur, low resolution, etc.) and target factors (posture, occlusion, expression, etc.), image recognition performance will be greatly reduced.

[0003] Image quality assessment is an effective means to improve image recognition performance. Currently, image quality assessment is generally carried out based on image acquisition conditions, target factors and other dimensions through a pre-trained quality assessment model. This often requires quality score annotation of model training samples, which leads to problems such as difficulty in annotation and difficulty in establishing quality assessment standards, resulting in inaccurate quality assessment results. Summary of the invention

[0004] In order to solve the above technical problems, the present application at least provides a method, device and storage medium for quality assessment of image features.

[0005] In a first aspect, the present application provides a method for evaluating the quality of image features, the method comprising: obtaining an input original image, performing a degrading process on the original image, and obtaining a degraded image; wherein the original image contains more image information than the degraded image; inputting the degraded image into a feature extraction network for image feature extraction, and obtaining degraded image features; inputting the degraded image features into an image restoration network for image restoration, and obtaining a restored image; calculating the image similarity between the original image and the restored image, and obtaining a quality score of the image features extracted by the feature extraction network based on the image similarity.

[0006] In one embodiment, the image similarity between the original image and the restored image is calculated, including: inputting the original image into a feature extraction network to extract image features to obtain original image features; and inputting the restored image into a feature extraction network to extract image features to obtain restored image features; calculating the feature similarity between the original image features and the restored image features to obtain the image similarity between the original image and the restored image.

[0007] In one embodiment, calculating the image similarity between the original image and the restored image includes: comparing pixel differences between the original image and the restored image; and calculating the image similarity between the original image and the restored image based on the pixel differences.

[0008] In one embodiment, calculating the image similarity between an original image and a restored image includes: inputting the original image into a feature extraction network to extract image features to obtain original image features; and inputting the restored image into a feature extraction network to extract image features to obtain restored image features; calculating the feature similarity between the original image features and the restored image features to obtain a first similarity; and, comparing the pixel difference between the original image and the restored image, and calculating a second similarity based on the pixel difference; and calculating the image similarity between the original image and the restored image by combining the first similarity and the second similarity.

[0009] In one embodiment, the image similarity between the original image and the restored image is calculated by combining the first similarity and the second similarity, including: obtaining weight parameters corresponding to the first similarity and the second similarity, performing weighted calculation on the first similarity and the second similarity based on the weight parameters, and using the weighted calculation result as the image similarity between the original image and the restored image; or, selecting the maximum value or the minimum value of the first similarity and the second similarity, and using the maximum value or the minimum value as the image similarity between the original image and the restored image.

[0010] In one embodiment, different degradation processing strategies are used to perform degradation processing on the original image to obtain multiple degraded images, each degraded image is input into a feature extraction network to extract image features to obtain multiple degraded image features, each degraded image feature is input into an image restoration network to perform image restoration processing to obtain multiple restored images; the image similarity between the original image and the restored image is calculated, and the quality score of the image features extracted by the feature extraction network is obtained based on the image similarity, including: calculating the image similarity between the original image and each restored image to obtain multiple image similarities; based on the multiple image similarities, the quality score of the image features extracted by the feature extraction network is calculated.

[0011] In one embodiment, different degradation processing strategies correspond to different degradation degrees, and a higher degradation degree is proportional to the image information lost in the original image; based on multiple image similarities, a quality score of the image features extracted by the feature extraction network is calculated, including: sorting the multiple degraded images in sequence according to the degradation degrees corresponding to the multiple degraded images; obtaining the image similarity corresponding to each degraded image after the sequential sorting to obtain a similarity sequence; detecting the numerical change of the image similarity in the similarity sequence to obtain the actual numerical change, calculating the difference between the actual numerical change and the preset numerical change to obtain the data change similarity; based on the data change similarity, a quality score of the image features extracted by the feature extraction network is obtained; wherein the data change similarity and the quality score are positively correlated.

[0012] In one embodiment, the degradation processing strategy includes one or more of the following: degradation processing strategy one: blocking part of the image information of the original image; degradation processing strategy two: disrupting part of the image information of the original image; degradation processing strategy three: blurring the image information of the original image; degradation processing strategy four: adding noise to the original image.

[0013] According to a second aspect of the present application, there is provided an image feature quality assessment device, which includes: a degradation processing module, which is used to obtain an input original image, perform degradation processing on the original image, and obtain a degraded image; wherein the original image contains more image information than the degraded image; a feature extraction module, which is used to input the degraded image into a feature extraction network to extract image features, and obtain degraded image features; a restoration processing module, which is used to input the degraded image features into an image restoration network to perform image restoration processing, and obtain a restored image; and a quality assessment module, which is used to calculate the image similarity between the original image and the restored image, and obtain a quality score of the image features extracted by the feature extraction network based on the image similarity.

[0014] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned image feature quality assessment method.

[0015] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned image feature quality assessment method when the program instructions are executed by a processor.

[0016] The above scheme obtains the input original image, degrades the original image to obtain a degraded image; inputs the degraded image into a feature extraction network to extract image features to obtain degraded image features; inputs the degraded image features into an image restoration network to restore the image to obtain a restored image; calculates the image similarity between the original image and the restored image, and obtains the quality score of the image features extracted by the feature extraction network based on the image similarity. By degrading and restoring the image, the feature representation ability of the image features is amplified. Then, based on the image similarity between the original image and the restored image, the quality score of the image features extracted by the feature extraction network can be derived. There is no need to train a quality assessment model, thereby improving the accuracy of image quality assessment.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0019] Figure 1 is a schematic diagram of a solution implementation environment shown in an exemplary embodiment of the present application;

[0020] Figure 2 is a flow chart of a method for evaluating the quality of image features shown in an exemplary embodiment of the present application;

[0021] Figure 3 is a schematic diagram showing a degradation process of an original image according to an exemplary embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a training restoration network shown in an exemplary embodiment of the present application;

[0023] Figure 5 is a schematic diagram of quality assessment of an original image shown in an exemplary embodiment of the present application;

[0024] Figure 6 is a block diagram of an image feature quality assessment device shown in an exemplary embodiment of the present application;

[0025] Figure 7 is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of the structure of a computer-readable storage medium shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0027] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0028] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0029] The term "and / or" in this article is only an association information describing the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0030] The quality assessment method of image features provided in the embodiments of the present application is described below.

[0031] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of a solution implementation environment shown in an exemplary embodiment of the present application. The solution implementation environment may include a terminal 110 and a server 120, and the terminal 110 and the server 120 are connected to each other for communication.

[0032] The number of the terminal 110 may be one or more. The terminal 110 may be a camera, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, etc., but is not limited thereto.

[0033] Server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0034] In one example, the server 120 may perform quality assessment processing on the original image obtained from the terminal 110 to obtain a quality assessment result. Of course, the server 120 may store the quality assessment result locally, transmit it back to the terminal 110, or transmit it to other terminals 110.

[0035] In one example, a client running a target application is installed in the terminal 110. For example, the target application may be an application that provides a quality assessment function. The target application is used to perform quality assessment processing on the original image to obtain a quality assessment result. The server 120 may be a background server of the target application, which is used to provide background services for the client of the target application.

[0036] In the image feature quality assessment method provided in the embodiment of the present application, the execution subject of each step can be the terminal 110, such as the client of the target application installed and running in the terminal 110, or the server 120, or the terminal 110 and the server 120 can interact and cooperate to execute, that is, part of the steps of the method are executed by the terminal 110 and the other steps are executed by the server 120.

[0037] The application scenarios of this application are illustrated by examples:

[0038] For example, in a target recognition scenario, an image to be recognized is obtained, and the quality assessment method of the image features of the present application is used to perform quality assessment on the image to be recognized, and the images to be recognized whose final quality scores are less than the preset score threshold are filtered out, and target detection is performed on the filtered images to be recognized. This can not only improve the accuracy of target recognition, but also prevent a large number of low-quality images from wasting computing resources on the target recognition system, thereby improving the robustness of the target recognition system.

[0039] For another example, in the sample generation scenario of model training, the quality assessment method of the image features of the present application is used to assess the quality of the image samples, filter the image samples whose final quality scores are less than the preset score threshold, and use the filtered image samples as the final image samples to train the model to be trained. A higher quality neural network model can be obtained by training with higher quality image samples.

[0040] It should be noted that the above application scenarios are only schematic illustrations, and the image feature quality assessment method of the present application can be applied to any scenario that requires image quality assessment, and the present application does not limit this.

[0041] See also Figure 2 , Figure 2 is a flow chart of an image feature quality assessment method shown in an exemplary embodiment of the present application. The image feature quality assessment method can be applied to Figure 1 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0042] like Figure 2 As shown, the image feature quality assessment method at least includes steps S210 to S240, which are described in detail as follows:

[0043] Step S210: obtaining an input original image, and performing degradation processing on the original image to obtain a degraded image.

[0044] Among them, the original image contains more image information than the degraded image.

[0045] The original image is degraded to delete or destroy part of the image information in the original image to obtain a degraded image.

[0046] For example, the original image is degraded according to the degradation processing strategy to obtain a degraded image. The degradation processing strategy includes one or more of the following:

[0047] Degradation processing strategy 1: Block part of the image information of the original image;

[0048] Degradation processing strategy 2: disrupt part of the image information of the original image;

[0049] Degradation processing strategy three: blurring the image information of the original image;

[0050] Degradation processing strategy 4: Add noise to the original image.

[0051] Of course, in addition to the above-mentioned degradation processing strategies, other image degradation methods can also be used to degrade the original image, and this application does not limit this.

[0052] For example, see Figure 3 , Figure 3 FIG. 1 is a schematic diagram showing a degradation process of an original image according to an exemplary embodiment of the present application. Figure 3 As shown, the degradation processing strategy one can be adopted to randomly select an occluded area of ​​a preset size from the original image, and delete the image information in the occluded area, such as uniformly replacing the values ​​of the pixels in the occluded area with preset pixel values ​​to obtain a degraded image; or, the degradation processing strategy two can be adopted to randomly select an information scrambled area of ​​a preset size from the original image, and scramble the image information in the information scrambled area, such as randomly replacing the values ​​of each pixel in the information scrambled area to obtain a degraded image; or, the degradation processing strategy three can be adopted to blur the image information of the original image to obtain a degraded image; or, the degradation processing strategy four can be adopted to randomly add noise to the original image, such as adding Gaussian noise, uniform noise, salt and pepper noise, etc. to obtain a degraded image.

[0053] It should be noted that the above-mentioned various degradation treatment strategies can be used in combination or individually, and this application does not limit this.

[0054] In some implementations, the degradation processing strategy further includes a degradation degree corresponding to the original image.

[0055] The higher the degree of degradation, the more image information the original image loses. Conversely, the lower the degree of degradation, the less image information the original image loses.

[0056] For example, the size of the occluded area and / or the information disrupted area in the above embodiment is limited by the degree of degradation, or the blur degree of blur processing in the above embodiment and / or the amount of added noise is limited.

[0057] The degradation degree corresponding to the original image may be pre-set or flexibly set. For example, the corresponding degradation degree may be flexibly determined according to the source of the original image and / or the size of the original image.

[0058] Step S220: inputting the degraded image into a feature extraction network to extract image features and obtain degraded image features.

[0059] The feature extraction network is used to extract image features from the input image. The image features include but are not limited to one or more combinations of texture features, spatial features, color features, and depth features.

[0060] Among them, the feature extraction network can be implemented based on convolutional neural networks (CNN), recurrent neural networks (RNN), etc., which is not limited in this application; and the feature extraction network can be a feature extraction network used in downstream image recognition tasks, such as a feature extraction network used for image feature extraction in a trained target detection neural network model.

[0061] The degraded image is input into a feature extraction network to extract image features, and degraded image features corresponding to the degraded image output by the feature extraction network are obtained.

[0062] Step S230: Inputting the degraded image features into the image restoration network for image restoration processing to obtain a restored image.

[0063] The restoration network is used to restore the image information lost during the degradation process of the original image based on the image information contained in the degraded image.

[0064] It should be noted that the functions of the restoration networks corresponding to different degradation processing strategies may be different. For example, if part of the image information of the original image is occluded, the restoration network is used to estimate the image information of the occluded area in the degraded image according to the degraded image characteristics; if part of the image information of the original image is disrupted, the restoration network is used to sequentially sort the values ​​of each pixel in the disrupted information area in the degraded image according to the degraded image characteristics; if the image information of the original image is blurred, the restoration network is used to enhance the degraded image according to the degraded image characteristics; if noise is added to the original image, the restoration network is used to filter out the noise in the degraded image according to the degraded image characteristics.

[0065] Among them, the restoration network can be implemented based on generative adversarial networks (GAN), variational autoencoders (VAE), convolutional neural networks (CNN), recurrent neural networks (RNN), etc., and the present application does not limit this; and the restoration network can be a pre-trained neural network model, which can be directly connected to the back of the above-mentioned feature extraction model. Of course, the restoration network can also be connected to the back of the feature extraction model and then trained.

[0066] For example, taking the case of image degradation by occluding an image as an example, the training process of the restoration network is explained. Figure 4 , Figure 4 A schematic diagram of a training restoration network shown as an exemplary embodiment of the present application, such as Figure 4 As shown, the sample original image is randomly occluded to obtain a sample occluded image, the sample occluded image is input into a feature extraction network to extract the occluded image features, the occluded image features are input into a restoration network to predict the image information of the occluded area, and the occluded restoration image output by the restoration network is obtained.

[0067] Then, the model training loss is calculated based on the occluded restored image.

[0068] For example, the similarity between the occluded restored image and the sample original image is calculated to obtain a first supervised loss; and / or, the occluded restored image and the sample original image are respectively input into a feature extraction network, the feature extraction network outputs restored image features and original image features respectively, and the similarity between the restored image features and the original image features is calculated to obtain a second supervised loss.

[0069] The network parameters of the restored network are adjusted according to the first supervised loss and / or the second supervised loss until the model training end condition is met to obtain a trained restored network.

[0070] For example, let the first supervised loss be Loss1, and the second supervised loss be Loss2. The first supervised loss and the second supervised loss are weightedly calculated to obtain the total supervised loss Loss=Loss1+a×Loss2, where a is the preset weight coefficient. The network parameters of the restored network are adjusted according to the total supervised loss.

[0071] It should be noted that the training of the restoration network does not affect the original feature extraction network.

[0072] Step S240: Calculate the image similarity between the original image and the restored image, and obtain a quality score of the image features extracted by the feature extraction network based on the image similarity.

[0073] Since the image restoration is performed based on the features of the degraded image, the feature representation capability of the degraded image features is amplified. Therefore, the higher the image similarity between the original image and the restored image, the stronger the feature representation capability of the degraded image features. That is, the more effective image features can be extracted from the degraded image, the higher the quality of the degraded image and the feature extraction network can be deduced, and the higher the quality of the original image can be further deduced. Conversely, the lower the image similarity between the original image and the restored image, the higher the quality of the degraded image and / or the feature extraction network can be deduced, and when the feature extraction network finally extracts image features from the original image, the quality of the extracted image features will be lower.

[0074] Therefore, in some embodiments, the image similarity between the original image and the restored image is calculated, and based on the positive correlation between the image similarity and the quality score, the quality score of the image features extracted by the feature extraction network is calculated according to the image similarity.

[0075] Some embodiments of calculating image similarity are illustrated by way of example:

[0076] Embodiment A1: Calculating the image similarity between an original image and a restored image, comprising: inputting the original image into a feature extraction network to extract image features to obtain original image features; and, inputting the restored image into a feature extraction network to extract image features to obtain restored image features; calculating the feature similarity between the original image features and the restored image features to obtain the image similarity between the original image and the restored image.

[0077] Among them, the cosine similarity algorithm, Euclidean distance algorithm, Manhattan distance algorithm, etc. can be used to calculate the feature similarity between the original image features and the restored image features. The specific calculation method can be flexibly selected according to the actual application scenario, and this application does not limit this.

[0078] The calculated feature similarity between the original image features and the restored image features is used as the image similarity between the original image and the restored image.

[0079] By extracting image features from the original image and calculating the feature similarity between the original image features and the restored image features, the feature representation capability of the original image can be amplified twice to obtain a quality score that is strongly correlated with the feature representation capability of the original image features.

[0080] Embodiment A2: Calculating the image similarity between the original image and the restored image, comprising: comparing the pixel difference between the original image and the restored image; and calculating the image similarity between the original image and the restored image based on the pixel difference.

[0081] Compare the pixel difference of pixels at the same position of the original image and the restored image. If the pixel difference is large, it indicates that the image similarity between the original image and the restored image is low; if the pixel difference is small, it indicates that the image similarity between the original image and the restored image is high.

[0082] By directly comparing the original image and the restored image from the image pixel dimension, the image similarity between the original image and the restored image can be obtained, which can reduce the computing power resources required for calculating the image similarity and improve the computing speed.

[0083] Embodiment A3: Calculating the image similarity between an original image and a restored image, comprising: inputting the original image into a feature extraction network to extract image features to obtain original image features; and inputting the restored image into a feature extraction network to extract image features to obtain restored image features; calculating the feature similarity between the original image features and the restored image features to obtain a first similarity; and, comparing the pixel difference between the original image and the restored image, and calculating a second similarity based on the pixel difference; and calculating the image similarity between the original image and the restored image by combining the first similarity and the second similarity.

[0084] That is, the steps of Embodiment A1 and Embodiment A2 are combined to calculate the first similarity and the second similarity respectively.

[0085] For example, the quality assessment process of the original image is described by taking the image degradation process achieved by occluding the image as an example. Figure 5 , Figure 5 A schematic diagram of quality assessment of an original image is shown as an exemplary embodiment of the present application. Figure 5 As shown, the original image is randomly occluded to obtain a degraded image, the degraded image is input into a feature extraction network to extract degraded image features, the degraded image features are input into a restoration network to predict image information of the occluded area, and the restored image output by the restoration network is obtained.

[0086] Then, the original image is input into the feature extraction network for image feature extraction to obtain the original image features; and the restored image is input into the feature extraction network for image feature extraction to obtain the restored image features; the feature similarity between the original image features and the restored image features is calculated to obtain the first similarity; and the pixel difference between the original image and the restored image is compared, and based on the pixel difference, the second similarity between the original image and the restored image is calculated.

[0087] The first similarity and the second similarity are combined to calculate the image similarity between the original image and the restored image.

[0088] For example, the weight parameters corresponding to the first similarity and the second similarity are obtained, the first similarity and the second similarity are weightedly calculated based on the weight parameters, and the weighted calculation result is used as the image similarity between the original image and the restored image.

[0089] For example, the first similarity is expressed as w1, and the second similarity is expressed as w2, and the first similarity and the second similarity are weightedly calculated to obtain the image similarity W=w1+b×w2.

[0090] For another example, the maximum value or the minimum value of the first similarity and the second similarity is selected, and the maximum value or the minimum value is used as the image similarity between the original image and the restored image.

[0091] It should be noted that the specific calculation method of image similarity can be flexibly selected according to the actual application scenario, and this application does not limit this.

[0092] Some implementation examples of calculating quality scores are given below:

[0093] Embodiment B1: A degradation processing strategy is used to degrade the original image. After obtaining the image similarity between the original image and the restored image corresponding to the degraded image, the image similarity is directly mapped to the quality score of the image features extracted by the feature extraction network based on the positive correlation between the image similarity and the quality score. The quality score can not only reflect the image feature extraction capability of the feature extraction network, that is, reflect the quality of the feature extraction network, but also reflect the quality of the image features that can be extracted from the original image.

[0094] Embodiment B2: Different degradation processing strategies are used to perform degradation processing on the original image to obtain multiple degraded images, each degraded image is input into a feature extraction network to perform image feature extraction to obtain multiple degraded image features, each degraded image feature is input into an image restoration network to perform image restoration processing to obtain multiple restored images.

[0095] The image similarity between the original image and the restored image is calculated, and the quality score of the image features extracted by the feature extraction network is obtained based on the image similarity, including: calculating the image similarity between the original image and each restored image respectively to obtain multiple image similarities; based on the multiple image similarities, calculating the quality score of the image features extracted by the feature extraction network.

[0096] The similarities of multiple images are calculated by combining different degradation processing strategies, and the quality scores of image features extracted by the feature extraction network are calculated by combining the similarities of multiple images, thereby improving the accuracy of the quality score.

[0097] In some implementations, a weighted calculation may be performed on multiple image similarities, and the weighted calculation result may be used as the final image similarity. Based on the positive correlation between image similarity and quality score, the final image similarity may be mapped to the quality score of the image features extracted by the feature extraction network.

[0098] Wherein, when performing weighted calculation on the similarities of multiple images, the weight parameters used may be preset or flexibly calculated.

[0099] For example, the degradation degree of the degradation processing strategy corresponding to the degraded image is obtained, and the weight parameter of the image similarity calculated corresponding to the degraded image is set based on the degradation degree, so as to perform weighted calculation on multiple image similarities based on the weight parameter corresponding to each image similarity, and use the weighted calculation result as the final image similarity. The larger the difference between the degradation degree and the preset degree threshold, the smaller the corresponding weight parameter, and the smaller the difference between the degradation degree and the preset degree threshold, the larger the corresponding weight parameter.

[0100] In some implementations, the maximum value or minimum value may be selected from multiple image similarities, and based on the positive correlation between image similarity and quality score, the selected maximum value or minimum value may be mapped to the quality score of the image feature extracted by the feature extraction network.

[0101] In some embodiments, different degradation processing strategies correspond to different degrees of degradation, and the higher the degree of degradation, the more image information is lost in the original image; based on the similarities of multiple images, the quality score of the image features extracted by the feature extraction network is calculated, including: sorting the multiple degraded images in sequence according to the degrees of degradation corresponding to the multiple degraded images; obtaining the image similarity corresponding to each degraded image after the sequential sorting to obtain a similarity sequence; detecting the numerical change of the image similarity in the similarity sequence to obtain the actual numerical change, calculating the difference between the actual numerical change and the preset numerical change to obtain the data change similarity; and obtaining the quality score of the image features extracted by the feature extraction network based on the data change similarity.

[0102] Among them, the data change similarity and quality score are positively correlated.

[0103] For original images with stable feature representation capabilities and feature extraction networks with stable feature extraction capabilities, the image similarity between the restored images and the original images restored by restoring the degraded image features corresponding to different degradation degrees should comply with preset numerical change rules, wherein the preset numerical change rules can be calculated based on multiple sample original images, and the preset numerical change rules may include rules for limiting the data fluctuation range, and may also include rules for limiting the degree of data change (such as the difference between the maximum and minimum values), and may also include rules for data change curves, which may be flexibly set according to actual conditions.

[0104] For example, the data fluctuation range is obtained according to the actual numerical change, and the difference between the data fluctuation range and the data fluctuation range in the preset numerical change rule is detected to obtain the data change similarity. The smaller the overlap between the data fluctuation range obtained from the actual numerical change and the data fluctuation range in the preset numerical change rule, the smaller the data change similarity. Conversely, the greater the overlap, the greater the data change similarity.

[0105] Similarly, the difference between the actual numerical change and the preset numerical change can be evaluated from dimensions such as the degree of data change and the data change curve.

[0106] Taking the example of occluding an image to achieve image degradation processing, the sizes of the occlusion areas corresponding to different degradation processing strategies are different. The degraded images are sorted in sequence (such as descending order or ascending order) according to the size of the occlusion area corresponding to each degraded image. For example, if the degraded images are sorted in ascending order according to the size of the occlusion area corresponding to each degraded image, P1, P2, P3, and P4 are obtained, among which the occlusion area corresponding to the degraded image P1 is the smallest, and the occlusion area corresponding to the degraded image P4 is the largest.

[0107] The image similarity S1 corresponding to the degraded image P1, the image similarity S2 corresponding to the degraded image P2, the image similarity S3 corresponding to the degraded image P3, and the image similarity S4 corresponding to the degraded image P4 are obtained, and based on the sorting results of the degraded images, the similarity sequences S1, S2, S3, and S4 are obtained. The numerical changes of the similarity sequences S1, S2, S3, and S4 are analyzed to obtain the data fluctuation range, and / or the difference between the maximum value and the minimum value, and / or the data change curve, and the analysis results are compared with the preset numerical change rules to obtain the data change similarity, and based on the positive correlation between the data change similarity and the quality score, the data change similarity is mapped to the quality score.

[0108] For example, if the numerical change indicates that the numerical values ​​of S1, S2, S3, and S4 in the similarity sequence decrease in sequence, and the difference between the maximum and minimum values ​​is less than the first preset threshold, and the data fluctuation range is within the preset fluctuation range, then it can be judged that the data change similarity between the actual numerical change and the preset numerical change is highly similar; if the numerical change indicates that the numerical values ​​of S1, S2, S3, and S4 in the similarity sequence decrease in sequence, and the difference between the maximum and minimum values ​​is less than the second preset threshold (the first preset threshold is less than the second threshold), and the overlap between the data fluctuation range and the preset fluctuation range is 80%, then it can be judged that the data change similarity between the actual numerical change and the preset numerical change is moderately similar.

[0109] In some implementations, multiple quality scores may be calculated in combination with the above multiple embodiments, and weighted calculation may be performed on the multiple quality scores to obtain a more accurate final quality score.

[0110] In some embodiments, the quality of the original image is evaluated from dimensions such as the brightness, and / or resolution, and / or clarity, and / or target posture, and / or target occlusion of the original image to obtain a quality score for the image content of the original image, and the above steps S210 to S240 are used to perform quality evaluation on the original image to obtain a quality score for the image features of the original image. The quality score of the image content and the quality score of the image features are combined to obtain a final quality score for the original image.

[0111] The image feature quality assessment method provided by the present application obtains an input original image, degrades the original image, and obtains a degraded image; wherein the original image contains more image information than the degraded image; the degraded image is input into a feature extraction network for image feature extraction to obtain degraded image features; the degraded image features are input into an image restoration network for image restoration to obtain a restored image; the image similarity between the original image and the restored image is calculated, and the quality score of the image features extracted by the feature extraction network is obtained based on the image similarity. By degrading and restoring the image, the feature representation ability of the image features is amplified, and then according to the image similarity between the original image and the restored image, the quality score of the image features extracted by the feature extraction network can be derived, without the need to train a quality assessment model, thereby improving the accuracy of image quality assessment.

[0112] Figure 6 FIG. 1 is a block diagram of an image feature quality assessment device shown in an exemplary embodiment of the present application. Figure 6 As shown, the exemplary image feature quality assessment device 600 includes:

[0113] The degradation processing module 610 is used to obtain an input original image, perform degradation processing on the original image, and obtain a degraded image; wherein the image information contained in the original image is greater than the image information contained in the degraded image;

[0114] A feature extraction module 620 is used to input the degraded image into a feature extraction network to extract image features and obtain degraded image features;

[0115] Restoration processing module 630, used for inputting degraded image features into an image restoration network to perform image restoration processing to obtain a restored image;

[0116] The quality assessment module 640 is used to calculate the image similarity between the original image and the restored image, and obtain the quality score of the image features extracted by the feature extraction network based on the image similarity.

[0117] It should be noted that the image feature quality assessment device provided in the above embodiment and the image feature quality assessment method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here. In practical applications, the image feature quality assessment device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0118] See also Figure 7 , Figure 7 7 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 700 includes a memory 701 and a processor 702, and the processor 702 is used to execute program instructions stored in the memory 701 to implement the steps in the above-mentioned quality assessment method embodiment of any image feature. In a specific implementation scenario, the electronic device 700 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 700 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0119] Specifically, the processor 702 is used to control itself and the memory 701 to implement the steps in the above-mentioned quality assessment method embodiment of any image feature. The processor 702 can also be called a central processing unit (CPU). The processor 702 may be an integrated circuit chip with signal processing capabilities. The processor 702 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 702 can be implemented by an integrated circuit chip.

[0120] See also Figure 8 , Figure 8 The computer-readable storage medium 800 stores program instructions 810 that can be executed by a processor, and the program instructions 810 are used to implement the steps in any of the above-mentioned image feature quality assessment method embodiments.

[0121] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0122] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0123] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0124] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

Claims

1. A method for evaluating the quality of image features, characterized in that: The method comprises: Obtain an input original image, and use different degradation processing strategies to perform degradation processing on the original image respectively to obtain multiple degraded images; wherein the image information contained in the original image is more than the image information contained in the degraded image, the different degradation processing strategies correspond to different degradation degrees, and the degradation degree is proportional to the image information lost in the original image; Inputting the multiple degraded images into a feature extraction network respectively to extract image features, thereby obtaining multiple degraded image features; Inputting the plurality of degraded image features into an image restoration network respectively for image restoration processing to obtain a plurality of restored images; wherein the image restoration network is used to amplify the feature characterization capability of the degraded image features, reflecting the number of effective image features extracted from the degraded images; respectively calculating image similarities between the original image and the plurality of restored images to obtain a plurality of image similarities; sorting the multiple degraded images in order according to the degradation degrees respectively corresponding to the multiple degraded images; Obtain the image similarity corresponding to each degraded image after sequential sorting to obtain a similarity sequence; Detecting the numerical change of the image similarity in the similarity sequence to obtain an actual numerical change, and calculating the difference between the actual numerical change and a preset numerical change to obtain data change similarity; A quality score of the image features extracted by the feature extraction network is obtained based on the data change similarity; wherein the data change similarity and the quality score are positively correlated.

2. The method according to claim 1, characterized in that Calculating the image similarity between the original image and the restored image, comprising: Inputting the original image into the feature extraction network to extract image features to obtain original image features; and, Inputting the restored image into the feature extraction network to extract image features to obtain restored image features; The feature similarity between the original image feature and the restored image feature is calculated to obtain the image similarity between the original image and the restored image.

3. The method according to claim 1, characterized in that Calculating the image similarity between the original image and the restored image, comprising: Comparing pixel differences between the original image and the restored image; Based on the pixel difference, the image similarity between the original image and the restored image is calculated.

4. The method according to claim 1, characterized in that: Calculating the image similarity between the original image and the restored image, comprising: Inputting the original image into the feature extraction network to extract image features, thereby obtaining original image features; and inputting the restored image into the feature extraction network to extract image features, thereby obtaining restored image features; and calculating feature similarity between the original image features and the restored image features, thereby obtaining a first similarity; and, comparing pixel differences between the original image and the restored image, and calculating a second similarity based on the pixel differences; The image similarity between the original image and the restored image is calculated by combining the first similarity and the second similarity.

5. The method according to claim 4, characterized in that The combining the first similarity and the second similarity to calculate the image similarity between the original image and the restored image includes: Obtaining weight parameters corresponding to the first similarity and the second similarity, performing weighted calculation on the first similarity and the second similarity based on the weight parameters, and using the weighted calculation result as the image similarity between the original image and the restored image; Alternatively, a maximum value or a minimum value between the first similarity and the second similarity is selected, and the maximum value or the minimum value is used as the image similarity between the original image and the restored image.

6. The method according to claim 1, characterized in that The degradation treatment strategies include one or more of the following: Degradation processing strategy 1: masking part of the image information of the original image; Degradation processing strategy 2: disrupting part of the image information of the original image; Degradation processing strategy three: blurring the image information of the original image; Degradation processing strategy 4: adding noise to the original image.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, and the processor is used to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and the program instructions can be executed by a processor to implement the steps in the method according to any one of claims 1 to 6.

Images