Method, medium, device and computing device for determining clarity

Abstract

The embodiments of the present invention provide a method, medium, device and computing device for determining clarity. The method includes: extracting features of a target image; determining the clarity level probability of each region of the target image and the weight of each region based on the features; and determining the clarity level of the target image using the clarity level probability of each region and the weight of each region. The embodiment of the present invention determines the clarity of the target image with reference to the weight and clarity level probability of each region of the target image. Compared with related technologies, the clarity determination process can be automated and made more efficient without manual labeling. In addition, compared with the image block extraction method, computing power can be saved, and the accuracy of determining the clarity level is higher.

Description

Background Art

[0002] This section is intended to provide a background or context to embodiments of the invention that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] At present, the image clarity judgment can be divided into reference judgment and non-reference judgment. Reference judgment means that there is an original image, and the relationship between the clarity of the original image and the reference image is judged. Non-reference judgment means that there is no reference image, and the relationship between the clarity of the original image and the pre-set clarity standard is judged.

[0004] For non-reference determination, related technologies rely on manual annotation or extracting image blocks from the image whose clarity is to be determined for clarity confirmation. However, related technologies have defects such as high annotation costs and low accuracy due to poor representativeness of the extracted image blocks and high overlap rates. Summary of the invention

[0005] The present invention is intended to provide a method, medium, device and computing equipment for determining clarity, so as to solve the problems of high cost of manual annotation and repeated calculation of image blocks, and to improve the efficiency and accuracy of image clarity determination.

[0006] In a first aspect of the embodiments of the present invention, a method for determining clarity is provided, which may include:

[0007] Extract the features of the target image;

[0008] According to the features, the clarity level probability of each area of ​​the target image and the weight of each area are determined;

[0009] The clarity level of the target image is determined using the clarity level probability of each region and the weight of each region.

[0010] In one embodiment of the present disclosure, determining the definition level of the target image using the definition level probability of each region and the weight of each region may include:

[0011] Perform dimensionality reduction processing on the clarity level probability and weight of each area to obtain the dimensionality reduction processing result;

[0012] Calculate the weight sum of each region;

[0013] The clarity level of the target image is determined by using the dimensionality reduction processing result and the weight sum.

[0014] In one embodiment of the present disclosure, determining the definition level of the target image by using the dimensionality reduction processing result and the weight sum may include:

[0015] Calculate the ratio of the dimension reduction result to the weight sum;

[0016] According to the ratio, the clarity level of the target image is obtained using the classification model.

[0017] In one embodiment of the present disclosure, extracting features of a target image includes:

[0018] Scaling the target image to obtain a target image with a predetermined resolution;

[0019] The feature extraction model is used to extract features of a target image with a predetermined resolution, and a feature matrix of the target image with the predetermined resolution is obtained; and the feature matrix is ​​used as a feature of the target image.

[0020] In one embodiment of the present disclosure, determining the definition level probability of each region of the target image and the weight of each region according to the feature may include:

[0021] Using the clarity level recognition model, the probability of different clarity levels corresponding to each area of ​​the target image is obtained according to the features;

[0022] Using the weight recognition model, the weight corresponding to each area of ​​the target image is obtained according to the features;

[0023] The probability of different sharpness levels for each region is associated with a weight.

[0024] In one embodiment of the present disclosure, it also includes:

[0025] Using the feature extraction model, extract the features of the target image;

[0026] Using the clarity level recognition model, the probability of different clarity levels corresponding to each area of ​​the target image is obtained;

[0027] Using the weight recognition model, the weight corresponding to each area of ​​the target image is obtained;

[0028] Using the classification model, determine the clarity level of the target image;

[0029] The feature extraction model, the clarity level recognition model, the weight recognition model and the classification model constitute an end-to-end model. The input of the end-to-end model is the target image, and the output is the clarity level of the target image.

[0030] In one embodiment of the present disclosure, when the target image is obtained by extracting a video frame from the target video, the method further includes:

[0031] The definition level of the target video is determined according to the definition level of each target image.

[0032] In a second aspect of the embodiments of the present invention, a device for determining clarity is provided, which may include:

[0033] A feature extraction module is used to extract the features of the target image;

[0034] An analysis module, used for determining the clarity level probability of each area of ​​the target image and the weight of each area according to the features;

[0035] The definition level determination module is used to determine the definition level of the target image by using the definition level probability of each area and the weight of each area.

[0036] In one embodiment of the present disclosure, the clarity level determination module may include:

[0037] The dimensionality reduction processing submodule is used to perform dimensionality reduction processing on the clarity level probability and weight of each area to obtain the dimensionality reduction processing result;

[0038] The weight and calculation submodule is used to calculate the weight and sum of each region;

[0039] The definition level determination execution submodule is used to determine the definition level of the target image by using the dimension reduction processing result and the weight sum.

[0040] In one embodiment of the present disclosure, the clarity level determination execution submodule includes:

[0041] A ratio calculation unit, used to calculate the ratio of the dimension reduction processing result and the weight sum;

[0042] The clarity level determination execution unit is used to obtain the clarity level of the target image according to the ratio using a classification model.

[0043] In one embodiment of the present disclosure, the feature extraction module may include:

[0044] A scaling processing submodule is used to perform scaling processing on the target image to obtain a target image with a predetermined resolution;

[0045] The feature extraction execution submodule is used to use the feature extraction model to extract features of the target image of the predetermined resolution, obtain a feature matrix of the target image of the predetermined resolution, and use the feature matrix as the feature of the target image.

[0046] In one embodiment of the present disclosure, the analysis module may include:

[0047] A clarity level probability determination submodule is used to obtain different clarity level probabilities corresponding to each area of ​​the target image according to the features by using a clarity level recognition model;

[0048] The weight determination submodule is used to obtain the weight corresponding to each area of ​​the target image according to the features by using the weight recognition model;

[0049] The association submodule is used to associate the different clarity level probabilities of each region with weights.

[0050] In one embodiment of the present disclosure, it also includes:

[0051] The feature extraction module uses the feature extraction model to extract the features of the target image;

[0052] The analysis module uses the clarity level recognition model to obtain the probability of different clarity levels corresponding to each area of ​​the target image;

[0053] The analysis module also uses the weight recognition model to obtain the weight corresponding to each area of ​​the target image;

[0054] The clarity level determination module determines the clarity level of the target image using a classification model;

[0055] The feature extraction model, the clarity level recognition model, the weight recognition model and the classification model constitute an end-to-end model. The input of the end-to-end model is the target image, and the output is the clarity level of the target image.

[0056] In one embodiment of the present disclosure, when the target image is obtained by extracting a video frame from the target video, the method further includes:

[0057] The target video definition level determination module is used to determine the definition level of the target video according to the definition levels of each target image.

[0058] In a third aspect of the embodiments of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the above-mentioned method for determining clarity are implemented.

[0059] In a fourth aspect of the embodiments of the present invention, a computing device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for determining clarity when executing the program.

[0060] According to the method and device for determining clarity of the embodiment of the present invention, features can be extracted based on the entire target image to obtain the weight and clarity level probability of each area of ​​the target image. The judgment of the clarity of the target image refers to the weight and clarity level probability of each area of ​​the target image. Compared with the related art, the clarity determination process can be automated and efficient without manual annotation. In addition, compared with the image block extraction method, the above-mentioned solution of the present application has a higher accuracy in determining the clarity level. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0062] Figure 1 The following schematically shows the implementation process of the method for determining the clarity according to an embodiment of the present invention. Figure 1 ;

[0063] Figure 2 A flowchart for implementing step S13 in a method for determining clarity according to an embodiment of the present invention is schematically shown;

[0064] Figure 3 A flowchart for implementing step S23 in a method for determining clarity according to an embodiment of the present invention is schematically shown;

[0065] Figure 4 A flowchart for implementing step S11 in a method for determining clarity according to an embodiment of the present invention is schematically shown;

[0066] Figure 5 A flowchart for implementing step S12 in a method for determining clarity according to an embodiment of the present invention is schematically shown;

[0067] Figure 6 The following schematically shows the implementation process of the method for determining the clarity according to an embodiment of the present invention. Figure 2 ;

[0068] Figure 7 A schematic diagram of a medium for determining a method for clarity according to an embodiment of the present invention is schematically shown;

[0069] Figure 8 The structure diagram of a device for determining clarity according to an embodiment of the present invention is schematically shown;

[0070] Fig. 9 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown schematically.

[0071] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0072] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0073] Those skilled in the art will appreciate that the embodiments of the present invention may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0074] According to an embodiment of the present invention, a method, medium, apparatus and computing device for determining clarity are proposed.

[0075] In this document, any number of elements in the drawings is used for illustration rather than limitation, and any naming is used only for distinction and does not have any limiting meaning.

[0076] The principle and spirit of the present invention are explained in detail below with reference to several representative embodiments of the present invention. SUMMARY OF THE INVENTION

[0078] The inventors have found that the existing non-reference determination of image clarity relies on manual annotation or extracting image blocks from the image whose clarity level is to be determined. The manual annotation method has a high annotation cost. In addition, due to the problems of poor representativeness and high overlap of the extracted image blocks, the accuracy of the determined image clarity is poor.

[0079] In view of this, the present invention provides a method and device for determining clarity, which can extract features based on the entire target image to obtain the weight and clarity level probability of each area of ​​the target image. The judgment of the clarity of the target image refers to the weight and clarity level probability of each area of ​​the target image. The high cost of manual annotation, poor representativeness of image blocks, and high overlap rate are solved, and the efficiency and accuracy of image clarity determination are improved.

[0080] After introducing the basic principles of the present invention, various non-limiting embodiments of the present invention are described in detail below.

[0081] Exemplary Methods

[0082] Reference below Figure 1 A method for determining clarity according to an exemplary embodiment of the present invention will be described.

[0083] like Figure 1 As shown, the method for determining the clarity of the embodiment of the present invention includes the following steps:

[0084] S11: extracting features of the target image;

[0085] S12: determining the clarity level probability of each area of ​​the target image and the weight of each area according to the features;

[0086] S13: Determine the clarity level of the target image using the clarity level probability of each region and the weight of each region.

[0087] The target image may be an image extracted from a video, or a photographic image, etc. The corresponding execution subject may be a server at the video display end, a server at the image display end, etc.

[0088] The features of the target image can be extracted using a pre-trained image feature extraction model. The image feature extraction model can include multiple convolutional layers, and a normalization layer connected to the terminal convolutional layer. By inputting the target image into the image feature extraction model, a feature map of the target image can be obtained. The feature map can characterize the features of the target image.

[0089] Using the characteristics of the target image, the sharpness level probability of each area of ​​the target image can be determined.

[0090] The clarity level can be set into multiple categories. For example, it can include four levels: high clarity, relatively high clarity, average clarity and blur. Correspondingly, the clarity level probability of the target image in each area includes four results. For example, for the first area of ​​the target image, the probability of high clarity is a%, the probability of relatively high clarity is b%, the probability of average clarity is c%, and the probability of blur is d%. Among them, a%+b%+c%+d%=100%.

[0091] The above-mentioned determination of the clarity level probability of each area of ​​the target image can be achieved by a pre-trained clarity level recognition model.

[0092] In addition, the weight of each area of ​​the target image can be determined by using the characteristics of the target image. For example, the area at the center of the target image has a higher weight. Alternatively, the area with rich content in the target image has a higher weight. For example, if the target image includes a ship drifting on the water, the weight of the area where the ship is located in the target image is higher, and the weight of the area where the water surface and the sky are located is lower.

[0093] The above-mentioned determination of the weight of each region of the target image can be achieved by a pre-trained weight recognition model.

[0094] According to the weight of each area of ​​the target image and the clarity level probability, the clarity level of the target image can be determined. For example, the area with a weight higher than the corresponding threshold can be selected as the effective area. The mean of the clarity level probability of the effective area is calculated. The clarity level corresponding to the mean is used as the clarity level of the target image.

[0095] For example, if the number of effective regions is m, the clarity level probabilities of all effective regions can be obtained.

[0096] For example, for the case where the level is high definition, the mean of the probability of high definition in all valid areas can be expressed as a 有效区域 %, the mean value of the probability of high clarity in all effective areas can be obtained by using formula (1):

[0097] a 有效区域 % = (a1% + a2% + ... + a m %) / m——(1).

[0098] In the calculation formula (1), a1% can represent the probability that the first effective area has high resolution, a m % can represent the probability that the mth effective area has high clarity.

[0099] Similarly, the mean value b of the probability of higher clarity in all effective areas can be calculated separately 有效区域 %, the mean value c of the probability that the clarity of all effective areas is average 有效区域 %, and the mean probability of blurring of all valid areas d 有效区域 %.

[0100] You can 有效区域 %、b 有效区域 %、c 有效区域 %, d 有效区域 The clarity level corresponding to the maximum value in % is determined as the clarity level of the target image.

[0101] For another example, the weight sum of all regions can be calculated, and the importance of each region in the target image can be obtained based on the ratio of the weight of each region to the weight sum. Then, based on the importance of each region and the clarity level probability of each region, the clarity level probability of the target image as a whole can be obtained. The mean of the clarity level probabilities is calculated, and the clarity level corresponding to the maximum mean is used as the clarity level of the target image.

[0102] Still taking the above-mentioned clarity including 4 levels as an example, for example, for the case where the clarity level is high, the product of the importance of each region and the probability of the clarity level being high is calculated respectively to obtain the product result. Then the product result of each region is summed up to obtain the sum of the weighted probabilities of the target image having a high clarity level as a whole.

[0103] According to the ratio of the sum of weighted probabilities that the target image has a high clarity level as a whole to the weighted sum, the mean of the probability that each region of the target image has a high clarity level can be obtained.

[0104] Similarly, the mean probability of each region having high clarity, the mean probability of each region having average clarity, and the mean probability of each region being blurred can be calculated respectively.

[0105] By comparison, the clarity level corresponding to the maximum value in the above averages can be determined as the clarity level of the target image.

[0106] The above process is particularly suitable for situations where there is no reference for judgment. Since the above scheme is based on the overall feature extraction of the target image, the weight and clarity level probability of each area of ​​the target image are obtained. The judgment of the clarity of the target image refers to the weight and clarity level probability of each area of ​​the target image. Compared with the related art, the automation and efficiency of clarity determination can be achieved without manual labeling. In addition, compared with the image block extraction method, the above scheme of the present application has a higher accuracy in determining the clarity level.

[0107] like Figure 2 As shown, in a possible implementation manner, the step S13 of determining the definition level of the target image may specifically include the following steps:

[0108] S21: performing dimensionality reduction processing on the clarity level probability and weight of each area to obtain a dimensionality reduction processing result;

[0109] S22: Calculate the weight sum of each region;

[0110] S23: Determine the clarity level of the target image by using the dimensionality reduction processing result and the weight sum.

[0111] Exemplarily, the clarity level probability of each region may be characterized in the form of logits, and the clarity level probability may be used as a parameter of each region.

[0112] After the weight of each region is determined, the weight can be used as a parameter of the corresponding region. The above parameters can be subjected to dimensionality reduction processing to obtain dimensionality reduction processing results. For example, the dimensionality reduction processing method can include global average pooling. For each clarity level, global average pooling can sum the probabilities of the clarity levels corresponding to each region whose weight is determined, that is, the object of the summation calculation can be the product of the weight of each region and the probabilities of different clarity levels.

[0113] For example, for the case where the clarity level is high, the product of the weight of each region and the probability that the clarity level is high is calculated to obtain the product result. The dimensionality reduction processing result may correspond to the sum of the product results of each region.

[0114] Similarly, the above method can be used to obtain the corresponding dimensionality reduction processing results for the case of high clarity, the case of average clarity, and the case of blurred clarity.

[0115] Compared with using a fully connected layer to process the clarity level probability and weight of each region, the above-mentioned global average pooling method can significantly reduce the number of parameters, thereby avoiding overfitting.

[0116] In addition, the dimensionality reduction processing method may also include global maximum pooling, etc., which is not limited here.

[0117] On the other hand, the weighted sum of each region of the target image can be calculated.

[0118] Take the above four clarity levels as an example:

[0119] For the case where the clarity level is high, the clarity level probability of each area is expressed as: a1%, a2%, ..., a n %.

[0120] For the case where the clarity level is higher, the clarity level probability of each area is expressed as: b1%, b2%, ..., b n %.

[0121] For the case where the clarity level is average, the clarity level probability of each area is expressed as: c1%, c2%, ..., c n %.

[0122] For the case where the clarity level is blurred, the clarity level probability of each area is expressed as: d1%, d2%, ..., d n %.

[0123] According to the dimensionality reduction processing result and the weight sum, the mean value of the probability that each area of ​​the target image corresponds to different clarity levels can be obtained.

[0124] For example, when the level is high definition, the probability of high definition in all regions may be averaged to obtain a'%.

[0125] Similarly, we obtain the probability mean b'% that all regions have high definition, the probability mean c'% that all regions have average definition, and the probability mean d'% that all regions are blurred.

[0126] By comparing the above probability means, the definition level of the target image can be obtained. Alternatively, the definition level of the target image can be obtained by using a classification model or other methods.

[0127] Through the above scheme, the dimension reduction processing method can be used to reduce the amount of calculation while ensuring the accuracy of the calculation results, thereby improving the overall efficiency of determining the clarity level.

[0128] like Figure 3 As shown, in a possible implementation manner, step S23 may specifically include the following steps:

[0129] S31: Calculate the ratio of the dimension reduction processing result to the weight sum;

[0130] S32: According to the ratio, a classification model is used to obtain a clarity level of the target image.

[0131] The dimensionality reduction processing result includes the weight information of each region and the clarity level probability information. The importance of each region in the target image can be determined by using the weight information in the dimensionality reduction processing result and the weight sum of each region calculated in the above-mentioned implementation. That is, the ratio of the dimensionality reduction processing result to the weight sum can characterize the importance of each region in the target image.

[0132] For example, the target image includes n regions. The weight of each region can be expressed as q i i can represent the sequence number of the region, 1<i<n. The sum of the weights of all regions can be expressed as q.

[0133] When the level is high definition, the probability that all regions have high definition can be expressed as a'%.

[0134] a'%=(a1%q1+a2%q2+……+a n %q n ) / q——(2).

[0135] a1% to a n % can correspond to the probability of high clarity of each area. In formula (2), the importance of each area is expressed as q i / q.

[0136] Similarly, the probabilities corresponding to other clarity levels of all areas can be obtained.

[0137] According to the importance of each area in the target image and the clarity level probability of each area, the clarity level of the target image can be obtained using the classification model.

[0138] The classification model can be pre-trained, for example, the input data of the classification model to be trained is a sample of the ratio of the dimensionality reduction processing result to the weight sum. The output of the classification model to be trained is the predicted value of the clarity level of the target image. The classification model to be trained is trained using the difference between the predicted value of the clarity level of the target image and the true value of the clarity level of the target image, until the difference between the predicted value of the clarity level of the target image and the true value of the clarity level of the target image is within an allowable range. In one example, the classification model can be implemented using a Softmax function.

[0139] Through the above scheme, the clarity level of the entire target image can be directly determined using the classification model.

[0140] like Figure 4 As shown, in a possible implementation manner, the extraction of features of the target image involved in step S11 may specifically include the following steps:

[0141] S41: scaling the target image to obtain a target image with a predetermined resolution;

[0142] S42: Using a feature extraction model, extract features from a target image of a predetermined resolution to obtain a feature matrix of the target image of the predetermined resolution; and use the feature matrix as features of the target image.

[0143] The received target images of different resolutions can be scaled to a predetermined resolution. The feature extraction, clarity level probability determination, weight determination and other processes involved in the embodiments of the present application can be implemented by a model. For the stability of model training, image samples of predetermined resolution can be used when training the model. Therefore, when the clarity of the target image is confirmed, the resolution of the target image can be adjusted to the same resolution as the image sample. That is, adjusting the resolution of the target image to a predetermined resolution can meet the accuracy of the model's confirmation result of the clarity level of the target image.

[0144] The feature extraction model may include multiple convolutional layers and a normalization layer connected to the last convolutional layer. Multiple convolutional layers are used to perform convolution operations on the target image to obtain a feature map. The feature map can be expressed in the form of a matrix, that is, it corresponds to a feature matrix.

[0145] For example, the feature matrix can be expressed as H*W*C, where H can represent the length of the feature matrix, W can represent the width of the feature matrix, and C can represent the number of channels (number of layers) of the feature matrix.

[0146] The normalization layer is used to normalize the data in the feature matrix output by the last convolutional layer to convert it into a predetermined numerical range. The predetermined numerical range can be between 0 and 1.

[0147] Through the above scheme, the feature extraction model can be used to realize the automatic extraction of target image features.

[0148] like Figure 5 As shown, in a possible implementation manner, the step S12 involves determining the definition level probability of each region of the target image and the weight of each region according to the features, which may specifically include the following steps:

[0149] S51: using a clarity level recognition model, obtaining different clarity level probabilities corresponding to each area of ​​the target image according to the features;

[0150] S52: using the weight recognition model to obtain the weight corresponding to each area of ​​the target image according to the features;

[0151] S53: Associating different clarity level probabilities of each region with weights.

[0152] In the current step, different models can be used to process the features of the target image, so as to obtain different clarity level probabilities corresponding to each area of ​​the target image and the weight corresponding to each area of ​​the target image.

[0153] The corresponding identification information can be loaded for each region of the target image. For any region, the probabilities and weights of different definition levels can be associated through the identification information of the region.

[0154] Through the above scheme, the model can be used to obtain the probabilities and weights of different clarity levels of the target image in each area.

[0155] In a possible implementation manner, steps S11 to S13 may also be specifically implemented in the following manner:

[0156] Using the feature extraction model, extract the features of the target image;

[0157] Using the clarity level recognition model, the probability of different clarity levels corresponding to each area of ​​the target image is obtained;

[0158] Using the weight recognition model, the weight corresponding to each area of ​​the target image is obtained;

[0159] Using the classification model, determine the clarity level of the target image;

[0160] The feature extraction model, the clarity level recognition model, the weight recognition model and the classification model constitute an end-to-end model. The input of the end-to-end model is the target image, and the output is the clarity level of the target image.

[0161] In the current implementation, an end-to-end model may be used to perform feature extraction of a target image, determination of probabilities and weights of different clarity levels for each region, determination of a clarity level of the target image, and the like.

[0162] Compared with the method of implementing each process using a corresponding model, the end-to-end model in the current embodiment is used as an overall model. The input of the model is the target image, and the output is the clarity level of the target image. Since the input of the end-to-end model is the target image, the model processes the target image as a whole, so that the output result of the end-to-end model can represent the overall clarity of the target image.

[0163] During training, the end-to-end model can be trained using labeled image samples. The image samples are input into the end-to-end model to obtain the clarity level prediction results of the image samples. The difference between the clarity level prediction results and the true value of the clarity level is used to train the end-to-end model as a whole. On the one hand, the training efficiency can be improved. On the other hand, the accuracy of the output results of the end-to-end model will be higher than the accuracy of the results obtained by training multiple models separately and splicing them together.

[0164] In a possible implementation manner, when the target image is obtained by extracting a video frame from the target video, the method further includes:

[0165] The definition level of the target video is determined according to the definition level of each target image.

[0166] The solution of the present application can also be applied to determine the clarity of a video. For example, multiple frames of target images can be extracted from a target video. The extraction method can be random extraction, fixed-interval frame extraction, etc.

[0167] By using the above scheme of the present application, the clarity of each extracted target image frame can be determined. By using the clarity level of each target image frame, the clarity level of the target video can be obtained. For example, the probabilities of four different clarity levels corresponding to each target image frame can be counted. For example, for the first target frame image, the probabilities of the four different clarity levels are A1%, B1%, C1%, and D1%, respectively. For the Nth target frame image, the probabilities of the four different clarity levels are A1%, B1%, C1%, and D1%, respectively. N %、B N %、C N %、D N %.

[0168] The sum of the probabilities of all target images in the target video at each definition level can be calculated, and the definition level corresponding to the maximum value of the sum of probabilities can be determined as the definition level of the target video.

[0169] In addition, the clarity level of each frame of the target image can be counted, and the clarity level with the most occurrences can be determined as the clarity level of the target video. Alternatively, the clarity level can be divided into values ​​1 to 4, and the average clarity level of each frame of the target image can be calculated. The level closest to the average value is used as the clarity level of the target video.

[0170] The definition level determination of the target video can be applied to the server side of the video application. For example, after the user uploads the video, the definition of the video uploaded by the user can be determined. For videos with lower definition, the video can be not received or deleted.

[0171] In addition, for the received videos, the corresponding transcoding level can be used for saving according to the different definition levels. For videos with higher definition levels, high-definition level transcoding can be used. For videos with lower definition levels, low-definition level transcoding can be used to reduce storage usage.

[0172] like Figure 6 As shown, the method for determining the clarity of the embodiment of the present invention includes the following steps:

[0173] S61: scaling the target image to a predetermined resolution.

[0174] The target image includes an image obtained by frame extraction from the target video.

[0175] S62: Extract features of the target image.

[0176] The feature extraction model can be used to extract the features of the target image. The feature extraction model can include multiple convolutional layers and a normalization layer connected to the last convolutional layer. Multiple convolutional layers are used to perform convolution operations on the target image to obtain a feature matrix. The feature matrix is ​​the matrix output by the last convolutional layer and processed by the normalization layer. The feature matrix is ​​used to characterize the features of the target image and can be expressed as H*W*C. Among them, H can represent the length of the feature matrix, W can represent the width of the feature matrix, and C can represent the number of channels (number of layers) of the feature matrix.

[0177] S63: Determine different clarity level probabilities for each area of ​​the target image according to the characteristics of the target image.

[0178] This step can be implemented using a clarity level recognition model.

[0179] S64: Determine the weight of each region of the target image according to the characteristics of the target image.

[0180] This step can be implemented using a weight recognition model.

[0181] S65: Associating different clarity level probabilities of each region with weights.

[0182] S66: Perform dimensionality reduction processing on the probabilities and weights of different clarity levels of each area in step S65 to obtain a dimensionality reduction processing result.

[0183] Dimensionality reduction processing may include global maximum pooling, global average pooling, etc.

[0184] S67: Calculate the weight sum of each region.

[0185] S68: Using the dimensionality reduction processing result and the weight sum, the probabilities of different clarity levels of the target image are obtained.

[0186] By using the dimensionality reduction processing results and the weight sum, the importance of each area of ​​the target image and the probability of different clarity levels can be obtained.

[0187] S69: Obtain the clarity level of the target image using the different clarity level probabilities of the target image.

[0188] By calculating the mean of the different clarity level probabilities of each area of ​​the target image according to the weights, the clarity level probability of the target image can be obtained, and then the clarity level of the target image can be obtained.

[0189] The aforementioned feature extraction model, clarity level recognition model, weight recognition model and classification model may be end-to-end models.

[0190] Exemplary Media

[0191] After introducing the method of the exemplary embodiment of the present invention, next, refer to Figure 7 A medium according to an exemplary embodiment of the present invention is described.

[0192] In some possible embodiments, each aspect of the present invention may also be implemented as a computer-readable medium having a program stored thereon, which, when executed by a processor, is used to implement the steps of the method for determining clarity according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0193] Specifically, the processor is used to implement the following steps when executing the program:

[0194] Extract the features of the target image;

[0195] Determine the clarity level probability of each area of ​​the target image and the weight of each area according to the feature;

[0196] The clarity level of the target image is determined by using the clarity level probability of each region and the weight of each region.

[0197] It should be noted that the above-mentioned medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0198] like Figure 7 As shown, a medium 70 according to an embodiment of the present invention is described, which can be a portable compact disk read-only memory (CD-ROM) and include a program and can be run on a device. However, the present invention is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device.

[0199] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to: electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0200] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0201] Exemplary Devices

[0202] After introducing the medium of the exemplary embodiment of the present invention, next, reference is made to Figure 8 A device according to an exemplary embodiment of the present invention is described.

[0203] like Figure 8 As shown, the clarity determination device of the embodiment of the present invention may include:

[0204] A feature extraction module 81 is used to extract features of a target image;

[0205] An analysis module 82, used to determine the clarity level probability of each area of ​​the target image and the weight of each area according to the feature;

[0206] The clarity level determination module 83 is used to determine the clarity level of the target image by using the clarity level probability of each region and the weight of each region.

[0207] In a possible implementation, the clarity level determination module 83 may further include:

[0208] A dimensionality reduction processing submodule 831 is used to perform dimensionality reduction processing on the clarity level probability and weight of each region to obtain a dimensionality reduction processing result;

[0209] A weight sum calculation submodule 832, used to calculate the weight sum of each region;

[0210] The definition level determination execution submodule 833 is used to determine the definition level of the target image using the dimension reduction processing result and the weight sum.

[0211] In a possible implementation, the clarity level determination execution submodule 833 may further include:

[0212] A ratio calculation unit 8331 is used to calculate the ratio of the dimension reduction processing result to the weight sum;

[0213] The clarity level determination execution unit 8332 is used to obtain the clarity level of the target image based on the ratio using a classification model.

[0214] In a possible implementation, the feature extraction module 81 may further include:

[0215] The scaling processing submodule 811 is used to perform scaling processing on the target image to obtain a target image with a predetermined resolution;

[0216] The feature extraction execution submodule 812 is used to perform feature extraction on the target image of the predetermined resolution using a feature extraction model to obtain a feature matrix of the target image of the predetermined resolution; and use the feature matrix as the feature of the target image.

[0217] In a possible implementation, the analysis module 82 may further include:

[0218] The clarity level probability determination submodule 821 is used to obtain different clarity level probabilities corresponding to each area of ​​the target image according to the features by using a clarity level recognition model;

[0219] A weight determination submodule 822 is used to obtain a weight corresponding to each region of the target image according to the feature using a weight recognition model;

[0220] The association submodule 823 is used to associate different clarity level probabilities of each region with weights.

[0221] In one possible implementation,

[0222] The feature extraction module 81 uses a feature extraction model to extract features of the target image;

[0223] The analysis module 82 uses a clarity level recognition model to obtain different clarity level probabilities corresponding to each area of ​​the target image;

[0224] The analysis module 82 also uses a weight recognition model to obtain a weight corresponding to each area of ​​the target image;

[0225] The definition level determination module 83 determines the definition level of the target image using a classification model;

[0226] The feature extraction model, the clarity level recognition model, the weight recognition model and the classification model constitute an end-to-end model, the input of the end-to-end model is a target image, and the output is the clarity level of the target image.

[0227] In a possible implementation manner, when the target image is obtained by extracting a video frame from a target video, the method further includes:

[0228] The target video definition level determination module 84 is used to determine the definition level of the target video according to the definition levels of each of the target images.

[0229] Exemplary Computing Devices

[0230] After introducing the method, medium and apparatus of the exemplary embodiments of the present invention, next, reference is made to Fig. 9 A computing device according to an exemplary embodiment of the present invention is described.

[0231] Those skilled in the art will appreciate that each aspect of the present invention may be implemented as a system, method or program product. Therefore, each aspect of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0232] In some possible implementations, a computing device according to an implementation of the present invention may include at least one processing unit and at least one storage unit. The storage unit stores a program code, and when the program code is executed by the processing unit, the processing unit executes the steps of the method for determining clarity according to various exemplary implementations of the present invention described in the above “Exemplary Method” section of this specification.

[0233] Refer to the following Fig. 9 1 and 2 to describe a computing device 90 according to this embodiment of the present invention. Fig. 9 The computing device 90 shown is only an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present invention.

[0234] like Fig. 9 As shown, the computing device 90 is in the form of a general computing device. The components of the computing device 90 may include but are not limited to: at least one processing unit 901, at least one storage unit 902, and a bus 903 connecting different system components (including the processing unit 901 and the storage unit 902).

[0235] The bus 903 includes a data bus, a control bus, and an address bus.

[0236] The storage unit 902 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 9021 and / or a cache memory 9022 , and may further include a readable medium in the form of a non-volatile memory, such as a read-only memory (ROM) 9023 .

[0237] The storage unit 902 may also include a program / utility 9025 having a set (at least one) of program modules 9024, such program modules 9024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0238] The computing device 90 may also communicate with one or more external devices 904 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 905. Furthermore, the computing device 90 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 906. Fig. 9 As shown, the network adapter 906 communicates with other modules of the computing device 90 via the bus 903. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the computing device 90, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0239] It should be noted that, although several units / modules or sub-units / sub-modules of the clarity determination device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above may be embodied in one unit / module. Conversely, the features and functions of one unit / module described above may be further divided to be embodied by multiple units / modules.

[0240] In addition, although the operations of the method of the present invention are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0241] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A method for determining clarity, characterized in that: include: Extract the features of the target image; Determine the clarity level probability of each area of ​​the target image and the weight of each area according to the feature; Determine the definition level of the target image by using the definition level probability of each region and the weight of each region; The clarity level is multi-classified, and the clarity level of the target image is determined by using the clarity level probability of each region and the weight of each region, including: Calculate the probability mean corresponding to each type of clarity level of the effective area, and use the clarity level corresponding to the maximum value as the clarity level of the target image, wherein the effective area is an area with a weight higher than a preset threshold; Wherein, determining the clarity level probability of each area of ​​the target image and the weight of each area according to the feature includes: Using a clarity level recognition model, obtaining different clarity level probabilities corresponding to each area of ​​the target image according to the features; Using a weight recognition model, a weight corresponding to each area of ​​the target image is obtained according to the features; The probability of different sharpness levels for each region is associated with a weight.

2. The method according to claim 1, characterized in that The step of extracting the features of the target image comprises: Scaling the target image to obtain a target image with a predetermined resolution; Using a feature extraction model, feature extraction is performed on the target image of the predetermined resolution to obtain a feature matrix of the target image of the predetermined resolution; and the feature matrix is ​​used as a feature of the target image.

3. The method according to claim 1, characterized in that Using the feature extraction model, extract the features of the target image; Using a clarity level recognition model, obtaining different clarity level probabilities corresponding to each area of ​​the target image; Using a weight recognition model, a weight corresponding to each area of ​​the target image is obtained; Determining the clarity level of the target image using a classification model; The feature extraction model, the clarity level recognition model, the weight recognition model and the classification model constitute an end-to-end model, the input of the end-to-end model is a target image, and the output is the clarity level of the target image.

4. The method according to any one of claims 1 to 3, characterized in that: In the case where the target image is obtained by extracting a video frame from a target video, the method further includes: The definition level of the target video is determined according to the definition level of each target image.

5. A device for determining clarity, characterized in that: include: A feature extraction module is used to extract the features of the target image; An analysis module, used to determine the clarity level probability of each area of ​​the target image and the weight of each area according to the features; A definition level determination module, used to determine the definition level of the target image by using the definition level probability of each region and the weight of each region; Wherein, the clarity level is of multiple categories, and the clarity level determination module is further used to calculate the probability mean corresponding to each type of clarity level of the effective area, and use the clarity level corresponding to the maximum value as the clarity level of the target image, and the effective area is an area with a weight higher than a preset threshold; The analysis module comprises: A clarity level probability determination submodule, used to obtain different clarity level probabilities corresponding to each area of ​​the target image according to the features by using a clarity level recognition model; A weight determination submodule, used to obtain a weight corresponding to each area of ​​the target image according to the features by using a weight recognition model; The association submodule is used to associate the different clarity level probabilities of each region with weights.

6. The device according to claim 5, characterized in that The feature extraction module comprises: A scaling processing submodule, used for performing scaling processing on the target image to obtain a target image with a predetermined resolution; The feature extraction execution submodule is used to use the feature extraction model to perform feature extraction on the target image of the predetermined resolution to obtain a feature matrix of the target image of the predetermined resolution; and use the feature matrix as the feature of the target image.

7. The device according to claim 5, characterized in that The feature extraction module uses a feature extraction model to extract features of the target image; The analysis module uses a clarity level recognition model to obtain different clarity level probabilities corresponding to each area of ​​the target image; The analysis module also uses a weight recognition model to obtain a weight corresponding to each area of ​​the target image; The definition level determination module determines the definition level of the target image using a classification model; The feature extraction model, the clarity level recognition model, the weight recognition model and the classification model constitute an end-to-end model, the input of the end-to-end model is a target image, and the output is the clarity level of the target image.

8. The device according to any one of claims 5 to 7, characterized in that: In the case where the target image is obtained by extracting a video frame from a target video, the method further includes: The target video definition level determination module is used to determine the definition level of the target video according to the definition level of each target image.

9. A medium storing a computer program, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

10. A computing device comprising: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 4.

Images