Image sharpness evaluation method

Abstract

An embodiment of the present invention provides an image sharpness evaluation method, and the method includes: obtaining an image to be evaluated; inputting the image to be evaluated into a pre-trained second sharpness evaluation network, where the second sharpness evaluation network includes: a convolution module, and the convolution module includes convolution kernels of multiple different sizes, a normalization channel, and an activation function. The convolution kernels of multiple different sizes perform convolution processing on the input of the convolution module to obtain a convolution kernel processing amount. The normalization channel takes the convolution kernel processing amount as an input and performs normalization processing to obtain a normalization channel processing amount. The activation function takes the normalization channel processing amount as an input and performs activation processing to obtain an activation processing amount. The convolution module combines the activation processing amounts to obtain a convolution processing amount; the second sharpness evaluation network outputs a sharpness evaluation value of the image to be evaluated. Thus, an accurate evaluation of the sharpness of the image is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image sharpness evaluation method. Background Art

[0002] When a camera performs autofocus, there are mainly two situations: active focus and passive focus. In related technologies, contrast autofocus is usually used for passive focus, and this autofocus method depends on the accuracy of calculating the sharpness evaluation value of an image. Summary of the Invention

[0003] The purpose of the embodiments of the present invention is to provide an image sharpness evaluation method to accurately evaluate the sharpness of an image. The specific technical solutions are as follows:

[0004] In the first aspect of the present invention, an image sharpness evaluation method is provided. The method includes:

[0005] Obtain an image to be evaluated;

[0006] Input the image to be evaluated into a pre-trained second sharpness evaluation network. The second sharpness evaluation network includes: a convolution module. Among them, the convolution module includes a plurality of convolution kernels of different sizes, a normalization channel, and an activation function. The plurality of convolution kernels of different sizes perform convolution processing on the input of the convolution module to obtain a convolution kernel processing amount. The normalization channel uses the convolution kernel processing amount as an input for normalization processing to obtain a normalization channel processing amount. The activation function uses the normalization channel processing amount as an input for activation processing to obtain an activation processing amount. The convolution module uses the activation processing amount as a convolution processing amount;

[0007] The second sharpness evaluation network outputs the sharpness evaluation value of the image to be evaluated.

[0008] In the second aspect of the present invention, an image sharpness evaluation method is provided. The method includes:

[0009] Obtain at least one image to be evaluated;

[0010] Process the at least one image to be evaluated through a convolution pooling unit in a pre-trained second sharpness evaluation network to obtain a convolution pooling feature map. Among them, the convolution pooling unit includes a convolution module and a pooling module, and is used to perform convolution processing and pooling processing on the input;

[0011] Perform global pooling on the convolution pooling feature map through a global pooling module in the second sharpness evaluation network to obtain a global pooling feature map;

[0012] The global pooling feature map is weighted by a fully connected module in the second sharpness evaluation network to obtain the sharpness evaluation value of the at least one image to be evaluated.

[0013] Advantages of the embodiments of the present invention:

[0014] A sharpness evaluation method, device and electronic device provided by an embodiment of the present invention can input an image to be evaluated into a trained second sharpness evaluation network to obtain the sharpness evaluation value of the image to be evaluated. The second sharpness evaluation network includes: a convolution module, and the convolution module includes a plurality of convolution kernels with different sizes, obtaining rich image features of the image to be evaluated from the input of the convolution module, through the normalization processing of the normalization channel and the activation processing of the activation function, the convolution module takes the obtained activation processing amount as the combined convolution processing amount including rich features of the image to be evaluated, so that the rich image features obtained by the convolution module to be evaluated are transmitted to the next layer module to a large extent, and further enables the second sharpness evaluation network to comprehensively consider various factors affecting the sharpness of the image to be evaluated to determine the sharpness evaluation value of the image to be evaluated, realizing an accurate evaluation of the sharpness of the image.

[0015] Of course, it is not necessary for any product or method implementing the present invention to achieve all the above advantages simultaneously. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0017] Figure 1 It is a schematic flowchart of a sharpness evaluation method provided by an embodiment of the present invention;

[0018] Figure 2 It is a schematic flowchart of another sharpness evaluation method provided by an embodiment of the present invention;

[0019] Figure 3 It is a schematic flowchart of another sharpness evaluation method provided by an embodiment of the present invention;

[0020] Figure 4 It is a schematic flowchart of a training method for a second sharpness evaluation network provided by an embodiment of the present invention;

[0021] Figure 5 It is a schematic flowchart of another training method for a second sharpness evaluation network provided by an embodiment of the present invention;

[0022] Figure 6 It is a schematic flowchart of a training method for another second sharpness evaluation network provided by an embodiment of the present invention;

[0023] Figure 7 It is a schematic structural diagram of a second sharpness evaluation network provided by an embodiment of the present invention;

[0024] Figure 8 It is a schematic flowchart of an image sharpness evaluation method provided by an embodiment of the present invention; Detailed implementation manners

[0025] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art based on this application belong to the scope of protection of the present invention.

[0026] The focusing process of the conventional contrast focusing is generally as follows: obtain an image and calculate the sharpness evaluation value; then move the focusing lens group in any direction, read the image again, calculate the sharpness evaluation value, compare the sharpness evaluation values of the images at the two positions before and after, and adjust the moving direction of the lens group to the direction where the sharpness evaluation value is larger; continue to move the lens group, read the image, calculate the sharpness evaluation value, compare the sharpness evaluation values of the two positions before and after, until the sharpness evaluation value of the image shows a decrease; adjust the moving direction of the lens group so that it is adjusted to the position where the sharpness evaluation value of the image appears at a peak, and the focusing is completed. It can be seen that this focusing method depends on the accuracy of the sharpness evaluation value of the image, and the ideal image sharpness evaluation curve is a bell-shaped curve similar to the normal distribution.

[0027] The existing sharpness evaluation methods can be basically divided into three types, namely, gray gradient function, frequency domain function, and information entropy function. Among them, the gray gradient function utilizes the characteristic that the gray difference of a clearer image is larger, and measures whether the image is clear by calculating the gray gradient value of the image. The frequency domain function utilizes the fact that there are more high-frequency signals in a clearer image. The image is processed by a high-pass filter to retain the high-frequency components, and the amount of high-frequency components is used as the basis for measuring whether the image is clear. The information entropy function utilizes the characteristic that a clearer image is more ordered and has a smaller entropy compared to a blurred image, and evaluates whether the image is clear by calculating the entropy of the image.

[0028] However, in the case of a relatively good image shooting environment, such as an environment with sufficient light and rich details, the clarity evaluation value of the image is relatively accurate. However, when the image shooting environment is poor, for example, when there are few details, the light is weak, or there is a lot of noise in the image, using the above methods to evaluate the clarity of the image, the clarity evaluation curve of the image will have poor monotonicity and even no peak. For another example, in a scene with point light sources, the light spot of the image becomes larger, and affected by factors such as the aperture, the edge of the light spot will become sharper. However, the clarity evaluation value of such an actually blurred image calculated by the gray gradient, frequency domain function, and information entropy function is even larger, and there are two peaks on the clarity evaluation curve, so the clarity of the image cannot be accurately evaluated. Based on this, the embodiments of the present invention provide a clarity evaluation method, as Figure 1 shown, which may include:

[0029] S101, obtain the image to be evaluated.

[0030] S102, input the image to be evaluated into the trained second clarity evaluation network.

[0031] S103, the second clarity evaluation network outputs the clarity evaluation value of the image to be evaluated.

[0032] By selecting this embodiment, the image to be evaluated can be input into the trained second clarity evaluation network to obtain the clarity evaluation value of the image to be evaluated. The second clarity evaluation network includes: a convolution module, and the convolution module includes a plurality of convolution kernels with different sizes. Rich image features of the image to be evaluated are obtained from the input of the convolution module. After the normalization processing of the normalization channel and the activation processing of the activation function, the convolution module takes the obtained activation processing amount as the combined convolution processing amount containing rich features of the image to be evaluated, so that the rich image features to be evaluated obtained by the convolution module are transmitted to the next layer module to a greater extent. Furthermore, the second clarity evaluation network can comprehensively consider various factors affecting the clarity of the image to be evaluated to determine the clarity evaluation value of the image to be evaluated, realizing the accurate evaluation of the clarity of the image.

[0033] Among them, in S101, the image to be evaluated is any image. And the number of images to be evaluated can be one or multiple. For the convenience of description in the following, only the case where the image to be evaluated is one image is taken as an example for illustration.

[0034] In S102, the second sharpness evaluation network includes: a convolution module. The convolution module includes multiple convolution kernels of different sizes, a normalization channel, and an activation function. The multiple convolution kernels of different sizes perform convolution processing on the input of the convolution module to obtain convolution kernel processing amounts. The normalization channel uses the convolution kernel processing amounts as inputs for normalization processing to obtain normalization channel processing amounts. The activation function uses the normalization channel processing amounts as inputs for activation processing to obtain activation processing amounts. The convolution module uses the activation processing amounts as convolution processing amounts.

[0035] The convolution module uses the output of the previous module as the input, and enables multiple convolution kernels of different sizes to perform convolution processing on it to obtain convolution kernel processing amounts. Thus, while obtaining multiple different features of the image to be evaluated, after a limited number of module feature extractions in the second sharpness evaluation network, it can cover enough image features of the image to be evaluated as soon as possible, reducing the complexity of the network. The multiple convolution kernel processing amounts are respectively normalized using the normalization channel according to the different obtained features to obtain multiple normalization channel processing amounts, facilitating subsequent processing of image features.

[0036] In a possible embodiment, the normalization method used by the normalization channel can be mean-variance normalization. Using this method can avoid the problems of gradient disappearance and output saturation that often occur during the training of multi-layer networks in the process of training the second sharpness evaluation network. In other possible embodiments, the normalization method used by the normalization channel can also be other normalization methods other than mean-variance normalization, such as min-max normalization.

[0037] The activation function respectively performs activation processing on the multiple normalization channel processing amounts obtained through normalization processing, adding non-linear factors to the second sharpness evaluation network and improving the expression ability of the second sharpness evaluation network to obtain multiple activation processing amounts. The convolution module uses the multiple activation processing amounts as convolution processing amounts. It can be understood that the convolution processing amounts contain multiple activation processing amounts, and the multiple activation processing amounts are obtained by activating the normalized multiple convolution kernel processing amounts using the activation function. The multiple activation processing amounts contain multiple features of the image to be evaluated. Therefore, the convolution processing amounts contain various image features of the image to be evaluated. Using the convolution processing amounts as the output of the convolution module, that is, various influencing factors affecting image sharpness in the image to be evaluated are obtained through the convolution module. The convolution processing amounts will be used as the input of the subsequent module after the convolution module, enabling the rich image features obtained through the convolution module to be transmitted to the lower layer module to a greater extent for further processing.

[0038] In a possible embodiment, the activation function can be the tanh (hyperbolic tangent) function. As mentioned before, since the present invention uses multiple convolution kernels of different sizes to extract the image features of the image to be evaluated, reducing the complexity of the network, the depth of the second sharpness evaluation network will not be very deep. Coupled with the normalization of the convolution kernel processing volume, using the tanh function at this time will not cause a serious problem of gradient disappearance during network training. And because the output of tanh is between -1 and 1 and symmetric about the origin, it is more suitable for the sharpness evaluation problem of images. Therefore, using the tanh function as the activation function of the convolution module can further improve the accuracy of the second sharpness evaluation network for image sharpness evaluation without affecting network training.

[0039] In a possible embodiment, the second sharpness evaluation network may include multiple convolution modules. It can be understood that through multiple convolution modules, the image features of the image to be evaluated can be extracted multiple times, so as to ensure that the second sharpness evaluation network can obtain rich enough image features from the image to be evaluated. Further, the second sharpness evaluation network can comprehensively consider various factors affecting the sharpness of the image to be evaluated to determine the sharpness evaluation value of the image to be evaluated, realizing an accurate evaluation of the sharpness of the image.

[0040] And the number of convolution modules included in the second sharpness evaluation network can be set by the user according to actual experience or requirements. The more convolution modules included in the second sharpness evaluation network, the more accurate the evaluation of the image sharpness. While the fewer convolution modules included in the second sharpness evaluation network, the higher the efficiency of the image sharpness evaluation.

[0041] In the actual evaluation of image sharpness, since the shooting methods and image parameters of different images may be different, and the parameters of different images to be evaluated will also vary greatly. Based on this, the present invention also provides a sharpness evaluation method. The second sharpness evaluation network further includes: a normalization module; the normalization module normalizes the image to be evaluated input into the second sharpness evaluation network to obtain a normalized processing volume. Among them, the normalization process is mean-variance normalization.

[0042] After the image to be evaluated is input into the second sharpness evaluation network, the normalization processing module can perform normalization processing on the image to be evaluated, converting different images to be evaluated into corresponding unique standard forms. The standard form images have invariant characteristics with respect to affine transformations such as translation, rotation, and scaling, enabling subsequent modules to efficiently process the images to be evaluated. The present invention uses the mean-variance normalization processing method to perform normalization processing on the image to be evaluated, which can remove the interference of the overall brightness value of the image on the sharpness evaluation value of the image to be evaluated while ensuring the adaptability of the second sharpness evaluation network to different depth images. It can be understood that when evaluating the sharpness of an image, we are not interested in the illuminance of the image but more concerned about its content, that is, the clarity of the fine details and their boundaries in the image. The use of mean-variance normalization processing can effectively remove the interference of the overall brightness value of the image.

[0043] Selecting this embodiment can improve the efficiency of the second sharpness evaluation network in evaluating the sharpness of the image to be evaluated because normalization processing is performed on the image to be evaluated. Moreover, the use of the mean-variance normalization processing method ensures the adaptability of the second sharpness evaluation network to different depth images and removes the interference of the overall brightness value of the image to be evaluated, further improving the applicability of the second sharpness evaluation network and the accuracy of sharpness evaluation.

[0044] In a possible embodiment, the present invention also provides a sharpness evaluation method. The second sharpness evaluation network further includes: a first pooling module and a global pooling module. The first pooling module includes: a max-pooling sub-module and an average-pooling sub-module, as Figure 2 shown, the method includes:

[0045] S201, the max-pooling sub-module and the average-pooling sub-module perform pooling processing on the convolution processing amount as input respectively to obtain a max-pooling amount and an average-pooling amount.

[0046] S202, the first pooling module uses the max-pooling amount and the average-pooling amount as the first pooling amount.

[0047] S203, the global pooling module performs global pooling processing on the first pooling amount as input to obtain a global pooling amount.

[0048] Among them, in S201, the convolution processing amount can be used as the input of the first pooling module, and the first pooling module includes a max-pooling sub-module and an average-pooling sub-module. Therefore, the max-pooling sub-module and the average-pooling sub-module can perform max-pooling and average-pooling on the convolution processing amount output by the convolution module respectively. Setting a pooling module after the convolution module reduces the data processing amount of the subsequent module of the pooling module on the one hand, and can also prevent overfitting of the second sharpness evaluation network on the other hand. The max-pooling adopted by the present invention can well retain the texture features of the image, while the average-pooling can well retain the background of the image. Therefore, the max-pooling amount can reflect the fine details of the image to be processed, and the average-pooling amount reduces the noise interference of the image to be processed.

[0049] In S202, the present invention adopts two different pooling methods in the first pooling module. After obtaining different pooling amounts, they are used as the first pooling amount. It can be understood that the first pooling amount can reflect the features of different sizes of the image to be evaluated on the one hand. On the other hand, since the max-pooling sub-module and the average-pooling sub-module perform pooling on the convolution processing amount respectively, the first pooling amount including the max-pooling amount and the average-pooling amount can reflect the fine details of the image to be processed and reduce the noise interference of the image to be processed.

[0050] In S203, the second sharpness evaluation network uses a global pooling module to perform global pooling on the first pooling amount. Global pooling can greatly reduce the network parameters of the second sharpness evaluation network, thereby reducing the overfitting phenomenon of the second sharpness evaluation network. It can be understood that after the image to be evaluated is input into the second sharpness evaluation network, multiple features can be obtained from the image to be evaluated through multiple convolution kernels of the convolution module, that is, multiple feature maps reflecting multiple features of the image to be evaluated. Even if different subsequent processes such as normalization, activation, merging, and pooling are performed on the multiple different feature maps, the multiple feature maps are not screened out. The number of features of the image to be processed included in the first pooling amount input in the global pooling remains unchanged, and this number of features is the number of convolution kernels in the convolution module. The global pooling module can reduce the dimension of the multiple feature maps reflecting multiple features of the image to be evaluated and map the multiple feature maps into multiple single values. Therefore, when using the global pooling amount obtained through the global pooling module as the input of the subsequent module, the limitation of the second sharpness evaluation network on the resolution of the image to be evaluated can be eliminated, and the compatibility of the second sharpness evaluation network can be improved.

[0051] Selecting this embodiment, the first pooling module can extract the features and fine details of different sizes of the image to be processed, and can also suppress the noise interference of the image to be processed. The global pooling module can further prevent overfitting of the second sharpness evaluation network and improve the compatibility of the second sharpness evaluation network. Based on this, the compatibility of the second sharpness network and the accuracy of sharpness evaluation are improved.

[0052] In a possible embodiment, the second sharpness evaluation network may include a plurality of convolutional modules and a plurality of first pooling modules. A first pooling module is arranged between every two adjacent convolutional modules. As described above, the plurality of convolutional modules can extract richer image features of the image to be processed, while the first pooling module can prevent overfitting of the second sharpness evaluation network, extract features of different sizes and fine details of the image to be evaluated, and reduce the noise interference of the image to be processed. Therefore, arranging a first pooling module between every two convolutional modules can further enable the second sharpness evaluation network to extract more important influencing factors of the image to be evaluated, eliminate irrelevant interference, and improve the accuracy of sharpness evaluation of the second sharpness evaluation network.

[0053] In a possible embodiment, the number of convolutional modules and first pooling modules in the present invention can also be set according to requirements. It can be understood that the present invention adopts a modular design, and the number of convolutional modules and first pooling modules can be adjusted, and the number of convolutional kernels in each convolutional module can also be configured to adjust the network depth and scale of the second sharpness evaluation network.

[0054] Therefore, by selecting this embodiment, the network training efficiency and flexibility are improved in the process of actually training the second sharpness evaluation network.

[0055] As an example, the network depth of the second sharpness evaluation network can be set to 5 - 6 layers, and the number of convolutional kernels in the convolutional module can be set to 30.

[0056] In a possible embodiment, the present invention also provides a sharpness evaluation method. The second sharpness evaluation network further includes: a fully connected module. Refer to Figure 3 , and the method includes:

[0057] S301, the fully connected module takes the global pooling amount as input and performs weighted accumulation to obtain an accumulated feature.

[0058] S302, the sigmoid (a non-linear activation function) activation function sub-module maps the accumulated feature to obtain a mapped amount.

[0059] S303, takes the mapped amount as the sharpness evaluation value of the image to be evaluated.

[0060] Among them, in S301, the fully connected module includes: a sigmoid activation function sub-module. The fully connected module can perform weighted accumulation on all features obtained from its previous layer module, perform feature fusion, and obtain an accumulated feature.

[0061] In S302, since the output of the sigmoid activation function is between 0 and 1, when the sigmoid activation function sub-module included in the fully connected module maps the accumulated features, it can enhance the non-linear expression ability of the second sharpness evaluation network while mapping the accumulated features into a mapping quantity with a value range between 0 and 1.

[0062] In S303, taking the mapping quantity with a value range between 0 and 1 as the sharpness evaluation value of the image to be evaluated can facilitate the evaluation of the image sharpness, and further facilitate the subsequent use of the sharpness evaluation value of the image to be evaluated. For example, automatic focusing of the camera can be achieved by comparing the sharpness evaluation values.

[0063] By selecting this embodiment, the fully connected module can fuse each image feature obtained from the global pooling quantity, and output a sharpness evaluation value convenient for use for the image to be evaluated through the sigmoid activation function. While ensuring the accuracy of the sharpness evaluation of the second sharpness evaluation network, it is convenient to perform subsequent processing based on the determined sharpness evaluation value, improving the applicability of the sharpness evaluation method.

[0064] In the related art, since the sharpness evaluation value of the image is not accurate, during the training process of the network for evaluating the image sharpness, it is impossible to use the images with the marked sharpness evaluation values to train the network. Therefore, it is also impossible to train the network using the supervised learning method. Based on this, the present invention also provides a training method for the second sharpness evaluation network, as Figure 4 shown, the method includes:

[0065] S401, input a set of training images into the initial sharpness evaluation network to obtain the initial sharpness evaluation values of each image in the training images.

[0066] S402, randomly select a first image from the training images and determine the first initial sharpness evaluation value corresponding to the first image.

[0067] S403, judge the magnitude relationship between the first initial sharpness evaluation value and the second initial sharpness evaluation value and the third initial sharpness evaluation value.

[0068] S404, determine the penalty amount of the initial sharpness evaluation network according to the magnitude relationship.

[0069] S405, adjust the initial sharpness evaluation network according to the penalty amount to obtain the second sharpness evaluation network.

[0070] Among them, in S401, the training images are a set of images arranged in the order from clear to blurred in terms of clarity. It can be understood that the training images in the present invention are a set of images that can be directly distinguished as clear and blurred by the naked eye without marked clarity evaluation values. They can be a set of images taken by continuously adjusting the focal length of a camera, or images obtained from the network. These training images can be arranged in the order from blurred to clear and input into the untrained initial clarity evaluation network in sequence, so that the initial clarity training network can output the corresponding initial clarity evaluation values in the order from clear to blurred of the training images.

[0071] In S402, a random image is selected from the training images as the first image. Since the initial clarity evaluation network has determined the initial clarity evaluation values of all the training images, after selecting the first image, the first initial clarity evaluation value corresponding to the first image can be determined.

[0072] In S403, the second initial clarity evaluation value is the maximum value of the initial clarity evaluation values of the training images sorted before the first image in the training images, and the third initial clarity evaluation value is the minimum value of the initial clarity evaluation values of the images sorted after the first image in the training images.

[0073] According to the sorting of the training images and their corresponding initial clarity evaluation values, after determining the first image, the sorting position of the first image can be determined. Furthermore, the maximum value of the initial clarity evaluation values before the first image in the set of training images from blurred to clear and the minimum value of the initial clarity evaluation values after the first image in the set of training images from blurred to clear can be determined, and used as the second initial clarity evaluation value and the third initial clarity evaluation value respectively. It can be understood that the images sorted before the first image in the training images are more blurred than the first image. Therefore, normally, their corresponding initial clarity evaluation values should be less than the first initial clarity evaluation value, while the images sorted after the first image in the training images are clearer than the first image, and their corresponding initial clarity evaluation values should be greater than the first initial clarity evaluation value. Based on this, the present invention selects the maximum value of the initial clarity evaluation values of the training images sorted before the first image as the second initial clarity evaluation value, and selects the minimum value of the initial clarity evaluation values of the training images sorted after the first image as the third initial clarity evaluation value, and compares them with the first initial clarity evaluation value to judge their size relationship, which can clearly reflect the evaluation ability of the initial clarity evaluation network for the image to be evaluated.

[0074] In S404, according to the magnitude relationship between the first initial clarity evaluation value and the second and third initial clarity evaluation values, the penalty amount corresponding to the initial clarity evaluation network is determined. The penalty amount can reflect the quality of the first initial clarity evaluation value, and further reflect the quality of the initial clarity evaluation network. It can be understood that if the first initial clarity evaluation value is less than the second initial clarity evaluation value or the first initial clarity evaluation value is greater than the third initial clarity evaluation value, and in fact, the first image is clearer than the image corresponding to the second initial clarity evaluation value and blurrier than the image corresponding to the third initial clarity evaluation value, then at this time, the initial clarity evaluation network fails to accurately determine the clarity evaluation value of the first image and does not reflect the true clarity level of the first image. At this time, the corresponding penalty amount can be obtained. The penalty amount is inversely proportional to the accuracy of the first initial clarity evaluation value, that is, the more accurate the first initial clarity evaluation value is, the smaller the absolute value of its penalty amount.

[0075] In S405, it can be understood that the penalty amount indirectly reflects the clarity evaluation accuracy of the initial clarity evaluation network. Further, it reflects the quality of the network parameters of the initial clarity evaluation network. Therefore, after obtaining the penalty amount, the network parameters of the initial clarity evaluation network can be adjusted according to the penalty amount to obtain a second clarity evaluation network that can accurately evaluate the clarity of the image.

[0076] Selecting this embodiment can achieve the initial clarity evaluation network without obtaining labeled training images, and obtain a second clarity evaluation network that can accurately evaluate the clarity of the image, thereby improving the network training efficiency and the accuracy of the clarity evaluation of the trained network.

[0077] In actual implementation, different methods can be used to determine the penalty amount. In one possible embodiment, the penalty amount of the initial clarity evaluation network is determined according to the magnitude relationship. As Figure 5 shown, it may include:

[0078] S501, determine the first difference and the second difference.

[0079] S502, use the sum of the first difference and the second difference as the penalty amount of the initial clarity evaluation network.

[0080] Among them, in S501, determining the first difference includes: if the first initial clarity evaluation value is greater than the second initial clarity evaluation value, determining the first difference to be 0; if the first initial clarity evaluation value is less than the second initial clarity evaluation value, determining the first difference to be the difference obtained by subtracting the second initial clarity evaluation value from the first initial clarity evaluation value; determining the second difference includes: if the first initial clarity evaluation value is less than the third initial clarity evaluation value, determining the second difference to be 0; if the first initial clarity evaluation value is greater than the third initial clarity evaluation value, determining the second difference to be the difference obtained by subtracting the third initial clarity evaluation value from the first initial clarity evaluation value.

[0081] When determining the first difference, there are two cases. If the first initial clarity evaluation value is greater than the second initial clarity evaluation value, the first difference is determined to be 0. If the first initial clarity evaluation value is less than the second initial clarity evaluation value, the first difference is determined to be the difference obtained by subtracting the second initial clarity evaluation value from the first initial clarity evaluation value. It can be understood that the first image is clearer than the image corresponding to the second initial clarity evaluation value. If the first initial clarity evaluation value is greater than the second initial clarity evaluation value, it means that the first initial clarity evaluation value of the first image output by the initial clarity evaluation network is partially accurate. As mentioned before, the absolute value of the penalty amount is negatively correlated with the accuracy of the first initial clarity evaluation value, reflecting the inaccuracy of the initial clarity evaluation network. Therefore, when the first initial clarity evaluation value is greater than the second initial clarity evaluation value, this evaluation value is relatively accurate compared to the images ranked before the first image in the training images. Therefore, the first difference is determined to be 0. When the first initial clarity evaluation value is less than the second initial clarity evaluation value, the initial clarity evaluation network's evaluation of the clarity of the first image is inaccurate. Therefore, the first difference is determined to be the difference obtained by subtracting the second initial clarity evaluation value from the first initial clarity evaluation value.

[0082] When determining the second difference, similar to the method of determining the first difference, there are also two cases. If the first initial clarity evaluation value is less than the third initial clarity evaluation value, the second difference is determined to be 0. If the first initial clarity evaluation value is greater than the third initial clarity evaluation value, the second difference is determined to be the difference obtained by subtracting the third initial clarity evaluation value from the first initial clarity evaluation value. It can be understood that the first image is blurrier than the image corresponding to the third initial clarity evaluation value. If the first initial clarity evaluation value is less than the second initial clarity evaluation value, it means that this evaluation value is relatively accurate compared to the images ranked after the first image in the training images. Therefore, the second difference is determined to be 0. When the first initial clarity evaluation value is greater than the third initial clarity evaluation value, the initial clarity evaluation network's evaluation of the clarity of the first image is inaccurate. Therefore, the second difference is determined to be the difference obtained by subtracting the third initial clarity evaluation value from the first initial clarity evaluation value.

[0083] In S502, it can be understood that the value of the first difference calculated above is 0 or negative, while the value of the second difference is 0 or positive. Therefore, if the penalty amount obtained by accumulating the first difference and the second difference is greater than 0, it indicates that the evaluation value of the initial clarity evaluation network for evaluating the clarity of the image is too large. If the penalty amount is less than 0, it indicates that the evaluation value of the initial clarity evaluation network for evaluating the clarity of the image is too small. Regardless of the positive or negative of the penalty amount, its absolute value can reflect the inaccuracy of the initial clarity evaluation network.

[0084] By selecting this embodiment, a reasonable penalty amount for the initial clarity evaluation network can be obtained, thereby improving the accuracy of the second initial clarity evaluation network obtained by training.

[0085] In a possible embodiment, the initial clarity evaluation network is adjusted according to the penalty amount to obtain a second clarity evaluation network. As Figure 6 shown, it may include:

[0086] S601, establish a loss function regarding the network adjustment parameter according to the penalty amount.

[0087] S602, determine the first network adjustment parameter that makes the output of the loss function less than a preset threshold.

[0088] S603, adjust the initial clarity evaluation network according to the first network adjustment parameter to obtain a second clarity evaluation network.

[0089] Among them, in S601, the network adjustment parameter is the adjustment amount for adjusting the parameters of the initial clarity evaluation network according to the output of the initial clarity evaluation network. There are various methods for establishing the loss function, such as the mean square error loss function, the huber loss function, and so on.

[0090] In S602, it can be understood that the process of training the initial clarity evaluation network to the second clarity evaluation network is a process of gradually reducing the output of the loss function. In a possible embodiment, the gradient descent algorithm can be used to update the network adjustment parameter so that the output of the loss function gradually decreases. Those skilled in the art can also preset an output threshold of the loss function in advance. When adjusting the initial clarity evaluation network according to a certain network adjustment parameter makes the output of the loss function less than the preset threshold, this network adjustment parameter can be determined as the first network adjustment parameter.

[0091] In S603, it can be understood that adjusting the initial clarity evaluation network according to the first network adjustment parameter can accurately evaluate the clarity of the image to be evaluated. Specifically, when adjusting the initial clarity evaluation network according to the first network adjustment parameter, the initial clarity evaluation network can be adjusted accordingly based on the network structure of the initial clarity evaluation network and the first network adjustment parameter. Exemplarily, the number of convolutional modules or the first pooling module of the initial clarity evaluation network can be adjusted according to the first network adjustment parameter, or the number and size of the convolutional kernels in the convolutional module can be adjusted according to the first network adjustment parameter, etc. All of these are ways to adjust the initial clarity evaluation network according to the first network adjustment parameter.

[0092] By selecting this embodiment, a reasonable first network adjustment parameter can be obtained according to the penalty amount to adjust the initial clarity evaluation network. Further, a second initial clarity evaluation network that can accurately evaluate the image clarity can be obtained.

[0093] In a possible embodiment, determining the first network adjustment parameter that makes the output of the loss function less than a preset threshold includes:

[0094] Adjust the initial network parameters of the initial clarity evaluation network in the gradient descent direction of the initial mapping function until the output of the loss function obtained after adjustment is less than the preset threshold. Take the network parameters obtained after adjustment as the first network adjustment parameter, where the adjustment amplitude is positively correlated with the penalty amount, and the initial mapping function is the mapping function implemented by the initial clarity evaluation network.

[0095] It can be understood that the initial clarity evaluation network is an untrained network, and the initial network parameters it contains are untrained and unadjusted network parameters. The initial mapping function is the function implemented by the initial clarity evaluation network. Specifically, the image to be evaluated can be set as x, and the initial network parameters can be set as w. Then, the clarity evaluation value obtained by evaluating the image to be evaluated through the initial clarity evaluation network can be f(x, w), and f(x, w) can be used as the initial mapping function, which expresses the relationship between the initial network parameters, the image to be evaluated, and the clarity evaluation value. When adjusting the initial network parameters, new clarity evaluation values will be continuously obtained according to the initial mapping function. The new clarity evaluation values are input into the constructed loss function to calculate the clarity evaluation value that makes the output of the loss function less than the preset threshold, and then the initial network parameters corresponding to this clarity evaluation value are inversely deduced according to the initial mapping function. Take this initial network parameter as the first network adjustment parameter.

[0096] By selecting this embodiment, an effective first network adjustment parameter can be determined, so that the second clarity evaluation network obtained subsequently according to the first network adjustment parameter can accurately evaluate the clarity of the image to be evaluated.

[0097] In a possible embodiment, the present invention further provides a specific method for adjusting the initial network parameters of the initial clarity evaluation network in the gradient descent direction of the initial mapping function to obtain a second clarity evaluation network. The specific implementation steps are as follows:

[0098] Assume that the first image is x, and the first initial clarity evaluation value is The network parameters of the initial clarity evaluation network are w, the ideal clarity evaluation value of the first image is FV, and the first initial clarity evaluation value is set The difference from the ideal clarity evaluation value FV of the first image is ΔFV, and the penalty amount is f x .

[0099] Then the established square loss function can be the following formula:

[0100]

[0101] It can be understood that the network adjustment parameter w is a variable, and the first initial clarity evaluation value should be the clarity evaluation value obtained after inputting the first image x and the network adjustment parameter set as w into the initial clarity evaluation network, and can be expressed as a function: And the gradient descent algorithm is used to update the network adjustment parameter until the output of the loss function is less than the preset threshold after the update. It can be carried out according to the following formula:

[0102]

[0103] Among them, η is the learning rate, is the partial derivative of the loss function with respect to the network adjustment parameter. The following derivation can be carried out according to the above formula:

[0104]

[0105] And in the above formula, actually the ideal clarity evaluation value FV of the first image is unknown, but it can be assumed that the first initial clarity evaluation value is set The deviation ΔFV from the ideal clarity evaluation value FV of the first image is used to simplify the above formula:

[0106]

[0107] As can be seen from the acquisition of the penalty amount described above, if the penalty amount is greater than 0, it indicates that the evaluation value of the initial clarity evaluation network for evaluating the clarity of the picture is on the high side. If the penalty amount is less than 0, it indicates that the evaluation value of the initial clarity evaluation network for evaluating the clarity of the picture is on the low side. That is, the absolute value of the penalty amount is negatively correlated with the accuracy of the first initial clarity evaluation value and positively correlated with ΔFV. Based on this, the penalty amount can be used to replace ΔFV:

[0108] ΔFV = k * f x

[0109] where k > 0. Based on this,

[0110]

[0111] can be transformed into

[0112]

[0113] will

[0114]

[0115] be transformed into

[0116]

[0117] Among them, the learning rate η and the parameter k are both hyperparameters that can be set by themselves. Therefore, they can be merged into the same parameter η1. Therefore, the above formula can be further simplified as:

[0118]

[0119] Based on the above formula, after obtaining the first image and its corresponding penalty amount, the updated network adjustment parameter w can be obtained, and then the initial clarity evaluation network can be continuously adjusted to obtain w1 that makes the output of the loss function lower than the preset threshold, and use w1 to update the initial clarity evaluation network to obtain a second clarity evaluation network that can accurately evaluate the picture to be evaluated.

[0120] In a possible embodiment, the present invention also provides a method for adjusting the initial clarity evaluation network according to the first network adjustment parameter to obtain a second clarity evaluation network, including:

[0121] Adjust the number and size of the convolutional kernels and the number of convolutional pooling units in the initial clarity evaluation network according to the first network adjustment parameter to obtain a second clarity evaluation network.

[0122] Specifically, as described above, the second sharpness evaluation network may include a plurality of convolutional modules and a plurality of first pooling modules. Each convolutional module may contain convolutional kernels of multiple different sizes. Therefore, when evaluating the initial sharpness evaluation network by adjusting parameters according to the first network, it may be to adjust the number of convolutional modules and first pooling modules of the initial sharpness evaluation network according to the parameters adjusted by the first network, or to adjust the number and size of convolutional kernels in the convolutional module according to the parameters adjusted by the first network, and so on. All of these are adjustments that can be made to the initial sharpness evaluation network according to the parameters adjusted by the first network.

[0123] Selecting this embodiment can quickly and accurately adjust the initial sharpness evaluation network according to the parameters of the first network, thereby improving the training efficiency of the second sharpness evaluation network.

[0124] In a possible embodiment, the present invention also provides a second sharpness evaluation network with a structure as Figure 7 shown, including a normalization module, a preset number of convolutional modules, a preset number of first pooling modules, a global pooling module, and a fully connected module.

[0125] In a possible embodiment, the present invention also provides an image sharpness evaluation method, as Figure 8 shown, the method includes:

[0126] S801, obtaining at least one image to be evaluated.

[0127] S802, processing at least one image to be evaluated through a convolutional pooling unit in a pre-trained second sharpness evaluation network to obtain a convolutional pooling feature map.

[0128] S803, performing global pooling on the convolutional pooling feature map through a global pooling module in the second sharpness evaluation network to obtain a global pooling feature map.

[0129] S804, performing weighted processing on the global pooling feature map through a fully connected module in the second sharpness evaluation network to obtain a sharpness evaluation value of at least one image to be evaluated.

[0130] Wherein, in S802, the convolutional pooling unit includes a convolutional module and a pooling module, and is used to perform convolutional processing and pooling processing on the input.

[0131] Specifically, when the convolution pooling unit receives an input, first, the convolution module can perform convolution processing on the input to obtain a convolution processed feature map, and the pooling module can use the convolution processed feature map output by the convolution module as an input to perform pooling processing to obtain a convolution pooling feature map. Then, the global pooling module performs global pooling processing on the convolution pooling feature map, and the fully connected module can perform weighted accumulation on all features in the global pooling feature map output by the global pooling module for feature fusion to obtain the sharpness evaluation value of the image to be evaluated.

[0132] It can be understood that the processing methods of the convolution module, the pooling module, the global pooling module, and the fully connected module have been described above and will not be elaborated here.

[0133] In a possible embodiment, the convolution pooling unit in the pre-trained second sharpness evaluation network processes at least one image to be evaluated to obtain a convolution pooling feature map, including:

[0134] The convolution pooling unit in the pre-trained second sharpness evaluation network processes the at least one image to be evaluated multiple times to obtain a convolution pooling feature map.

[0135] Specifically, the second sharpness evaluation network may include multiple convolution pooling units. The multiple convolution pooling units can be arranged, and at least one image to be evaluated is input into the first convolution pooling unit to obtain the output of the first convolution pooling unit, and this output can be used as the input of the next convolution pooling unit. In this order, the at least one image to be evaluated can be processed a preset number of times through a preset number of convolution pooling units. Furthermore, the output of the last convolution pooling unit can be used as the convolution pooling feature map.

[0136] It can be understood that the multiple convolution pooling units can extract the image features of the image to be evaluated multiple times, so as to ensure that the second sharpness evaluation network can obtain sufficient and rich image features from the image to be evaluated. Further, the second sharpness evaluation network can comprehensively consider various factors affecting the sharpness of the image to be evaluated to determine the sharpness evaluation value of the image to be evaluated, realizing an accurate evaluation of the sharpness of the image. And the number of convolution pooling units included in the second sharpness evaluation network can be set by the user according to actual experience or requirements.

[0137] In a possible embodiment, the convolution module includes at least 3 feature channels, where the feature channels are used to sequentially perform convolution processing, normalization processing, and tanh function activation processing on at least one image to be evaluated.

[0138] Specifically, the convolution module may include a convolution kernel, a normalization channel, and a tanh activation function. One feature channel can include a convolution kernel, a normalization channel, and a tanh activation function. Therefore, one feature channel can perform convolution processing, normalization processing, and activation processing of the tanh activation function on the image to be evaluated.

[0139] In a possible embodiment, at least three feature channels respectively include convolution kernels of sizes 1x1, 3x3, and 5x5.

[0140] Specifically, a convolution module may include at least three feature channels, and different feature channels may include convolution kernels of different sizes. For example, the first feature channel may include a convolution kernel of size 1x1, the second feature channel may include a convolution kernel of size 3x3, and the third feature channel may include a convolution kernel of size 5x5.

[0141] In a possible embodiment, the pooling module includes a max pooling module and an average pooling module.

[0142] An embodiment of the present invention further provides a schematic structural diagram of an image sharpness evaluation device, which may include:

[0143] An image acquisition unit, configured to acquire an image to be evaluated;

[0144] An image evaluation unit, configured to input the image to be evaluated into a trained second sharpness evaluation network. The second sharpness evaluation network includes: a convolution module, where the convolution module includes a plurality of convolution kernels of different sizes, a normalization channel, and an activation function. The plurality of convolution kernels of different sizes perform convolution processing on the input of the convolution module to obtain a convolution kernel processing amount. The normalization channel uses the convolution kernel processing amount as an input to perform normalization processing to obtain a normalization channel processing amount. The activation function uses the normalization channel processing amount as an input to perform activation processing to obtain an activation processing amount. The convolution module uses the activation processing amount as a convolution processing amount;

[0145] An evaluation acquisition unit, configured to output a sharpness evaluation value of the image to be evaluated by the second sharpness evaluation network.

[0146] In a possible embodiment, the second sharpness evaluation network further includes: a normalization module;

[0147] The image evaluation unit is further configured to perform normalization processing on the image to be evaluated input into the second sharpness evaluation network by the normalization module to obtain a normalization processing amount, where the normalization processing is mean-variance normalization processing.

[0148] In a possible embodiment, the second sharpness evaluation network further includes: a first pooling module and a global pooling module. The first pooling module includes: a max pooling sub-module and an average pooling sub-module;

[0149] The image evaluation unit is further configured to: the max pooling sub-module and the average pooling sub-module respectively perform pooling processing on the convolution processing amount as input to obtain a max pooling amount and an average pooling amount; the first pooling module uses the max pooling amount and the average pooling amount as a first pooling amount; the global pooling module performs global pooling processing on the first pooling amount as input to obtain a global pooling amount.

[0150] In a possible embodiment, the second sharpness evaluation network further includes: a fully connected module;

[0151] The image evaluation unit is further configured to: the fully connected module performs weighted accumulation on the global pooling amount as input to obtain an accumulated feature, where the fully connected module includes: a sigmoid activation function sub-module; the sigmoid activation function sub-module maps the accumulated feature to obtain a mapped amount; and uses the mapped amount as the sharpness evaluation value of the image to be evaluated.

[0152] In a possible embodiment, the sharpness evaluation device further includes a training unit, configured to train the second sharpness evaluation network in the following manner:

[0153] Input a set of training images into the initial sharpness evaluation network to obtain the initial sharpness evaluation values of the images in the training images. The training images are a set of images arranged in the order of clarity from clear to blurred;

[0154] Randomly select a first image from the training images and determine the first initial sharpness evaluation value corresponding to the first image;

[0155] Judge the magnitude relationship between the first initial sharpness evaluation value, the second initial sharpness evaluation value, and the third initial sharpness evaluation value. The second initial sharpness evaluation value is the maximum value of the initial sharpness evaluation values in the training images sorted before the first image, and the third initial sharpness evaluation value is the minimum value of the initial sharpness evaluation values in the images sorted after the first image in the training images;

[0156] Determine the penalty amount of the initial sharpness evaluation network according to the magnitude relationship;

[0157] Adjust the initial sharpness evaluation network according to the penalty amount to obtain the second sharpness evaluation network.

[0158] In a possible embodiment, the training unit is further configured to determine a penalty amount for the initial clarity evaluation network according to the size relationship, including:

[0159] Determine a first difference and a second difference;

[0160] Use the sum of the first difference and the second difference as the penalty amount for the initial clarity evaluation network;

[0161] Wherein, determining the first difference includes:

[0162] If the first initial clarity evaluation value is greater than the second initial clarity evaluation value, determine the first difference to be 0;

[0163] If the first initial clarity evaluation value is less than the second initial clarity evaluation value, determine the first difference to be the difference obtained by subtracting the second initial clarity evaluation value from the first initial clarity evaluation value;

[0164] Wherein, determining the second difference includes:

[0165] If the first initial clarity evaluation value is less than the third initial clarity evaluation value, determine the second difference to be 0;

[0166] If the first initial clarity evaluation value is greater than the third initial clarity evaluation value, determine the second difference to be the difference obtained by subtracting the third initial clarity evaluation value from the first initial clarity evaluation value.

[0167] In a possible embodiment, the training unit is further configured to adjust the initial clarity evaluation network according to the penalty amount to obtain a second clarity evaluation network, including:

[0168] Establish a loss function for network adjustment parameters according to the penalty amount, where the network adjustment parameters are the adjustment amounts for adjusting the parameters of the initial clarity evaluation network according to the output of the initial clarity evaluation network;

[0169] Determine the first network adjustment parameter that makes the output of the loss function less than a preset threshold;

[0170] Adjust the initial clarity evaluation network according to the first network adjustment parameter to obtain a second clarity evaluation network.

[0171] In a possible embodiment, the training unit is specifically configured to adjust the initial network parameters of the initial clarity evaluation network in the gradient descent direction of the initial mapping function until the output of the loss function is less than a preset threshold after the adjustment, and use the network parameters obtained after the adjustment as the first network adjustment parameters, where the adjustment amplitude is positively correlated with the penalty amount, and the initial mapping function is the mapping function implemented by the initial clarity evaluation network.

[0172] In a possible embodiment, the training unit is specifically configured to adjust the initial network parameters according to the following formula:

[0173]

[0174] where w is the first network adjustment parameter, x is at least one image to be evaluated, η1 is a hyperparameter, f x is the penalty amount corresponding to x, and f(x, w) is the first initial clarity evaluation value.

[0175] In a possible embodiment, the training unit is specifically configured to adjust the number and size of the convolution kernels, the number of convolution modules, and the number of the first pooling modules in the initial clarity evaluation network according to the first network adjustment parameters to obtain a second clarity evaluation network.

[0176] In another embodiment provided by the present invention, a computer-readable storage medium is further provided. A computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of any of the above image clarity evaluation methods are implemented.

[0177] In another embodiment provided by the present invention, a computer program product including instructions is further provided. When it runs on a computer, the computer is caused to execute any of the above image clarity evaluation methods in the embodiments.

[0178] In the above embodiments, they can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state disk (SSD)), etc.

[0179] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0180] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0181] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. An image sharpness evaluation method, characterized in that, The method includes: Obtaining an image to be evaluated; Inputting the image to be evaluated into a trained second sharpness evaluation network, where the second sharpness evaluation network includes: a convolution module, and the convolution module includes convolution kernels of multiple different sizes, a normalization channel, and an activation function. The convolution kernels of multiple different sizes perform convolution processing on the input of the convolution module to obtain a convolution kernel processing amount. The normalization channel takes the convolution kernel processing amount as input and performs normalization processing to obtain a normalization channel processing amount. The activation function takes the normalization channel processing amount as input and performs activation processing to obtain an activation processing amount. The convolution module takes the activation processing amount as a convolution processing amount; The second sharpness evaluation network outputs a sharpness evaluation value of the image to be evaluated; The second sharpness evaluation network is obtained through the following training method: Inputting a set of training images into an initial sharpness evaluation network to obtain initial sharpness evaluation values of each image in the training images, where the training images are a set of images arranged in the order from clear to blurred in terms of sharpness; Randomly selecting a first image from the training images and determining a first initial sharpness evaluation value corresponding to the first image; Judging the magnitude relationship between the first initial sharpness evaluation value, a second initial sharpness evaluation value, and a third initial sharpness evaluation value. The second initial sharpness evaluation value is the maximum value of the initial sharpness evaluation values among the training images sorted before the first image in the training images. The third initial sharpness evaluation value is the minimum value of the initial sharpness evaluation values among the images sorted after the first image in the training images; Determining a first difference and a second difference; Taking the sum of the first difference and the second difference as the penalty amount of the initial sharpness evaluation network. Among them, determining the first difference includes: if the first initial sharpness evaluation value is greater than the second initial sharpness evaluation value, determining the first difference to be 0; if the first initial sharpness evaluation value is less than the second initial sharpness evaluation value, determining the first difference to be the difference obtained by subtracting the second initial sharpness evaluation value from the first initial sharpness evaluation value. Determining the second difference includes: if the first initial sharpness evaluation value is less than the third initial sharpness evaluation value, determining the second difference to be 0; if the first initial sharpness evaluation value is greater than the third initial sharpness evaluation value, determining the second difference to be the difference obtained by subtracting the third initial sharpness evaluation value from the first initial sharpness evaluation value; Adjusting the initial sharpness evaluation network according to the penalty amount to obtain a second sharpness evaluation network.

2. The method according to claim 1, wherein The second sharpness evaluation network further includes: a normalization module; The normalization module performs normalization processing on the image to be evaluated input into the second sharpness evaluation network to obtain a normalization processing amount, where the normalization processing is mean-variance normalization processing.

3. The method according to claim 1, wherein The second sharpness evaluation network further includes: a first pooling module and a global pooling module. The first pooling module includes: a max pooling sub-module and an average pooling sub-module; The maximum pooling sub-module and the average pooling sub-module perform pooling processing on the convolution processing amount as input respectively to obtain a maximum pooling amount and an average pooling amount; The first pooling module uses the maximum pooling amount and the average pooling amount as the first pooling amount; The global pooling module performs global pooling processing on the first pooling amount as input to obtain a global pooling amount.

4. The method according to claim 3, characterized in that, The second sharpness evaluation network further includes: a fully connected module; The fully connected module performs weighted accumulation on the global pooling amount as input to obtain an accumulated feature, where the fully connected module includes: a sigmoid activation function sub-module; The sigmoid activation function sub-module maps the accumulated feature to obtain a mapped amount; The mapped amount is used as the sharpness evaluation value of the image to be evaluated.

5. The method according to claim 3, characterized in that, The adjusting the initial sharpness evaluation network according to the penalty amount to obtain a second sharpness evaluation network includes: Establishing a loss function regarding network adjustment parameters according to the penalty amount, where the network adjustment parameters are adjustment amounts for adjusting the parameters of the initial sharpness evaluation network according to the output of the initial sharpness evaluation network; Determining a first network adjustment parameter that makes the output of the loss function less than a preset threshold; Adjusting the initial sharpness evaluation network according to the first network adjustment parameter to obtain a second sharpness evaluation network.

6. The method according to claim 5, characterized in that The determining the first network adjustment parameter that makes the output of the loss function less than a preset threshold includes: Adjusting the initial network parameters of the initial sharpness evaluation network in the gradient descent direction of the initial mapping function until the network parameters obtained after adjustment make the output of the loss function less than a preset threshold, and taking the network parameters obtained after adjustment as the first network adjustment parameter, where the adjustment amplitude is positively correlated with the penalty amount, and the initial mapping function is the mapping function implemented by the initial sharpness evaluation network.

7. The method according to claim 6, characterized in that The adjusting the initial network parameters of the initial sharpness evaluation network in the gradient descent direction of the initial mapping function includes: Adjusting the initial network parameters according to the following formula: Among them, w is the first network adjustment parameter, x is at least one image to be evaluated, η1 is a hyperparameter, and f x is the penalty amount corresponding to x, and f(x, w) is the first initial sharpness evaluation value.

8. The method according to claim 5, wherein The adjusting the initial sharpness evaluation network according to the first network adjustment parameter to obtain a second sharpness evaluation network includes: Adjusting the number and size of the convolution kernels, the number of convolution modules, and the number of the first pooling modules in the initial sharpness evaluation network according to the first network adjustment parameter to obtain a second sharpness evaluation network.

9. An image sharpness evaluation method, characterized in that, The method includes: Obtaining at least one image to be evaluated; Processing the at least one image to be evaluated through a convolution pooling unit in a pre-trained second sharpness evaluation network to obtain a convolution pooling feature map, where the convolution pooling unit includes a convolution module and a pooling module for performing convolution processing and pooling processing on the input; Performing global pooling on the convolution pooling feature map through the global pooling module in the second sharpness evaluation network to obtain a global pooling feature map; Performing weighted processing on the global pooling feature map through the fully connected module in the second sharpness evaluation network to obtain the sharpness evaluation value of the at least one image to be evaluated; The second clarity evaluation network is obtained through the following training method: Input a set of training images into the initial clarity evaluation network to obtain the initial clarity evaluation values of each image in the training images. The training images are a set of images arranged in the order of clarity from clear to blurred; Randomly select a first image from the training images and determine the first initial clarity evaluation value corresponding to the first image; Judge the magnitude relationship between the first initial clarity evaluation value, the second initial clarity evaluation value, and the third initial clarity evaluation value. The second initial clarity evaluation value is the maximum value of the initial clarity evaluation values in the training images sorted before the first image, and the third initial clarity evaluation value is the minimum value of the initial clarity evaluation values in the images sorted after the first image in the training images; Determine the first difference and the second difference; Take the sum of the first difference and the second difference as the penalty amount of the initial clarity evaluation network. Among them, determining the first difference includes: if the first initial clarity evaluation value is greater than the second initial clarity evaluation value, determine the first difference to be 0; if the first initial clarity evaluation value is less than the second initial clarity evaluation value, determine the first difference to be the difference obtained by subtracting the second initial clarity evaluation value from the first initial clarity evaluation value; determining the second difference includes: if the first initial clarity evaluation value is less than the third initial clarity evaluation value, determine the second difference to be 0; if the first initial clarity evaluation value is greater than the third initial clarity evaluation value, determine the second difference to be the difference obtained by subtracting the third initial clarity evaluation value from the first initial clarity evaluation value; Adjust the initial clarity evaluation network according to the penalty amount to obtain the second clarity evaluation network.

10. The method according to claim 9, wherein Processing the at least one image to be evaluated through the convolutional pooling unit in the pre-trained second clarity evaluation network to obtain a convolutional pooling feature map includes: Processing the at least one image to be evaluated multiple times through the convolutional pooling unit in the pre-trained second clarity evaluation network to obtain a convolutional pooling feature map.

11. The method according to claim 9, wherein The convolutional module includes at least 3 feature channels, where the feature channels are used to sequentially perform convolutional processing, normalization processing, and tanh function activation processing on the at least one image to be evaluated.

12. The method according to claim 11, wherein The at least 3 feature channels respectively include convolutional kernels with sizes of 1x1, 3x3, and 5x5.

13. The method according to claim 11, wherein The pooling module includes a max pooling module and an average pooling module.

Images