Quality evaluation model training method and device, electronic equipment and storage medium

Abstract

The application provides a quality evaluation model training method and device, electronic equipment and a storage medium. The method comprises the following steps: performing data labeling on a plurality of sample images by using a data labeling function provided in a general framework, and obtaining a plurality of sample labels. The sample label is an image quality category corresponding to the sample image. The neural network model provided in the general framework is trained by taking the plurality of sample images as training data and taking the plurality of sample labels as training labels, and a quality evaluation model is obtained. The quality evaluation model is used for quality evaluation of a to-be-processed image. Data labeling and model training are respectively performed by using the data labeling function and the neural network model provided by the general framework, so that the problem of reextracting key influencing factors when the collection environment changes is avoided, and the applicability of evaluating the image quality is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning and deep learning, and in particular, to a quality evaluation model training method and device, electronic equipment and a storage medium. BACKGROUND

[0002] At present, the image quality is usually evaluated according to the mathematical features extracted from the image by artificial means. For example, the key influencing factors of image quality (such as contrast, sharpness, offset, information entropy, number and quality of lines, etc.) are analyzed, and then the corresponding quantitative formula is selected according to these key influencing factors to calculate the scores of each factor. Finally, the scores of each factor are weighted and averaged to obtain the image quality level (such as high and low quality levels) or image quality score (such as 0 to 10 points, etc.). The above-mentioned evaluation image can be an image in the application fields of pedestrian recognition, finger vein recognition, face recognition and fingerprint recognition, etc. In order to facilitate understanding and description, the quality evaluation of the image in the application field of finger vein recognition will be described in detail below.

[0003] In the specific practice process, it is found that when the collection environment changes, the key influencing factors need to be re-extracted, and the image quality is evaluated according to the re-extracted key influencing factors. For example, when using an open finger vein collection device for collection, the main influencing factors to be extracted are light and dust (since the finger position is relatively fixed and difficult to move and rotate, etc., factors such as finger movement, rotation and internal and external translation are not considered). However, when using a closed finger vein collection device for collection, since the light in the closed environment is basically unchanged, the main factors to be extracted for evaluation are the movement, rotation and internal and external translation of the fingertip. Therefore, the applicability of the method based on mathematical features for evaluating the image quality is not strong. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a quality evaluation model training method and device, electronic equipment and a storage medium, which can improve the problem of poor applicability of evaluating image quality.

[0005] The embodiment of the present application provides a quality evaluation model training method, comprising: performing data labeling on a plurality of sample images by using a data labeling function provided in a general framework, to obtain a plurality of sample labels, wherein the sample label is an image quality category corresponding to the sample image; taking the plurality of sample images as training data and taking the plurality of sample labels as training labels, to train a neural network model provided in the general framework, to obtain a quality evaluation model, wherein the quality evaluation model is used for quality evaluation on a to-be-processed image. In the implementation process of the above scheme, the data labeling function and the neural network model provided by the general framework are used for data labeling and model training respectively, so that the general framework can parameterize key influencing factor parameters, and modification of parameters (for example, the data labeling function and the neural network model) in the general framework can be applicable to the neural network model of the current scene, thereby avoiding the problem of re-extracting key influencing factors when the collection environment changes, and effectively improving the applicability of evaluating image quality.

[0006] Optionally, in the embodiment of the present application, before the data labeling function provided in the general framework is used to perform data labeling on the plurality of sample images, the method further comprises: expanding the sample images by using a sample expansion function provided in the general framework, to obtain expanded images; and adding the expanded images to the plurality of sample images. In the implementation process of the above scheme, the sample expansion function provided in the general framework is used to expand the sample images, and the expanded images are added to the plurality of sample images, thereby avoiding the problem of data imbalance of the plurality of sample images, and effectively improving the applicability and robustness of using the general framework to evaluate image quality.

[0007] Optionally, in the embodiment of the present application, the general framework further comprises: a quality category sequence; after the expanded images are obtained, the method further comprises: determining the quality category corresponding to the expanded images from the quality category sequence; and adding the quality category corresponding to the expanded images to the plurality of sample labels. In the implementation process of the above scheme, the quality category corresponding to the expanded images is determined from the quality category sequence, and the quality category corresponding to the expanded images is added to the plurality of sample labels, thereby realizing the function of using the general framework to perform data labeling on the sample images.

[0008] Optionally, in the embodiment of the present application, the quality category corresponding to the augmented image is determined from the quality category sequence, comprising: calculating the similarity between the augmented image and the homologous sample image in the plurality of sample images, the homologous sample image being a different sample image belonging to the same target object; calculating the ratio value between the similarity and the average similarity of the plurality of sample images; and determining the quality category corresponding to the augmented image from the quality category sequence according to the ratio value. In the implementation process of the above scheme, the similarity between the augmented image and the homologous sample image is calculated first, then the ratio value between the similarity and the average similarity of the plurality of sample images is calculated, and finally the quality category corresponding to the augmented image is determined from the quality category sequence according to the ratio value, thereby realizing the function of data labeling of the sample image using the general framework.

[0009] Optionally, in the embodiment of the present application, the quality category sequence comprises: a first quality category and a second quality category, the first quality category being lower than the second quality category; and the quality category corresponding to the augmented image is determined from the quality category sequence according to the ratio value, comprising: judging whether the ratio value is less than a preset threshold; and if yes, determining the first quality category as the quality category corresponding to the augmented image.

[0010] Optionally, in the embodiment of the present application, after judging whether the ratio value is less than the preset threshold, it further comprises: if the ratio value is greater than or equal to the preset threshold, determining the second quality category as the quality category corresponding to the augmented image. In the implementation process of the above scheme, the quality category corresponding to the augmented image is determined from the quality category sequence according to the ratio value, thereby realizing the function of data labeling of the sample image using the general framework.

[0011] Optionally, in the embodiment of the present application, after obtaining the quality evaluation model, it further comprises: performing quality evaluation on the to-be-processed image using the quality evaluation model to obtain the quality evaluation result of the to-be-processed image. In the implementation process of the above scheme, the quality evaluation model in the general framework is used to perform quality evaluation on the to-be-processed image, since the quality evaluation model in the general framework can consider various key influencing factors affecting the image quality, and does not need to make too many adjustments to the model training and model inference, thereby effectively reducing the workload of quality evaluation and improving the efficiency of quality evaluation.

[0012] The embodiment of the present application further provides a quality evaluation model training device, comprising: an image data labeling module, configured to label data of a plurality of sample images by using a data labeling function provided in a general framework, and obtain a plurality of sample labels, wherein the sample label is an image quality category corresponding to the sample image; a network model training module, configured to train a neural network model provided in the general framework by taking the plurality of sample images as training data and taking the plurality of sample labels as training labels, and obtain a quality evaluation model, wherein the quality evaluation model is used for quality evaluation of a to-be-processed image.

[0013] Optionally, in the embodiment of the present application, the quality evaluation model training device further comprises: an expanded image obtaining module, configured to expand the sample image by using a sample expansion function provided in the general framework, and obtain an expanded image; and an expanded image adding module, configured to add the expanded image to the plurality of sample images.

[0014] Optionally, in the embodiment of the present application, the general framework further comprises: a quality category sequence; and the quality evaluation model training device further comprises: a quality category determining module, configured to determine the quality category corresponding to the expanded image from the quality category sequence; and a quality category adding module, configured to add the quality category corresponding to the expanded image to the plurality of sample labels.

[0015] Optionally, in the embodiment of the present application, the quality category determining module comprises: a similarity calculation submodule, configured to calculate a similarity degree between the expanded image and a homologous sample image in the plurality of sample images, wherein the homologous sample image is a different sample image belonging to a same target object; a proportion value calculation submodule, configured to calculate a proportion value between the similarity degree and an average similarity of the plurality of sample images; and a quality category determining submodule, configured to determine the quality category corresponding to the expanded image from the quality category sequence according to the proportion value.

[0016] Optionally, in the embodiment of the present application, the quality category sequence comprises: a first quality category and a second quality category, wherein the first quality category is lower than the second quality category; and the quality category determining submodule comprises: a preset threshold judgment unit, configured to judge whether the proportion value is less than a preset threshold; and a first category determining unit, configured to determine the first quality category as the quality category corresponding to the expanded image if the proportion value is less than the preset threshold.

[0017] Optionally, in the embodiment of the present application, the quality category determining submodule further comprises: a second category determining unit, configured to determine the second quality category as the quality category corresponding to the expanded image if the proportion value is greater than or equal to the preset threshold.

[0018] Optionally, in the embodiment of the present application, the quality evaluation model training device further comprises: an image quality evaluation module, configured to perform quality evaluation on the to-be-processed image by using the quality evaluation model, and obtain a quality evaluation result of the to-be-processed image.

[0019] The embodiment of the present application further provides an electronic device, comprising a processor and a memory, the memory storing machine readable instructions executable by the processor, and the machine readable instructions are executed by the processor to perform the method as described above.

[0020] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium storing a computer program, and the computer program is executed by the processor to perform the method as described above.

[0021] Other features and advantages of the embodiments of the present application will be described in the following description and, in part, will become apparent from the description or will be learned by practice of the embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application, and it should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0023] Figure 1 The flowchart of the quality evaluation model training method provided by the embodiment of the present application is shown;

[0024] Figure 2 The sample expansion diagram of the finger vein image provided by the embodiment of the present application is shown;

[0025] Figure 3 The flowchart of the quality evaluation using the quality evaluation model provided by the embodiment of the present application is shown;

[0026] Figure 4 The structural diagram of the quality evaluation model training device provided by the embodiment of the present application is shown;

[0027] Figure 5 The structural diagram of the electronic device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed embodiments of the present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the embodiments of the present application.

[0029] It can be understood that "first", "second" in the embodiments of the present application are used to distinguish similar objects. Those skilled in the art can understand that "first", "second" and the like do not limit the number and execution order, and "first", "second" and the like do not necessarily mean different.

[0030] Before introducing the quality evaluation model training method provided by the embodiments of the present application, some concepts involved in the embodiments of the present application are introduced first:

[0031] Finger-vein identification is a new biometric identification technology. Finger-vein identity authentication uses the distribution image of the veins in the finger to identify the identity.

[0032] It should be noted that the quality evaluation model training method provided by the embodiments of the present application can be executed by an electronic device. The electronic device herein refers to a device terminal or a server having the function of executing a computer program, such as a smart phone, a personal computer, a tablet computer, a personal digital assistant, or a mobile Internet device. The server refers to a device that provides computing services through a network, such as an x86 server and a non-x86 server, including a mainframe, a minicomputer, and a UNIX server.

[0033] The application scenarios suitable for the quality evaluation model training method are introduced below. The application scenarios include, but are not limited to, pedestrian recognition, finger-vein identification, face recognition, and fingerprint recognition, and the like. In order to facilitate understanding and description, the quality evaluation of finger-vein identification application field images (hereinafter referred to as finger-vein images) is taken as an example for detailed description. Since the finger-vein image is personal identity sensitive private information, it is difficult to collect a large number of original finger-vein images in the usual real situation, and therefore the quality evaluation model training method can be used for sample expansion, data labeling, and model training, and the like.

[0034] It can be understood that the quality evaluation model training method can also be used to train the neural network model set in the general framework, and the trained quality evaluation model can be used to evaluate the quality of the to-be-processed image, so that the general framework can parameterize the key influencing factors, and modifying the parameters (such as the data labeling function and the neural network model) in the general framework can be applicable to the neural network model of the current scene, avoiding the problem of re-extracting the key influencing factors when the acquisition environment changes, thereby effectively improving the applicability of evaluating the image quality and the like.

[0035] Please refer to Figure 1 The flowchart of the quality evaluation model training method provided by the embodiments of the application is shown.

[0036] Step S110: using the data labeling function set in the general framework to perform data labeling on the plurality of sample images to obtain a plurality of sample labels, the sample label being the image quality category corresponding to the sample image.

[0037] The general framework refers to a software framework for quality evaluation, and the general framework can be provided with a data labeling function, a sample expansion function, a neural network model, and / or a quality category sequence, etc. The general framework can be represented as M(A, C, F, N), wherein A represents the data labeling function, C represents the quality category sequence, F represents the sample expansion function, N represents the neural network model, and M represents the general framework.

[0038] It can be understood that if the user does not set the data labeling function, the sample expansion function, the neural network model, or the quality category sequence in the general framework, the data labeling function, the sample expansion function, the neural network model, or the quality category sequence set by default (i.e., selected by default) in the general framework can be used.

[0039] Step S120: using the plurality of sample images as training data and the plurality of sample labels as training labels, training the neural network model set in the general framework to obtain a quality evaluation model, the quality evaluation model being used for quality evaluation of the to-be-processed image.

[0040] The neural network model refers to the model of the neural network set in the general framework after being trained, and the neural network model is a model used for quality evaluation of the to-be-processed image. The neural network model includes but is not limited to Siamese-Network model, MobileNet model, and Squeeze model, etc., wherein the MobileNet model includes MobileNet V1 model, MobileNet V2 model, and MobileNet V3 model, etc.

[0041] The implementation of step S120 can be, for example, using the neural network model set in the general framework to predict the image quality category corresponding to the sample image, obtaining the predicted category, and using the mean square error (MSE), cross-entropy loss function, square loss function, exponential loss, zero-one loss, or absolute value loss function to calculate the loss value between the predicted category and the image quality category in the sample label, and then updating the network weight parameters of the neural network model according to the loss value until the accuracy of the neural network model no longer increases or the number of iterations (epoch) is greater than a preset threshold, that is, the trained quality evaluation model can be obtained. The preset threshold can be set according to specific conditions, for example, 100 or 1000, and the quality evaluation model is used for quality evaluation of the image to be processed.

[0042] It can be understood that, since the general framework abstracts various parameters (i.e., data labeling function, sample expansion function, neural network model, or quality category sequence), and the general framework has a default setting (i.e., default selection) of data labeling function, sample expansion function, neural network model, or quality category sequence, the user can directly modify the general framework (e.g., data labeling function and neural network model) to adapt to the neural network model of the current scene, thereby improving the flexibility and adaptability of the general framework.

[0043] In the implementation process of the above scheme, data labeling and model training are performed by the data labeling function and the neural network model provided by the general framework, respectively, so that the general framework can parameterize the key influencing factors. Modifying the parameters (e.g., data labeling function and neural network model) in the general framework can adapt to the neural network model of the current scene, avoiding the problem of reextracting key influencing factors when the collection environment changes, thereby effectively improving the applicability of evaluating image quality.

[0044] As an optional implementation of the above quality evaluation model training method, image expansion (also known as image enhancement) can be performed before data labeling using the data labeling function set in the general framework. This implementation can include:

[0045] Step S111: Expanding the sample image using the sample expansion function set in the general framework to obtain an expanded image.

[0046] Please refer to Figure 2The illustration shows a sample augmentation diagram of a finger vein image provided in an embodiment of this application. The sample augmentation function refers to a function set in a general framework for augmenting (also known as image enhancement) a sample image. Here, the image augmentation of a finger vein image is used as an example. Sample augmentation functions that can be used for finger vein images include, but are not limited to: Gaussian blur function, random translation along the X-axis (X-shift) function, random translation along the Y-axis (Y-shift) function, random translation along the Z-axis (Z-shift) function, rotation along the X-axis (Roll) function, rotation along the Y-axis (Pitch) function, rotation along the Z-axis (Yaw) function, and image contrast attenuation function, etc.

[0047] An example implementation of step S111 above is as follows: From the high-quality category and the low-quality category, determine the quality category with fewer sample images. Assuming the high-quality category has 2000 sample images and the low-quality category has 150 sample images, it is clear that the number of sample images in the low-quality category is much lower than that in the high-quality category. Therefore, the sample augmentation function set in the general framework can be used to augment the low-quality category sample images to obtain augmented images. The above image augmentation (also known as image enhancement) process can be expressed by the formula (I1′,…,I′). m )=F(I1,…,I n ), where m represents the number of augmented images, (I1′,…,I′) m ) represents the augmented image, I1,…,I n This represents the number of sample images to be augmented, where n represents the number of sample images, and F represents the set of sample augmentation functions set in the general framework. There can be many types of sample augmentation functions, for example, expressed by the formula: F = (f1, ..., f i ,…,f t ), where f i (i = 1, ..., t) represents one of the sample augmentation functions in the set of sample augmentation functions.

[0048] Step S112: Add the augmented image to multiple sample images to obtain multiple augmented sample images.

[0049] For example, the implementation of step S112 above can be described by adding the augmented image to multiple sample images using the formula I = (I′1, ..., I′). m )∪(I1,…,I n )=(I1′,…,I′ m ,…,I1,…,I n), thereby obtaining the augmented plurality of sample images; wherein I represents the augmented plurality of sample images, m represents the number of augmented augmented images, (I'1,..., I'm) represents the augmented augmented images, I1,..., In represent the sample images that need to be augmented, and n represents the number of sample images. m ) represents the augmented augmented images, I1,..., In represent the sample images that need to be augmented, and n represents the number of sample images. n ) represents the augmented augmented images, I1,..., In represent the sample images that need to be augmented, and n represents the number of sample images.

[0050] As an optional implementation of the above quality evaluation model training method, the above general framework can further include: a quality category sequence; after obtaining the augmented images in step S111, the quality category of the augmented images (i.e. data labeling) can be determined.

[0051] The first implementation of the above data labeling can perform data labeling on the augmented images, and this implementation can include:

[0052] Step S113: determining the quality category corresponding to the augmented image from the quality category sequence.

[0053] It can be understood that the quality category corresponding to the augmented image can be determined from the quality category sequence C = (C1, C2) of two quality categories, or from the quality category sequence C = (C1, C2, C3) of three quality categories; wherein C represents the quality category sequence, C1 represents the first quality category, C2 represents the second quality category, and C3 represents the third quality category, which will be discussed in detail below.

[0054] Step S114: adding the quality category corresponding to the augmented image to the plurality of sample labels.

[0055] The implementation of the above step S114, for example, uses an executable program compiled or interpreted by a pre-set programming language to add the quality category corresponding to the augmented image to the plurality of sample labels; the programming languages that can be used include C, C++, Java, BASIC, JavaScript, LISP, Shell, Perl, Ruby, Python, and PHP, etc.

[0056] As an optional implementation of the above step S113, in the process of determining the quality category corresponding to the augmented image (i.e. data labeling), the similarity of the homologous sample images can also be used for screening, and this implementation can include:

[0057] Step S113a: calculating the similarity between the augmented image and the homologous sample images in the plurality of sample images, wherein the homologous sample images are different sample images belonging to the same target object.

[0058] The implementation of the above step S113a, for example, uses the formula S i,j represents the similarity degree between the jth augmented image and the ith category homologous sample image in the plurality of sample images, I i,j represents the image feature of the jth augmented image in the ith category homologous sample image, I i,k represents the kth image feature in the ith category homologous sample image, h(I i,j ,I i,k represents the distance function between the two, where the distance function includes but is not limited to: cosine distance function, Euclidean distance function or Hamming distance function, etc., N i represents the number of sample images in the ith category homologous sample image. The homologous sample image refers to different sample images belonging to the same target object as the augmented image, for example: images of the same finger taken at different direction angles, or images of different parts of the same finger (such as left and right sides).

[0059] Step S113b: Calculate the ratio value between the similarity degree and the average similarity of the plurality of sample images.

[0060] The implementation of the above step S113b is, for example: first, according to the formula calculate the average similarity of the plurality of sample images; wherein, represents the average similarity of the other sample images in the ith category homologous sample image in the plurality of sample images except the jth augmented image, S i,k represents the similarity degree between the kth augmented image and the ith category homologous sample image in the plurality of sample images, N i represents the number of sample images in the ith category homologous sample image. Then, the ratio value between the similarity degree and the average similarity of the plurality of sample images can be calculated, which can be represented by the formula wherein, S i,j represents the similarity degree between the jth augmented image and the ith category homologous sample image in the plurality of sample images, represents the average similarity of the other sample images in the ith category homologous sample image in the plurality of sample images except the jth augmented image, T represents the ratio value between the similarity degree and the average similarity of the plurality of sample images.

[0061] Step S113c: Determine the quality category corresponding to the augmented image from the quality category sequence according to the ratio value.

[0062] As a first implementation of the above step S113c, from the quality category sequence of two quality categories, the quality category sequence includes: a first quality category and a second quality category, the first quality category is lower than the second quality category; wherein the first quality category can be a low quality category, and the second quality category can be a high quality category, which can be represented by a formula, for example, C=(C1, C2); C represents the quality category sequence, C1 represents the first quality category, and C2 represents the second quality category. According to the proportion value, the quality category corresponding to the augmented image is determined from the quality category sequence. This implementation can include: judging whether the proportion value T is less than a preset threshold T0; if the proportion value is less than the preset threshold, i.e. T < T0, the first quality category is determined as the quality category corresponding to the augmented image. If the proportion value is greater than or equal to the preset threshold, i.e. T ≥ T0, the second quality category is determined as the quality category corresponding to the augmented image. Wherein, the preset threshold T0 can be set according to specific circumstances, for example: the preset threshold T0 is set to 65% or 85% and the like.

[0063] As a second implementation of the above step S113c, from the quality category sequence of three quality categories, the quality category sequence includes: a first quality category, a second quality category and a third quality category, the first quality category is lower than the second quality category, and the second quality category is lower than the third quality category. Wherein the first quality category can be a low quality category, and the second quality category can be a medium quality category, and the third quality category can be a high quality category, which can be represented by a formula, for example, C=(C1, C2, C3); C represents the quality category sequence, C1 represents the first quality category, C2 represents the second quality category, and C3 represents the third quality category. According to the proportion value, the quality category corresponding to the augmented image is determined from the quality category sequence. This implementation can include: if the proportion value T is less than or equal to a first preset threshold T1, i.e. T ≤ T1, the first quality category is determined as the quality category corresponding to the augmented image. If the proportion value T is greater than the first preset threshold T1 and less than or equal to a second preset threshold T2, i.e. T1 < T ≤ T2, the second quality category is determined as the quality category corresponding to the augmented image. If the proportion value T is greater than the second preset threshold T2, i.e. T2 < T, the second quality category is determined as the quality category corresponding to the augmented image. Wherein, the first preset threshold and the second preset threshold can be set according to specific circumstances, for example: the first preset threshold is set to 40% or 50%, and the second preset threshold is set to 70% or 80% and the like.

[0064] The second implementation of the above data labeling can label the sample images in the initial sample library of the user, which can include:

[0065] Step S115: Obtain the sample image in the initial sample library.

[0066] The implementation of step S115 can be, for example, determining whether the number of sample images corresponding to the quality categories in the initial sample library is balanced. Specifically, determining whether the difference between the number of sample images of the high-quality category and the number of sample images of the low-quality category in the initial sample library is greater than a preset threshold. If the difference between the number of sample images of the high-quality category and the number of sample images of the low-quality category is greater than the preset threshold, the quality category with fewer sample images of the high-quality category and the low-quality category can be expanded (also known as image enhancement), and an expanded initial sample library is obtained. Then, the sample image in the expanded initial sample library is obtained. The preset threshold can be set according to specific conditions, for example, the preset threshold can be set to 10 or 20, etc.

[0067] Step S116: Determine the quality category corresponding to the sample image from the quality category sequence.

[0068] The implementation principle and implementation manner of step S116 are similar to those of step S113, and therefore, the implementation principle and implementation manner of step S116 are not described here. If there is any unclear place, please refer to the description of step S113.

[0069] In the implementation process of the above scheme, the sample expansion function provided in the general framework is used to expand the sample image, and the expanded image is added to the plurality of sample images, thereby avoiding the problem of data imbalance of the plurality of sample images, and effectively improving the applicability and robustness of using the general framework to evaluate the image quality.

[0070] Please refer to Figure 3 The flowchart of using the quality evaluation model to perform quality evaluation provided by the embodiment of the application is shown. As an optional implementation manner of the quality evaluation model training method, after obtaining the quality evaluation model, the quality evaluation model can also be used to perform quality evaluation. This implementation manner can include:

[0071] Step S210: Obtain the image to be processed.

[0072] The image to be processed refers to an image that needs to be evaluated, for example, a vein image, a fingerprint image, a face image, a video image, and an infrared image, etc.

[0073] The implementation of the step S210 includes but is not limited to: a first obtaining manner, using a terminal device such as a camera, a video recorder or a color camera to capture the target object to obtain the to-be-processed image; then the terminal device sends the to-be-processed image to the electronic device, and then the electronic device receives the to-be-processed image sent by the terminal device, and the electronic device can store the to-be-processed image into a file system, a database or a mobile storage device; a second obtaining manner, obtaining a pre-stored to-be-processed image, for example, obtaining the to-be-processed image from the file system, the database or the mobile storage device; and a third obtaining manner, using a browser or other application to obtain the to-be-processed image on the Internet.

[0074] Step S220: performing quality evaluation on the to-be-processed image using the quality evaluation model to obtain a quality evaluation result of the to-be-processed image.

[0075] The implementation of the step S220 includes but is not limited to: using the quality evaluation model to determine the quality category corresponding to the to-be-processed image from the quality category sequence, so as to complete the quality evaluation on the to-be-processed image, and determine the quality category corresponding to the to-be-processed image as the quality evaluation result. Specifically, the quality category corresponding to the to-be-processed image can be determined from a quality category sequence C=(C1, C2) of two quality categories, or the quality category corresponding to the to-be-processed image can be determined from a quality category sequence C=(C1, C2, C3) of three quality categories.

[0076] Please refer to Figure 4 The embodiment of the application provides a structure schematic diagram of the quality evaluation model training device; the embodiment of the application provides a quality evaluation model training device 300, which comprises:

[0077] An image data labeling module 310 is configured to label data of a plurality of sample images using a data labeling function provided in the general framework to obtain a plurality of sample labels, and the sample label is an image quality category corresponding to the sample image.

[0078] A network model training module 320 is configured to train a neural network model provided in the general framework using the plurality of sample images as training data and using the plurality of sample labels as training labels to obtain a quality evaluation model, and the quality evaluation model is configured to perform quality evaluation on a to-be-processed image.

[0079] Optionally, in the embodiment of the application, the quality evaluation model training device further comprises:

[0080] An expanded image obtaining module is configured to expand the sample image using a sample expansion function provided in the general framework to obtain an expanded image.

[0081] The augmented image adding module is configured to add the augmented image into the plurality of sample images.

[0082] Optionally, in the embodiment of the present application, the general framework further comprises: a quality category sequence; and the quality evaluation model training apparatus further comprises:

[0083] The quality category determining module is configured to determine the quality category corresponding to the augmented image from the quality category sequence.

[0084] The quality category adding module is configured to add the quality category corresponding to the augmented image into the plurality of sample labels.

[0085] Optionally, in the embodiment of the present application, the quality category determining module comprises:

[0086] The similarity calculating submodule is configured to calculate the similarity between the augmented image and a homologous sample image in the plurality of sample images, the homologous sample image being a different sample image of the same target object.

[0087] The proportion value calculating submodule is configured to calculate a proportion value between the similarity and an average similarity of the plurality of sample images.

[0088] The quality category determining submodule is configured to determine the quality category corresponding to the augmented image from the quality category sequence according to the proportion value.

[0089] Optionally, in the embodiment of the present application, the quality category sequence comprises: a first quality category and a second quality category, the first quality category being lower than the second quality category; and the quality category determining submodule comprises:

[0090] The preset threshold judging unit is configured to judge whether the proportion value is less than a preset threshold.

[0091] The first category determining unit is configured to determine the first quality category as the quality category corresponding to the augmented image if the proportion value is less than the preset threshold.

[0092] Optionally, in the embodiment of the present application, the quality category determining submodule further comprises:

[0093] The second category determining unit is configured to determine the second quality category as the quality category corresponding to the augmented image if the proportion value is greater than or equal to the preset threshold.

[0094] Optionally, in the embodiment of the present application, the quality evaluation model training apparatus further comprises:

[0095] The image quality evaluation module is configured to perform quality evaluation on the to-be-processed image using the quality evaluation model to obtain a quality evaluation result of the to-be-processed image.

[0096] It should be understood that the device corresponds to the quality evaluation model training method embodiments described above, and can perform each step involved in the above method embodiments. The specific functions of the device can be referred to the description above. To avoid repetition, the detailed description is appropriately omitted here. The device includes at least one software function module stored in the memory in the form of software or firmware or solidified in the operating system (OS) of the device.

[0097] Please refer to Figure 5 The electronic device provided by the embodiments of the present application is shown in the structural schematic diagram. The electronic device 400 provided by the embodiments of the present application includes a processor 410 and a memory 420. The memory 420 stores machine readable instructions executable by the processor 410. When the machine readable instructions are executed by the processor 410, the method described above is performed.

[0098] The embodiments of the present application also provide a computer readable storage medium 430, which stores a computer program. When the computer program is run by the processor 410, the method described above is performed.

[0099] Among them, the computer readable storage medium 430 can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0100] It should be noted that each embodiment in the specification adopts a progressive manner for description, and each embodiment focuses on the difference from other embodiments. The same and similar parts between each embodiment can be referred to each other. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0101] It should be understood that all the apparatus and methods disclosed in the embodiments of the present application can be implemented by other ways. The embodiments described above are only schematic. The use of the terms "include", "comprise", "comprising", "have", "having", "contain", "containing", "include”, “comprise”, “comprising”, “contain” or “containing” etc. should not be construed as being limited to a position of the described features that they are described herein to exclude other elements or limit them to the position stated herein in this context. The above-described apparatus and methods only cover some of the embodiments of the present application. However, it is readily apparent to a person skilled in the art that the present application is not limited thereto. Any modifications and changes within the spirit of the present application will be covered by the appended claims.

[0102] In addition, each functional module in each of the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. In addition, in the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0103] In this paper, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations.

[0104] The above description is only an optional implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered by the protection scope of the present application.

Claims

1. A method for training a quality assessment model, characterized in that, The method comprises the following steps: annotating a plurality of sample images using a data annotation function provided in a general framework to obtain a plurality of sample labels, the sample labels being image quality categories corresponding to the sample images; training a neural network model provided in the general framework using the plurality of sample images as training data and the plurality of sample labels as training labels to obtain a quality evaluation model, the quality evaluation model being used for quality evaluation of a to-be-processed image; before the step of annotating the plurality of sample images using the data annotation function provided in the general framework, the method further comprises the following steps: augmenting a sample image using a sample augmentation function provided in the general framework to obtain an augmented image; adding the augmented image to the plurality of sample images; the general framework further comprises a quality category sequence; after the step of obtaining the augmented image, the method further comprises the following steps: determining a quality category corresponding to the augmented image from the quality category sequence; adding the quality category corresponding to the augmented image to the plurality of sample labels; the step of determining the quality category corresponding to the augmented image from the quality category sequence comprises the following steps: calculating a similarity degree between the augmented image and a homologous sample image in the plurality of sample images, the homologous sample image being a different sample image of a same target object; calculating a ratio value between the similarity degree and an average similarity of the plurality of sample images; wherein the similarity between the augmented image and the homologous sample image in the plurality of sample images is calculated using the formula wherein the similarity between the augmented image and the homologous sample image in the plurality of sample images is calculated using the formula According to the formula The average similarity of the plurality of sample images is calculated; in, Indicates the first j The first expanded image and the first sample image in multiple sample images i The degree of similarity between images of the same category and origin. Indicates the first i The first class of homologous sample images j Image features of an augmented image Indicates the first i The first class of homologous sample images k Image features, The distance function represents the distance between the two. Represents the first of multiple sample images i In the same category of homologous sample images, except for the first j The average similarity of the sample images other than the augmented image. Indicates the first k The first expanded image and the first sample image in multiple sample images i The degree of similarity between images of the same category and origin. Indicates the first i The number of sample images in a class of homologous sample images; determining the quality category corresponding to the augmented image from the quality category sequence according to the ratio value.

2. The method of claim 1, wherein, The quality category sequence comprises a first quality category and a second quality category, the first quality category being lower than the second quality category; the step of determining the quality category corresponding to the augmented image from the quality category sequence according to the ratio value comprises the following steps: determining whether the ratio value is less than a preset threshold value; if yes, determining the first quality category as the quality category corresponding to the augmented image.

3. The method of claim 2, wherein, after the step of determining whether the ratio value is less than the preset threshold value, the method further comprises the following steps: if the ratio value is greater than or equal to the preset threshold value, determining the second quality category as the quality category corresponding to the augmented image.

4. The method according to any of claims 1 to 3, characterized in that, after the step of obtaining the quality evaluation model, the method further comprises the following step: performing quality evaluation on the to-be-processed image using the quality evaluation model to obtain a quality evaluation result of the to-be-processed image. 5.A quality assessment model training apparatus, characterized by, The method comprises the following steps: an image data annotation module is configured to annotate a plurality of sample images using a data annotation function provided in a general framework to obtain a plurality of sample labels, the sample labels being image quality categories corresponding to the sample images; before the step of annotating the plurality of sample images using the data annotation function provided in the general framework, the method further comprises the following steps: augmenting a sample image using a sample augmentation function provided in the general framework to obtain an augmented image; adding the augmented image to the plurality of sample images; the general framework further comprises a quality category sequence; after the step of obtaining the augmented image, the method further comprises the following steps: determining a quality category corresponding to the augmented image from the quality category sequence; adding the quality category corresponding to the augmented image to the plurality of sample labels; The determining the quality category corresponding to the augmented image from the quality category sequence comprises: calculating a similarity degree between the augmented image and a homologous sample image in the plurality of sample images, the homologous sample image being a different sample image belonging to a same target object; calculating a ratio value between the similarity degree and an average similarity of the plurality of sample images; wherein the similarity between the augmented image and the homologous sample image in the plurality of sample images is calculated using the formula wherein the similarity between the augmented image and the homologous sample image in the plurality of sample images is calculated using the formula According to the formula The average similarity of the plurality of sample images is calculated; in, Indicates the first j The first expanded image and the first sample image in multiple sample images i The degree of similarity between images of the same category and origin. Indicates the first i The first class of homologous sample images j Image features of an augmented image Indicates the first i The first class of homologous sample images k Image features, The distance function represents the distance between the two. Represents the first of multiple sample images i In the same category of homologous sample images, except for the first j The average similarity of the sample images other than the augmented image. Indicates the first k The first expanded image and the first sample image in multiple sample images i The degree of similarity between images of the same category and origin. Indicates the first i The number of sample images in a class of homologous sample images; determining the quality category corresponding to the augmented image from the quality category sequence according to the ratio value; a network model training module, configured to take the plurality of sample images as training data, take the plurality of sample labels as training labels, train a neural network model arranged in the general framework, and obtain a quality evaluation model, the quality evaluation model being configured to perform quality evaluation on a to-be-processed image.

6. An electronic device, comprising: comprise: a processor and a memory, the memory storing machine readable instructions executable by the processor, the machine readable instructions being executed by the processor to perform the method of any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the method of any one of claims 1 to 4.

Images