Image quality assessment method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide an image quality assessment method, device, electronic device and storage medium, which relate to the fields of cloud technology and artificial intelligence technology. The method includes: obtaining an image to be evaluated; determining the parameter value of each image quality assessment parameter corresponding to the image to be evaluated; obtaining the evaluation threshold of each image quality assessment parameter; and determining the quality assessment result of the image to be evaluated based on each parameter value and each evaluation threshold. In the technical solution of the present application, the evaluation threshold of each image quality assessment parameter is determined by the parameter value of each test sample image corresponding to the image quality assessment parameter and the image quality category label of each test sample image, thereby avoiding the problem of wasting a lot of testers' time and high testing costs caused by releasing test products multiple times to finally determine the evaluation threshold through manual testing, thereby improving the efficiency of image quality assessment.

Description

Technical Field

[0001] The present application relates to the fields of cloud technology and artificial intelligence technology. Specifically, the present application relates to an image quality assessment method, device, electronic device and storage medium. Background Art

[0002] With the development of computer technology and communication technology, image recognition technology has been widely used in various fields. For example, using a camera or a webcam to capture images containing human faces, and then confirming the user's identity by detecting and recognizing the face.

[0003] The accuracy of image recognition results depends heavily on the quality of the image being recognized. Therefore, before performing image recognition, image quality assessment is necessary. Existing techniques typically require publishing images to a test environment, where they are then repeatedly tested by internal testers and rejected as substandard. This wastes significant tester time, increases testing costs, and inefficiencies in image quality assessment. Summary of the Invention

[0004] The embodiments of the present application provide an image quality assessment method, device, electronic device, and storage medium. Based on this solution, the efficiency of image quality assessment can be effectively improved.

[0005] To achieve the above objectives, the specific technical solutions provided in the embodiments of the present application are as follows:

[0006] In one aspect, an embodiment of the present application provides an image quality assessment method, the method comprising:

[0007] Obtaining the image to be evaluated;

[0008] Determining parameter values of the image to be evaluated corresponding to each image quality evaluation parameter;

[0009] Obtaining an evaluation threshold value of each image quality evaluation parameter;

[0010] Determine a quality assessment result of the image to be assessed based on each parameter value and each assessment threshold;

[0011] For any image quality assessment parameter, the assessment threshold corresponding to the image quality assessment parameter is determined by:

[0012] Obtain a test data set corresponding to the image quality assessment parameter, the test data set including each test sample image and an image quality category label of each test sample image;

[0013] Obtaining parameter values corresponding to image quality assessment parameters for each test sample image;

[0014] An evaluation threshold of the image quality evaluation parameter is determined based on the parameter value of each test sample image corresponding to the image quality evaluation parameter and each image quality category label.

[0015] On the other hand, an embodiment of the present invention further provides an image quality assessment device, the device comprising:

[0016] An image acquisition module, used to acquire an image to be evaluated;

[0017] A parameter determination module, used to determine parameter values corresponding to various image quality assessment parameters of the image to be assessed;

[0018] A threshold acquisition module is used to obtain the evaluation threshold of each image quality evaluation parameter;

[0019] A result determination module, configured to determine a quality assessment result of the image to be assessed based on each parameter value and each assessment threshold;

[0020] For any image quality assessment parameter, the assessment threshold corresponding to the image quality assessment parameter is determined by:

[0021] Obtain a test data set corresponding to the image quality assessment parameter, the test data set including each test sample image and an image quality category label of each test sample image;

[0022] Obtaining parameter values corresponding to image quality assessment parameters for each test sample image;

[0023] An evaluation threshold of the image quality evaluation parameter is determined based on the parameter value of each test sample image corresponding to the image quality evaluation parameter and each image quality category label.

[0024] An embodiment of the present invention also provides an electronic device, which includes one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and the one or more computer programs are configured to execute the method provided in the above-mentioned image quality assessment method or various optional implementations of the image quality assessment method of the present application.

[0025] An embodiment of the present invention also provides a computer-readable storage medium, which is used to store a computer program. When the computer program runs on a processor, the processor can execute the method provided in the above-mentioned image quality assessment method or various optional implementations of the image quality assessment method of the present application.

[0026] An embodiment of the present invention further provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned image quality assessment method or the methods provided in various optional implementations of the image quality assessment method.

[0027] The beneficial effects of the technical solution provided by this application are:

[0028] The present application provides an image quality assessment method, apparatus, electronic device and storage medium, which determine the evaluation threshold of the image quality assessment parameter by comparing the parameter value of each test sample image in a test data set with each image quality assessment parameter and each image quality category label. Based on the parameter value of each image quality assessment parameter and each evaluation threshold, the quality of the image to be assessed is assessed, thereby avoiding the problem of wasting a lot of testers' time and high testing costs caused by releasing test products multiple times to finally determine the evaluation threshold through manual testing, thereby improving the efficiency of image quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0030] Figure 1 A flowchart of an image quality assessment method provided in an embodiment of the present application;

[0031] Figure 2 A schematic diagram of a face recognition model provided in an embodiment of the present application processing an image to be evaluated;

[0032] Figure 3 A schematic diagram of a user interface provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of the face detection prompt information provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of the face detection prompt information provided in an embodiment of the present application;

[0035] Figure 6 A schematic diagram of the face detection prompt information provided in an embodiment of the present application;

[0036] Figure 7 A schematic diagram of the processing process of the multi-stage mass filter provided in an embodiment of the present application;

[0037] Figure 8A schematic diagram of a process for determining the evaluation thresholds of various quality evaluation parameters and an image quality evaluation process provided in an embodiment of the present application;

[0038] Figure 9 A schematic diagram of the structure of an image quality assessment device provided in an embodiment of the present application;

[0039] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0041] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0042] The embodiments of the present application are directed to the problem in the prior art that image quality assessment usually requires publishing images to a test environment, performing multiple tests by internal testers, and returning images with substandard quality, which wastes a lot of testers' time, results in high testing costs, and low efficiency in image quality assessment. The image quality assessment method provided by the embodiments of the present application determines the assessment thresholds of the image quality assessment parameters by comparing the parameter values of the image quality assessment parameters and the image quality category labels corresponding to each test sample image in the test data set. Based on the parameter values of the image quality assessment parameters and the assessment thresholds, the quality of the image to be assessed is assessed, thus avoiding the problem of wasting a lot of testers' time and high testing costs caused by manually releasing test products multiple times to finally determine the assessment thresholds, thereby improving the efficiency of image quality assessment.

[0043] The implementation subject of the technical solution of this application is a computer device, including but not limited to servers, personal computers, laptops, tablet computers, smartphones, etc. Computer devices include user devices and network devices. Among them, user devices include but are not limited to computers, smartphones, PADs, etc.; network devices include but are not limited to a single network server, a server group consisting of multiple network servers, or a cloud composed of a large number of computers or network servers in cloud computing. Among them, cloud computing is a type of distributed computing, which is a super virtual computer composed of a group of loosely coupled computers. Among them, the computer device can run alone to implement this application, or it can be connected to a network and implement this application through interactive operations with other computer devices in the network. Among them, the network where the computer device is located includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, VPN network, etc.

[0044] The solutions provided in the embodiments of this application relate to cloud technology, big data, artificial intelligence and other fields in computer technology.

[0045] The data processing involved in the embodiments of the present application can be achieved through cloud technology, and the data calculation involved can be achieved through cloud computing in cloud technology.

[0046] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network that provides these resources is called the "cloud." To users, these resources appear infinitely scalable and can be accessed at any time, used on demand, expanded at any time, and paid for on a per-use basis.

[0047] As a provider of cloud computing infrastructure, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose and use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0048] Based on logical functional divisions, the PaaS (Platform as a Service) layer can be deployed on top of the IaaS (Infrastructure as a Service) layer, and the SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. SaaS can also be deployed directly on top of IaaS. PaaS is a platform for software execution, such as databases and web containers. SaaS is a variety of business software, such as web portals and text messaging apps. Generally speaking, SaaS and PaaS are layers above IaaS.

[0049] Cloud computing refers to the delivery and usage model of IT infrastructure, enabling on-demand, scalable access to required resources over the internet. In a broader sense, cloud computing refers to the delivery and usage model of services, enabling on-demand, scalable access to required services over the internet. These services can be IT-related, software-related, internet-related, or other services. Cloud computing is the product of the convergence of traditional computer and network technologies, including grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0050] Cloud computing has rapidly grown, driven by the internet, real-time data streams, the diversification of connected devices, and the growing demand for search services, social networks, mobile commerce, and open collaboration. Unlike previous parallel and distributed computing approaches, the emergence of cloud computing will fundamentally revolutionize the entire internet and enterprise management model.

[0051] The face recognition model involved in the embodiments of the present application can be implemented through machine learning in artificial intelligence technology.

[0052] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0053] Artificial intelligence technology is a comprehensive discipline covering a wide range of fields, encompassing both hardware and software technologies. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. The AI technologies involved in the embodiments of this application primarily encompass machine learning and deep learning.

[0054] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0055] The test sample images involved in the embodiments of the present application may be big data obtained from the Internet.

[0056] Big data refers to collections of data that cannot be captured, managed, and processed within a specific timeframe using conventional software tools. These massive, rapidly growing, and diverse information assets require new processing models to enhance decision-making, insight discovery, and process optimization. With the advent of the cloud era, big data has attracted increasing attention. Big data requires specialized technologies to effectively process large amounts of time-sensitive data. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the internet, and scalable storage systems.

[0057] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0058] The embodiment of the present application provides an image quality assessment method. The execution subject of the method can be any electronic device. Optionally, the execution subject of the method can be a server of an application with an image recognition function. The server can recognize the quality of the image to be recognized based on the method provided by the embodiment of the present application, and can process the image to be recognized according to the recognition result. For example, the above-mentioned image recognition function can be a face recognition function. The server can judge whether the image quality of the face image to be recognized meets the requirements through the parameter value of the image quality assessment parameter. When the quality of the image to be recognized meets the requirements, the subsequent face recognition process can be performed on the face image to be recognized, such as Figure 1 As shown, the method may include:

[0059] Step S101, obtaining an image to be evaluated;

[0060] The image to be evaluated may be any image that requires image quality evaluation, for example, an image containing a human face.

[0061] Specifically, the image to be evaluated may be acquired by an image acquisition device of the user terminal, for example, by acquiring a user's facial image through a camera; or, the image to be evaluated may be acquired from a preset storage space.

[0062] Step S102, determining the parameter values of the image to be evaluated corresponding to each image quality evaluation parameter;

[0063] Among them, the image quality assessment parameters are parameters used to evaluate the quality of the image to be evaluated, such as image clarity, image brightness, etc. The parameter value of the image quality assessment parameter indicates the degree of quality acceptance or rejection of the image to be evaluated in the evaluation dimension corresponding to the image quality assessment parameter. There can be at least one image quality assessment parameter, and the quality of the image to be evaluated can be evaluated from at least one evaluation dimension using the quality assessment parameter. The specific dimensions from which the quality of the image to be evaluated is evaluated, that is, the specific image quality assessment parameters selected, can be set according to specific needs and are not limited in this application.

[0064] After determining each image quality assessment parameter of the image to be assessed, parameter values corresponding to each image quality assessment parameter are calculated for the image to be assessed. Optionally, the parameter values of each image quality assessment parameter can be obtained using a neural network model. Optionally, image features of the image to be assessed can also be obtained using the neural network model, and the parameter values of the image quality assessment parameters are calculated based on the obtained image features.

[0065] In different application scenarios, the content of the image to be evaluated is different. The corresponding image quality evaluation parameters are determined according to the content of the image to be evaluated. See the following embodiments for details:

[0066] In one possible implementation, the image to be evaluated includes a face image, and the image quality evaluation parameter includes at least one of the following:

[0067] Face confidence, face tilt, face offset, proportion of face area in face image, resolution of face image, clarity of face image, brightness of face image.

[0068] In face recognition applications, facial image quality assessment is required. Image quality assessment parameters include at least one of the following: face confidence, face tilt, face offset, face area ratio in the face image, face image resolution, face image clarity, and face image brightness.

[0069] In some optional embodiments, the parameter value of the face confidence score can be obtained by a face recognition model. A face image is input into the face recognition model, and the face confidence score (also called category confidence score) output by the face recognition model is obtained, that is, the probability value of the image to be recognized being a face image. In addition, the face recognition model can also output the location of the facial region in the image and the coordinates of key facial points (e.g., left eye, right eye, nose, left corner of the mouth, right corner of the mouth, etc.). The face recognition model can be any neural network model, such as a convolutional neural network model. The specific structure of the face recognition model is not limited in this application.

[0070] In one example, if Figure 2As shown, the image to be evaluated is input into the face recognition model, and is processed by the various processing layers of the face recognition model (the various rectangles shown in the figure, each rectangle represents a processing layer), and the face confidence (the category confidence as shown in the figure) is output, that is, the probability value that the image to be recognized is a face image. The face recognition model also outputs the position box of the face area in the image (the face box as shown in the figure), and the coordinates of the key points of the face (for example, the left eye, right eye, nose, left corner of the mouth, right corner of the mouth, etc.). The output image contains the marked positions of the face confidence (0.999 as shown in the figure), the face box (the black rectangular box corresponding to the face as shown in the figure), and the key points of the face (the marked points corresponding to the left eye, right eye, nose, left corner of the mouth, and right corner of the mouth as shown in the figure), as shown in the figure. Figure 2 shown.

[0071] In one example, the face inclination can be calculated using the coordinates of key facial points. By calculating the eye inclination, mouth inclination, and nose deviation, these three indicators can be used to comprehensively represent the face inclination.

[0072] The eye tilt can be calculated using the following formula (1):

[0073] (1)

[0074] Where eye_gradient represents the eye gradient, which is the slope of the line connecting the two eyes. eye_left_y represents the ordinate of the center point of the left eye; eye_right_y represents the ordinate of the center point of the right eye; eye_left_x represents the abscissa of the center point of the left eye; and eye_right_x represents the abscissa of the center point of the right eye.

[0075] The mouth inclination can be calculated using the following formula (2):

[0076] (2)

[0077] Where mouth_gradient represents the inclination of the mouth, which is the inverse of the slope between the center of the eyes and the center of the mouth. eye_left_y represents the y-coordinate of the center of the left eye; eye_right_y represents the y-coordinate of the center of the right eye; eye_left_x represents the abscissa of the center of the left eye; and eye_right_x represents the abscissa of the center of the right eye. mouth_left_y represents the y-coordinate of the center of the left corner of the mouth; mouth_right_y represents the y-coordinate of the center of the right corner of the mouth; mouth_left_x represents the abscissa of the center of the left corner of the mouth; and mouth_right_x represents the abscissa of the center of the right corner of the mouth.

[0078] Nose offset is calculated as follows:

[0079] First, calculate the center positions of the two eyes using formula (3):

[0080] (3)

[0081] in, Indicates the positions of the center points of the two eyes; eye0[0] indicates the horizontal coordinate of the center point of the left eye; eye1[0] indicates the horizontal coordinate of the center point of the right eye; eye0[1] indicates the vertical coordinate of the center point of the left eye; eye1[1] indicates the vertical coordinate of the center point of the right eye.

[0082] The center point of the mouth is calculated using formula (4):

[0083] (4)

[0084] Among them, mouse_middle represents the position of the center point of the mouth; mouse0[0] represents the horizontal coordinate of the center point of the left corner of the mouth; mouse1[0] represents the horizontal coordinate of the center point of the right corner of the mouth; mouse0[1] represents the vertical coordinate of the center point of the right corner of the mouth; mouse1[1] represents the vertical coordinate of the center point of the right corner of the mouth.

[0085] The inverse of the slope of the straight line between the center points of the two eyes and the center point of the mouth is calculated using formula (5):

[0086] (5)

[0087] in, It represents the inverse of the slope of the straight line between the center points of the two eyes and the center point of the mouth; eye_middle[0] represents the horizontal coordinate of the center points of the two eyes; mouse_middle[0] represents the horizontal coordinate of the center point of the mouth; eye_middle[1] represents the vertical coordinate of the center points of the two eyes; mouse_middle[1] represents the vertical coordinate of the center point of the mouth.

[0088] The offset of the straight line between the center points of the two eyes and the center point of the mouth is calculated using formula (6): (6)

[0089] Among them, mouse_eye_b_reverse represents the offset of the straight line between the center points of the two eyes and the center point of the mouth; eye_middle[0] represents the horizontal coordinate of the center points of the two eyes; mouse_middle[0] represents the horizontal coordinate of the center point of the mouth; eye_middle[1] represents the vertical coordinate of the center points of the two eyes; mouse_middle[1] represents the vertical coordinate of the center point of the mouth.

[0090] The horizontal coordinate of the intersection of the nose horizontal line and the midline is calculated by formula (7):

[0091] (7)

[0092] Among them, nose_x_head represents the horizontal coordinate of the intersection of the horizontal line and the midline of the nose; mouse_eye_slope_reverse represents the inverse of the slope of the straight line between the center points of the two eyes and the center point of the mouth; mouse_eye_b_reverse represents the offset of the straight line between the center points of the two eyes and the center point of the mouth; nose[1] represents the vertical coordinate of the center point of the nose.

[0093] The deviation of the nose is calculated by formula (8):

[0094] (8)

[0095] Among them, delt_rate represents the deviation of the nose; nose_x_head represents the horizontal coordinate of the intersection of the horizontal line and the midline of the nose; nose[0] represents the horizontal coordinate of the center point of the nose; eye0[0] represents the horizontal coordinate of the center point of the left eye; eye1[0] represents the horizontal coordinate of the center point of the right eye.

[0096] In summary, the inclination of a face includes values of three dimensions: eye inclination, mouth inclination, and nose deviation. The evaluation threshold corresponding to the inclination of a face includes the evaluation threshold corresponding to each of the above three dimensions.

[0097] The face offset is determined by the position of the face area in the face image, that is, the distance between the face position box and the upper boundary, lower boundary, left boundary, and right boundary of the face image, as shown in formulas (9)-(12):

[0098]

[0099] Among them, top_distance represents the distance from the upper boundary of the face location box to the upper boundary of the face image; bottom_distance represents the distance from the face location box to the lower boundary of the face image; left_distance represents the distance from the face location box to the left boundary of the face image; right_distance represents the distance from the face location box to the right boundary of the face image; face_box_ymin represents the minimum vertical coordinate of the face location box; image_height represents the height of the face image, that is, the maximum vertical coordinate of the face image; face_box_ymax represents the maximum vertical coordinate of the face location box; face_box_xmin represents the minimum horizontal coordinate of the face location box; image_width represents the width of the face image, that is, the maximum horizontal coordinate of the face image; face_box_xmax represents the maximum horizontal coordinate of the face location box. Among them, face_box_ymin, image_height, face_box_ymax, face_box_xmin, face_box_xmax, and image_width can be output by the face recognition model.

[0100] The proportion of the face area in the face image is calculated using the following formula (13):

[0101] (13)

[0102] Among them, face_area_ratio represents the ratio of the area of the face location box to the area of the face image; face_xmin represents the horizontal coordinate of the upper left corner of the face location box in the face image. face_ymin represents the vertical coordinate of the upper left corner of the face location box in the face image. face_xmax represents the horizontal coordinate of the lower right corner of the face location box in the face image. face_ymax represents the vertical coordinate of the lower right corner of the face location box in the face image. image_xmin represents the minimum value of the horizontal coordinate of the face image, which can be equal to 0. image_ymin represents the minimum value of the vertical coordinate of the face image, which can be equal to 0. image_xmax represents the maximum value of the horizontal coordinate of the face image, which is equal to the width of the face image. image_ymax represents the maximum value of the vertical coordinate of the face image, which is equal to the width of the face image.

[0103] The resolution of the face image is calculated using the following formula (14):

[0104] (14)

[0105] Among them, resolution represents the resolution of the face image; image_width represents the number of pixels corresponding to the width of the face image; image_height represents the number of pixels corresponding to the height of the face image.

[0106] In one possible implementation, the clarity of the facial image is determined by:

[0107] Obtaining the global clarity of the face image and the local clarity of the face area in the face image;

[0108] The clarity of the face image is obtained by fusing the global clarity and local clarity;

[0109] The brightness of the face image is determined in the following way:

[0110] Obtaining the global brightness of the face image and the local brightness of the face area in the face image;

[0111] The brightness of the face image is obtained by fusing the global brightness and local brightness.

[0112] In practical applications, the clarity of a facial image can be obtained by fusing the global clarity of the facial image with the local clarity of the facial region within the facial image. Alternatively, the mean square error of the global and local clarity values can be used as the clarity of the facial image. The local clarity of the facial region refers to the clarity of the image within the facial location box output by the facial recognition model.

[0113] In one example, the global clarity of a face image is calculated using the following formula (15):

[0114] (15)

[0115] Where definition(image) represents the global definition of the face image; image_gray represents the two-dimensional matrix of the grayscale image after grayscale processing of the face image; image_gray_mean represents the average value of each grayscale value in the grayscale two-dimensional matrix; mean() represents the mean value. The local definition of the face region can also be calculated using formula (15), replacing the parameter values with the parameter values of the face region image.

[0116] The clarity of a face image can be calculated using the following formula (16):

[0117] (16)

[0118] Among them, definition represents the clarity of the face image; definition(image) represents the global clarity of the face image; definition(face) represents the local clarity of the face area.

[0119] In this embodiment, the mean square error of the local clarity of the facial area and the global clarity of the facial image is used as the clarity of the facial image. The clarity of the facial image tends to be the larger value between the local clarity of the facial area and the global clarity of the facial image. The clarity of the facial image is obtained by fusion, which avoids the problem of inaccurate clarity calculation of the facial image caused by single factors such as the single background color of the facial image or facial makeup.

[0120] In one example, the global brightness of a face image is calculated using the following formula (17):

[0121] (17)

[0122] Where light(image) represents the global brightness of the face image, which can be expressed as the average grayscale value; image_gray represents the two-dimensional matrix of the grayscale image of the face image; and mean() represents the mean of image_gray. The local brightness of the face region can also be calculated using formula (17), replacing the parameter values with the parameter values of the face region image.

[0123] The brightness of a face image can be calculated using the following formula (18):

[0124] (18)

[0125] Among them, light represents the brightness of the face image; light(face) represents the local brightness of the face area; light(image) represents the global brightness of the face image.

[0126] In this embodiment, the brightness of the facial image is calculated using the mean square error of the local brightness of the facial area and the global brightness of the facial image. The brightness of the facial image is obtained by fusion, avoiding the problem of inaccurate brightness calculation of the facial image due to the facial area being too dark or the background being too dark.

[0127] Step S103, obtaining the evaluation threshold of each image quality evaluation parameter;

[0128] Specifically, the evaluation threshold of each image quality evaluation parameter may be predetermined and stored. When performing quality evaluation on the image to be evaluated, the evaluation threshold corresponding to each image quality evaluation parameter may be directly called from the preset storage space.

[0129] Optionally, a JavaScript Object Notation (JSON) format file is used to store the evaluation thresholds for each quality assessment parameter. The evaluation thresholds for each quality assessment parameter are stored in a configuration file using JSON format. When performing image quality assessment, the evaluation thresholds for each quality assessment parameter are read from the configuration file. Storing the evaluation thresholds in JSON format facilitates easy access and optimization of the evaluation thresholds at any time.

[0130] Step S104: determining a quality evaluation result of the image to be evaluated based on each parameter value and each evaluation threshold.

[0131] Specifically, the quality assessment conditions corresponding to the image quality assessment parameters of the image to be assessed may be pre-configured. The quality assessment conditions may be configured according to specific needs, and this application does not limit this.

[0132] Optionally, for a certain image quality assessment parameter, the quality assessment condition may be: if the parameter value is greater than or equal to the corresponding assessment threshold, the quality of the image to be assessed is qualified; for another image quality assessment parameter, the quality assessment condition may be: if the parameter value is less than the corresponding assessment threshold, the quality of the image to be assessed is qualified.

[0133] Based on the parameter value and evaluation threshold of each image quality evaluation parameter of the image to be evaluated, and the quality qualification condition corresponding to the image quality evaluation parameter, it is determined whether the image to be evaluated is of qualified quality.

[0134] Optionally, the parameter value of each image quality assessment parameter is evaluated in turn to see whether it meets the quality qualification condition. If the parameter value of one of the image quality assessment parameters does not meet the quality qualification condition, the quality of the image to be assessed is determined to be unqualified. Optionally, if the parameter values of two or more image quality assessment parameters do not meet the quality qualification condition, the quality of the image to be assessed is determined to be unqualified.

[0135] For any image quality assessment parameter, the assessment threshold corresponding to the image quality assessment parameter is determined by:

[0136] A test data set corresponding to the image quality assessment parameter is obtained, where the test data set includes each test sample image and an image quality category label for each test sample image; a parameter value corresponding to the image quality assessment parameter is obtained for each test sample image; based on the parameter value corresponding to the image quality assessment parameter of each test sample image and each image quality category label, the image quality category label represents the true quality category of the test sample image corresponding to the quality assessment parameter; and an evaluation threshold of the image quality assessment parameter is determined.

[0137] Each test sample image may be a sample image selected according to the image content of the image to be evaluated. For example, if the image to be evaluated is a face image, multiple images containing faces may be selected as test sample images.

[0138] Optionally, the test sample images include difficult-to-classify sample images and image quality category labels corresponding to the difficult-to-classify sample images. A difficult-to-classify sample image, also known as a hard sample, refers to a sample of this type that, when performing sample classification, has a small probability of being classified into various categories, is close to a classification boundary, and is therefore difficult to classify into any category.

[0139] For any image quality assessment parameter, each test image is obtained, and the parameter value of each test image corresponding to each image quality assessment parameter is calculated. Optionally, an EXCEL format file can be used to store each test image and the parameter value of the quality assessment parameter of each test image. The EXCEL format file can present each test image and the corresponding parameter value in a visual manner, which is convenient for users to edit. For each image quality assessment parameter, the test images are sorted in order from large to small or from small to large according to the parameter value, and the sorted test images and the corresponding parameter values are presented in a visual manner. The test images with parameter values within a preset range can be determined as sample images that are difficult to classify, that is, test sample images. The test sample images can be labeled through human-computer interaction, and the image quality category labels of each test sample image input by the user are received, thereby obtaining a test data set.

[0140] Optionally, parameter values of various image quality assessment parameters may be obtained through a neural network model.

[0141] Optionally, the image features of the image to be evaluated may be obtained through a neural network model, and the parameter values of the image quality evaluation parameters may be calculated based on the obtained image features.

[0142] For any image quality assessment parameter, the quality classification result of each test sample image can be determined based on the parameter value of each test sample image corresponding to the image quality assessment parameter and the quality category label of each image, including two cases: one is linearly separable; the other is linearly inseparable. According to different types of quality classification results, the evaluation threshold of the image quality assessment parameter is determined through different processing methods.

[0143] In one possible implementation, each test sample image includes a positive sample image and a negative sample image. For any image quality assessment parameter, determining an assessment threshold of the image quality assessment parameter based on the parameter value and the image quality category label of each test sample image includes:

[0144] Determine the quality classification result corresponding to each test sample image based on the parameter value and image quality category label of each test sample image;

[0145] If the quality classification results corresponding to each test sample image are linearly separable, determining an evaluation threshold based on the parameter value of each target test sample image, wherein each target test sample image includes a set of sample images with adjacent parameter values in the test sample image, and the set of sample images with adjacent parameter values includes at least one positive sample image and at least one negative sample image;

[0146] If the quality classification results corresponding to each test sample image are linearly inseparable, then determining the classification intersection area of the positive sample image and the negative sample image based on the parameter value of the image quality assessment parameter corresponding to each test sample image;

[0147] An evaluation threshold is determined based on the parameter value of the image quality evaluation parameter corresponding to the classification intersection area and each image quality category label.

[0148] In practical applications, each test sample image includes a positive sample image and a negative sample image. The positive sample image can be an image of acceptable quality, while the negative sample image can be an image of unacceptable quality. For any image quality assessment parameter, the positive sample image and the negative sample image each have their own corresponding image quality category label. For example, the image quality category label for a positive sample image can be 1, while the image quality category label for a negative sample image can be -1.

[0149] For any image quality assessment parameter, the image quality classification results of the positive sample image and the negative sample image meet any of the following conditions, then they are linearly separable, otherwise they are linearly inseparable:

[0150] In a case where a larger parameter value of the image quality assessment parameter indicates better image quality, a minimum parameter value of the image quality assessment parameter of the positive sample image is greater than a maximum parameter value of the image quality assessment parameter of the negative sample image;

[0151] In the case where the smaller the parameter value of the image quality evaluation parameter, the better the image quality, the maximum parameter value of the image quality evaluation parameter of the positive sample image is smaller than the minimum parameter value of the image quality evaluation parameter of the negative sample image.

[0152] If the image quality classification results of the positive sample image and the negative sample image are linearly separable, at least two target test sample images are selected from the positive sample image and the negative sample image, and the at least two target test sample images include a set of sample images with adjacent parameter values, and the set includes at least one positive sample image and at least one negative sample image.

[0153] Among them, the number of target test sample images determined from the positive sample images can be at least one, the number of target test sample images determined from the negative sample images can also be at least one, and the number of target test sample images determined from the positive sample images and the number of target test sample images determined from the negative sample images can be the same or different.

[0154] Specifically, for any image quality assessment parameter, the parameter values of the test sample images can be sorted from large to small. In one example, the parameter value sorting result is: 0.8, 0.7, 0.65, 0.5, 0.43, where each value corresponds to a sample image, the parameter values corresponding to the positive sample images are 0.8, 0.7, and 0.65, respectively, and the parameter values corresponding to the negative sample images are 0.5 and 0.43, respectively. 0.65 and 0.5 are adjacent parameter values, and include both positive and negative sample images. Therefore, the sample images corresponding to 0.65 and 0.5 can be used as target sample images, and the assessment threshold is determined based on the parameter values of these two target sample images. 0.65, 0.5, and 0.43 are adjacent parameter values, and include both positive and negative sample images. Therefore, the sample images corresponding to 0.65, 0.5, and 0.43 can be used as target sample images, and the assessment threshold is determined based on the parameter values of these three target sample images. 0.8, 0.7, 0.65, and 0.5 are adjacent parameter values, and they include positive sample images and negative sample images. Therefore, the sample images corresponding to 0.8, 0.7, 0.65, and 0.5 can be used as target sample images, and the evaluation threshold is determined according to the parameter values of these four target sample images.

[0155] In the case of linear separability, since the parameter value of the target test sample image is the classification boundary of the positive and negative samples, or is closest to the classification boundary of the positive and negative samples, the evaluation threshold obtained according to the parameter value of the target test sample image is more accurate. Applying this evaluation threshold to the quality assessment of the image to be evaluated will result in a more accurate evaluation result.

[0156] For any image quality assessment parameter, if the image quality classification results of the positive sample image and the negative sample image do not meet the linear separability condition, they are linearly inseparable. Based on the parameter value of the image quality assessment parameter of the positive sample image and the parameter value of the image quality assessment parameter of the negative sample image, a classification intersection region is determined. The classification intersection region is the intersection of the set corresponding to the positive sample image and the set corresponding to the negative sample image. Therefore, the parameter value of the image quality assessment parameter corresponding to the classification intersection region includes the parameter value of the image quality assessment parameter of the positive sample image and also includes the parameter value of the image quality assessment parameter of the negative sample image. The evaluation threshold of the image quality assessment parameter can be determined based on the parameter value of the image quality assessment parameter corresponding to the classification intersection region and each image quality category label.

[0157] In the case of linear inseparability, the parameter value of the image quality assessment parameter corresponding to the classification intersection area of the positive sample image and the negative sample image is closer to the optimal assessment threshold. The evaluation threshold obtained in this way is applied to image quality assessment, which can make the evaluation result of image quality assessment more accurate.

[0158] In one possible implementation, determining an evaluation threshold based on a parameter value of each target sample image includes:

[0159] The evaluation threshold of the image quality evaluation parameter is obtained by fusing the parameter values of each target test sample image corresponding to the image quality evaluation parameter.

[0160] In practical applications, since the parameter values of the target test sample images are at or closest to the classification boundary between positive and negative samples, the evaluation threshold obtained by fusing the parameter values of each target test sample image is more accurate and closer to the optimal evaluation threshold. Applying this evaluation threshold to the quality assessment of the image to be evaluated will result in more accurate evaluation results. The specific method of fusion can be any method and is not limited in this application.

[0161] In one example, the larger the parameter value of the image quality assessment parameter, the better the image quality. The test sample image with the smallest parameter value of the image quality assessment parameter of the positive sample image is used as the first target test sample image, and the test sample image with the largest parameter value of the image quality assessment parameter of the negative sample image is used as the second target test sample image. The average value of the first parameter value of the image quality assessment parameter of the first target test sample image and the second parameter value of the image quality assessment parameter of the second target test sample image are calculated, and the average value is used as the evaluation threshold of the image quality assessment parameter.

[0162] In another example, the smaller the parameter value of the image quality assessment parameter, the better the image quality. The test sample image with the largest parameter value of the image quality assessment parameter of the positive sample image is used as the first target test sample image, and the test sample image with the smallest parameter value of the image quality assessment parameter of the negative sample image is used as the second target test sample image. The average value of the first parameter value of the image quality assessment parameter of the first target test sample image and the second parameter value of the image quality assessment parameter of the second target test sample image are calculated, and the average value is used as the evaluation threshold of the image quality assessment parameter.

[0163] In one possible implementation, determining an evaluation threshold of the image quality evaluation parameter based on the parameter value of the image quality evaluation parameter corresponding to the classification intersection region and each image quality category label includes:

[0164] The parameter values corresponding to the test sample images whose quality classification results are located in the classification intersection area are respectively used as the first candidate evaluation thresholds;

[0165] For each first candidate evaluation threshold, determining a first classification result for each test sample image based on the parameter value of each test sample image in the test data set and the first candidate evaluation threshold;

[0166] For each first candidate evaluation threshold, determining a classification loss value corresponding to the first candidate evaluation threshold based on the first classification result and the image quality category label corresponding to each test sample image;

[0167] The first candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the first candidate evaluation thresholds is used as the evaluation threshold.

[0168] In practical applications, there may be one or more test sample images whose quality classification results are located in the classification intersection area of the positive sample image and the negative sample image. At least one test sample image is selected from them, and the parameter value of the selected test sample image is used as the first candidate evaluation threshold. Based on the first candidate evaluation threshold, the parameter value of each test sample image is quality classified to obtain the first classification result corresponding to each test sample image. Based on the first classification result and the image quality category label corresponding to each test sample image, a classification loss value is determined. The first candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the first candidate evaluation thresholds is used as the final evaluation threshold.

[0169] In one example, the brightness value of the test sample image a1 determined in the classification intersection area is 0.5, and 0.5 is used as the first candidate evaluation threshold. A brightness value greater than 0.5 is a qualified image, and a brightness value less than or equal to 0.5 is an unqualified image. The brightness value of a test sample image is 0.6, then the test sample image is a qualified image, and the classification result corresponding to the qualified image is 1. The brightness value of another test sample image is 0.4, then the test sample image is an unqualified image, and the classification result corresponding to the unqualified image is -1.

[0170] In an embodiment of the present application, a test sample image is determined in a classification intersection area of a positive sample image and a negative sample image. The parameter value of the test sample image in the classification intersection area is closer to the optimal evaluation threshold than the test sample images in other areas. The parameter value of the test sample image in the classification intersection area is used as a candidate evaluation threshold and applied to image quality classification, which can make the evaluation results of image quality evaluation more accurate.

[0171] In one possible implementation, determining an evaluation threshold of the image quality evaluation parameter based on the parameter value of the image quality evaluation parameter corresponding to the classification intersection region and each image quality category label includes:

[0172] determining a parameter value range of an image quality assessment parameter corresponding to a classification intersection region;

[0173] Select at least one value within the parameter value range as a second candidate evaluation threshold;

[0174] For each second candidate evaluation threshold, determining a second classification result for each test sample image based on the parameter value of each test sample image in the test data set and the second candidate evaluation threshold;

[0175] For each second candidate evaluation threshold, determining a classification loss value corresponding to the second candidate evaluation threshold based on the second classification result and the image quality category label corresponding to each test sample image;

[0176] The second candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the second candidate evaluation thresholds is used as the evaluation threshold.

[0177] In practical applications, the parameter value range corresponding to the image quality assessment parameter in the classification intersection region can be a numerical range between the minimum and maximum values of the image quality assessment parameter of the test sample images in the classification intersection region. At least one value within this numerical range can be selected as the second candidate assessment threshold. Optionally, the numerical range can be evenly divided into multiple grids, and the value in each grid is used as the second candidate assessment threshold, thereby obtaining multiple second candidate assessment thresholds. The number of second candidate assessment thresholds can be determined based on specific needs.

[0178] Based on the second candidate evaluation threshold, the parameter values of each test sample image are quality-classified to obtain a second classification result corresponding to each test sample image. A classification loss value is determined based on the second classification result and the image quality category label corresponding to each test sample image. The second candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the second candidate evaluation thresholds is used as the final evaluation threshold.

[0179] In an embodiment of the present application, multiple candidate evaluation thresholds are determined within the parameter value range corresponding to the classification intersection area of the positive sample image and the negative sample image. Compared with determining the candidate evaluation thresholds in the parameter value of the test sample image in the classification intersection area, more candidate evaluation thresholds can be obtained. In this way, the evaluation threshold finally determined from the multiple candidate evaluation thresholds is closer to the optimal evaluation threshold, and its application in image quality assessment can make the evaluation results of the image quality assessment more accurate.

[0180] In one example, the classification loss is calculated by the following formula (19):

[0181] (19)

[0182] Where N represents the number of test sample images, Represents the parameter value of the quality assessment parameter of each test sample image, theta represents the candidate assessment threshold (the first candidate assessment threshold or the second candidate assessment threshold), y represents the absolute value of the difference between the classification result (the first classification result or the second classification result) of each test sample image and the image quality category label when the threshold is theta, and y=abs( ) into formula (1), we get the following formula (20):

[0183] (20)

[0184] Among them, Loss represents the value of classification loss; represents the parameter value of the quality assessment parameter of the i-th test sample image; theta represents the candidate assessment threshold based on this calculation; express The classification results under the evaluation threshold theta, for example, It can be 0 or 1, 0 means the test sample image quality is qualified, 1 means the test sample image quality is unqualified; express The corresponding image quality category label can be 0 or 1, where 0 indicates that the labeled test sample image quality is qualified and 1 indicates that the labeled test sample image quality is unqualified. Each time a candidate evaluation threshold is selected, a loss is calculated, and the candidate evaluation threshold with the smallest loss is selected as the final evaluation threshold.

[0185] In some optional embodiments, if the image quality classification results of the positive sample images and the negative sample images in the difficult-to-classify samples are linearly inseparable, a perceptron (Perception) model, a neural network model, or a logistic regression (LR) model can be used to input the difficult-to-classify sample images and the image quality category labels of each image, determine the loss function, and use the gradient descent method. After multiple iterative training, a trained perceptron model is obtained, and finally an evaluation threshold is obtained.

[0186] In some optional embodiments, if the image quality classification results of the positive sample images and the negative sample images in the difficult-to-classify samples are linearly inseparable, a support vector machine (SVM) model can be used to input the difficult-to-classify sample images and the image quality category labels of each image, determine the loss function, and after multiple iterative training, obtain a trained SVM model, determine the support vector, and calculate the mean of a preset number of support vectors as the evaluation threshold.

[0187] In some optional embodiments, all test sample images (including difficult samples and non-difficult samples) are labeled, and the labeled test sample images are used to train the image quality classification model. After multiple iterative training, the final evaluation threshold is obtained.

[0188] The following is an introduction to the specific application of the technical solution of this application through a specific application scenario.

[0189] The technical solution of this application can be applied to the anti-cheating system of the game "Peace Elite". The anti-cheating system collects photos and recordings uploaded by users, evaluates the quality of the photos, and prompts users to re-upload photos if the quality of the photos is not up to standard. The user interface of the anti-cheating system is as follows: Figure 3 The specific process is as follows:

[0190] When receiving the user's trigger operation for the "Start Game" button, such as Figure 4 As shown, the image acquisition device of the user terminal is started to acquire the user's facial image. The quality assessment process of the technical solution of the present application can be implemented in the form of a multi-stage quality filter. Each level of the multi-stage quality filter evaluates each quality assessment parameter of the user's facial image, and determines the evaluation result of the image to be evaluated based on each parameter value and each evaluation threshold. If the parameter value of one of the quality assessment parameters is unqualified, a corresponding prompt message is displayed.

[0191] like Figure 5 As shown, if the user's face image is not detected in the picture, a prompt message is displayed, such as Figure 5 The message "No face detected, please align your face with the box and take a new photo" is displayed; if a face image is detected, the quality of the face image is evaluated. If the quality of the face image is unqualified, a prompt message is displayed based on the unqualified quality evaluation parameters. If the parameter value of the face deviation does not meet the requirements (for example, the deviation is greater than the evaluation threshold corresponding to the deviation), a prompt message is displayed, such as Figure 5 As shown in the figure, "The face is not centered, please align the face with the frame and take the photo again"; if the parameter value of the face inclination does not meet the requirements (for example, the inclination is greater than the evaluation threshold corresponding to the inclination), a prompt message will be displayed, such as Figure 6 As shown in the figure, "The face is tilted, please take a photo of the front face"; if the face image quality is qualified, it will jump to the following Figure 3 The user interface shown uploads a face image when a user triggers an “upload” button; and exits the anti-cheating system when a user triggers a “logout” button.

[0192] In the embodiment of the present application, the technical solution of the present application is applied to the anti-cheating system of the game, and high-quality images can be collected from the product application level, avoiding the misjudgment of the image detection of the cheating system due to image quality problems, and providing a strong guarantee for anti-cheating in e-sports.

[0193] The specific process of image quality evaluation of the technical solution of this application is introduced below through a specific embodiment.

[0194] like Figure 7 As shown, the image to be evaluated is input into a multi-stage quality filter. Each stage of the multi-stage quality filter evaluates each quality assessment parameter of the user image. Based on the parameter values and the evaluation thresholds, the evaluation result of the image to be evaluated is determined. In other words, the parameter values of each quality assessment parameter of the user image are determined to be qualified (as shown in the figure). If the image quality is qualified, the quality evaluation process ends. If the image quality is unqualified, the reason for the failure is indicated based on the unqualified quality assessment parameter, and the quality evaluation process ends.

[0195] The following describes in detail the process of determining the evaluation thresholds of the quality evaluation parameters and the image quality evaluation process of the technical solution of the present application through a specific embodiment. This embodiment is described by taking a face image as an example.

[0196] like Figure 8 As shown, this embodiment first introduces the evaluation threshold determination process:

[0197] Obtain each test image, calculate the parameter value of each test image corresponding to each image quality assessment parameter. In this embodiment, the parameter value of each test image corresponding to each image quality assessment parameter is calculated by a sample structured script. The specific process includes: traversing and parsing each test image (traversing and parsing the image as shown in the figure), detecting faces in each test image, and calculating the face confidence of each test image (obtaining confidence as shown in the figure), face inclination (calculating inclination as shown in the figure), clarity of the face image (calculating clarity as shown in the figure), brightness of the face image (calculating brightness as shown in the figure), and the area ratio of the face region in the face image (as shown in the figure). The calculated area ratio), the resolution of the face image (the calculated resolution shown in the figure) is calculated and stored, and each quality assessment parameter (the feature dimension as shown in the figure) and each parameter value (the numerical value of each image dimension as shown in the figure) of each test image are stored in an EXCEL file (in a storage table file as shown in the figure), and the parameter value of each quality assessment parameter is sorted (single-dimensional parameter adjustment as shown in the figure). In this embodiment, taking the brightness of the test image as an example, the sorted test images and the brightness values of each test image (5, 7, 8, 11, 85, 85.6, 86, 86, 150, 151, 152 as shown in the figure) are visualized. Test images with brightness values within a certain range are identified as difficult-to-classify sample images (as shown in the figure, the corresponding brightness values of these difficult-to-classify sample images are 85, 85.6, 86, and 86 within the rectangular box). These difficult-to-classify sample images are labeled through human-computer interaction (as shown in the figure, which illustrates the human-computer interaction for calibrating several difficult samples). User-entered image quality category labels are then received and labeled as acceptable or unacceptable. For example, a label of 1 indicates acceptable and -1 indicates unacceptable. Each difficult-to-classify sample image and its corresponding image quality category label is used as the test dataset. Based on the brightness values and image quality category labels of each test sample image, a parameter self-learning script is used to determine the evaluation threshold (as shown in the figure, which involves automatic parameter tuning of the cascade filter). The specific process includes normalizing the parameter values, for example, by converting them to a range of 0 to 1. Based on the parameter values and image quality category labels of each test sample image, image quality classification results are determined for positive samples (acceptable quality images) and negative samples (unacceptable quality images). There are two scenarios: one in which the image quality classification results are linearly separable, and one in which they are not linearly separable. If they are linearly separable, a positive sample image and a negative sample image with adjacent parameter values are selected as target test sample images (find the support points as shown in the figure), and the average value of the parameter values of the quality assessment parameters of each target test sample image is calculated (calculate the support point mean as shown in the figure), and the average value is used as the evaluation threshold of the quality assessment parameter.If the image quality classification results are linearly inseparable, m values are searched for in the intersection of the positive and negative image classifications as candidate evaluation thresholds to determine the classification loss. The classification loss can be a mean square error loss function (designed mean square error loss function as shown in the figure). Based on each candidate evaluation threshold, each test sample image is classified to obtain the classification result corresponding to each test sample image. Based on the classification results and the quality category labels of each test sample image, the classification loss corresponding to each candidate evaluation threshold is determined, and the candidate evaluation threshold with the smallest classification loss is selected as the final evaluation threshold (grid search optimal solution as shown in the figure). A parameter self-learning script is used to automatically adjust the parameters to obtain the evaluation thresholds for each quality assessment parameter, including: face confidence threshold (as shown in the figure), face tilt threshold (as shown in the figure), clarity threshold (as shown in the figure), brightness threshold (as shown in the figure), face area ratio threshold (as shown in the figure), and face resolution threshold (as shown in the figure). Each evaluation threshold is stored in a JSON file (as shown in the figure).

[0198] The specific implementation process of image quality assessment in this embodiment is as follows:

[0199] The image to be evaluated is fed into a cascade of filters. Each filter in the cascade evaluates each quality assessment parameter of the image to be evaluated. First, the first filter is entered (as shown in the figure), and the parameter values for each quality assessment parameter are calculated (as shown in the calculation dimension features). The corresponding assessment threshold for each quality assessment parameter is retrieved from the josn file storing the assessment thresholds. A determination is made as to whether the parameter value exceeds the assessment threshold (as shown in the determination box "Feature exceeds threshold"). If the parameter value exceeds the assessment threshold, the quality assessment result of the image to be evaluated is determined to be unqualified. If the parameter value does not exceed the assessment threshold, a determination is made as to whether this is the last filter. If so, the quality assessment result of the image to be evaluated is determined to be acceptable. If this is not the last filter, the next filter is entered, and the parameter value of the next quality assessment parameter is determined to be acceptable. If the parameter values of all quality assessment parameters do not exceed the corresponding assessment thresholds, the quality assessment result of the image to be evaluated is determined to be acceptable.

[0200] The image quality assessment method provided by the present application determines the evaluation threshold of the image quality assessment parameter by measuring the parameter value of each test sample image in the test data set corresponding to the image quality assessment parameter and each image quality category label. Based on the parameter value of each image quality assessment parameter and each evaluation threshold, the quality of the image to be evaluated is evaluated. This avoids the problem of wasting a lot of testers' time and high testing costs caused by releasing test products multiple times to finally determine the evaluation threshold through manual testing, thereby improving the efficiency of image quality assessment.

[0201] and Figure 1 Based on the same principle as the method shown in , an embodiment of the present disclosure further provides an image quality assessment device 20, such as Figure 9 As shown, the image quality assessment device 20 includes:

[0202] An image acquisition module 21 is used to acquire an image to be evaluated;

[0203] A parameter determination module 22 is used to determine parameter values corresponding to various image quality assessment parameters of the image to be assessed;

[0204] A threshold acquisition module 23 is used to obtain an evaluation threshold of each image quality evaluation parameter;

[0205] A result determination module 24 is used to determine the quality assessment result of the image to be assessed based on the parameter values and the assessment thresholds;

[0206] For any image quality assessment parameter, the assessment threshold corresponding to the image quality assessment parameter is determined by:

[0207] Obtain a test data set corresponding to the image quality assessment parameter, the test data set including each test sample image and an image quality category label of each test sample image;

[0208] Obtaining parameter values corresponding to image quality assessment parameters for each test sample image;

[0209] An evaluation threshold of the image quality evaluation parameter is determined based on the parameter value of each test sample image corresponding to the image quality evaluation parameter and each image quality category label.

[0210] In one possible implementation, each test sample image includes a positive sample image and a negative sample image. For any image quality assessment parameter, the threshold acquisition module 23, when determining the assessment threshold of the image quality assessment parameter based on the parameter value corresponding to the image quality assessment parameter of each test sample image and each image quality category label, is configured to:

[0211] Determine the quality classification result corresponding to each test sample image based on the parameter value and image quality category label of each test sample image;

[0212] If the quality classification results corresponding to each test sample image are linearly separable, determining an evaluation threshold based on the parameter value of each target test sample image, wherein each target test sample image includes a set of sample images with adjacent parameter values in the test sample image, and the set of sample images with adjacent parameter values includes at least one positive sample image and at least one negative sample image;

[0213] If the quality classification results corresponding to each test sample image are linearly inseparable, then determining the classification intersection area of the positive sample image and the negative sample image based on the parameter value of the image quality assessment parameter corresponding to each test sample image;

[0214] An evaluation threshold is determined based on the parameter value of the image quality evaluation parameter corresponding to the classification intersection area and each image quality category label.

[0215] In a possible implementation, when determining the evaluation threshold based on the parameter value of each target sample image, the threshold acquisition module 23 is configured to:

[0216] The evaluation threshold of the image quality evaluation parameter is obtained by fusing the parameter values of each target test sample image corresponding to the image quality evaluation parameter.

[0217] In one possible implementation, when determining the evaluation threshold of the image quality evaluation parameter based on the parameter value of the image quality evaluation parameter corresponding to the classification intersection region and each image quality category label, the threshold acquisition module 23 is configured to:

[0218] The parameter values corresponding to the test sample images whose quality classification results are located in the classification intersection area are respectively used as the first candidate evaluation thresholds;

[0219] For each first candidate evaluation threshold, determining a first classification result for each test sample image based on the parameter value of each test sample image in the test data set and the first candidate evaluation threshold;

[0220] For each first candidate evaluation threshold, determining a classification loss value corresponding to the first candidate evaluation threshold based on the first classification result and the image quality category label corresponding to each test sample image;

[0221] The first candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the first candidate evaluation thresholds is used as the evaluation threshold.

[0222] In one possible implementation, when determining the evaluation threshold of the image quality evaluation parameter based on the parameter value of the image quality evaluation parameter corresponding to the classification intersection region and each image quality category label, the threshold acquisition module 23 is configured to:

[0223] determining a parameter value range of an image quality assessment parameter corresponding to a classification intersection region;

[0224] Select at least one value within the parameter value range as a second candidate evaluation threshold;

[0225] For each second candidate evaluation threshold, determining a second classification result for each test sample image based on the parameter value of each test sample image in the test data set and the second candidate evaluation threshold;

[0226] For each second candidate evaluation threshold, determining a classification loss value corresponding to the second candidate evaluation threshold based on the second classification result and the image quality category label corresponding to each test sample image;

[0227] The second candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the second candidate evaluation thresholds is used as the evaluation threshold.

[0228] In one possible implementation, the image to be evaluated includes a face image, and the image quality evaluation parameter includes at least one of the following:

[0229] Face confidence, face tilt, face offset, proportion of face area in face image, resolution of face image, clarity of face image, brightness of face image.

[0230] In one possible implementation, the clarity of the facial image is determined by:

[0231] Obtaining the global clarity of the face image and the local clarity of the face area in the face image;

[0232] The clarity of the face image is obtained by fusing the global clarity and local clarity;

[0233] The brightness of the face image is determined in the following way:

[0234] Obtaining the global brightness of the face image and the local brightness of the face area in the face image;

[0235] The brightness of the face image is obtained by fusing the global brightness and local brightness.

[0236] The image quality evaluation device of the embodiment of the present disclosure can perform the image quality evaluation of the embodiment of the present disclosure. Figure 1 The corresponding image quality assessment method has a similar implementation principle. The actions performed by each module in the image quality assessment device in the embodiment of the present disclosure correspond to the steps in the image quality assessment method in the embodiment of the present disclosure. For the detailed functional description of each module of the image quality assessment device, please refer to the description in the corresponding image quality assessment method shown in the previous text, and will not be repeated here.

[0237] The image quality assessment device provided in the embodiment of the present application determines the evaluation threshold of the image quality assessment parameter by using the parameter value of each image quality assessment parameter and each image quality category label corresponding to each test sample image in the test data set, and performs quality assessment on the image to be assessed based on the parameter value of each image quality assessment parameter and each evaluation threshold. This avoids the problem of wasting a lot of testers' time and high testing costs caused by releasing test products multiple times to finally determine the evaluation threshold through manual testing, thereby improving the efficiency of image quality assessment.

[0238] Among them, the image quality assessment device can be a computer program (including program code) running in a computer device, for example, the image quality assessment device is an application software; the device can be used to execute the corresponding steps in the method provided in the embodiment of the present application.

[0239] In some embodiments, the image quality assessment device provided by the embodiments of the present invention can be implemented by a combination of software and hardware. As an example, the image quality assessment device provided by the embodiments of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the image quality assessment method provided by the embodiments of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0240] In other embodiments, the image quality assessment device provided by the embodiment of the present invention can be implemented in software. Figure 9 An image quality assessment device stored in a memory is shown, which can be software in the form of a program or plug-in, and includes a series of modules, including an image acquisition module 21, an image restoration module 22, a threshold acquisition module 23, and a result determination module 24, for implementing the image quality assessment method provided in an embodiment of the present invention.

[0241] The above embodiment introduces an image quality assessment device from the perspective of a virtual module. The following describes an electronic device from the perspective of a physical module, as shown below:

[0242] The present application embodiment provides an electronic device, such as Figure 10 As shown, Figure 10The electronic device 8000 shown includes a processor 8001 and a memory 8003. The processor 8001 and the memory 8003 are connected, for example, via a bus 8002. Optionally, the electronic device 8000 may further include a transceiver 8004. It should be noted that in actual applications, the number of transceivers 8004 is not limited to one, and the structure of the electronic device 8000 does not constitute a limitation on the embodiments of the present application.

[0243] The processor 8001 may be a CPU, a general-purpose processor, a GPU, a DSP, an ASIC, an FPGA, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 8001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0244] The bus 8002 may include a path for transmitting information between the above components. The bus 8002 may be a PCI bus or an EISA bus, etc. The bus 8002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0245] The memory 8003 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0246] The memory 8003 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 8001. The processor 8001 is used to execute the application code stored in the memory 8003 to implement the content shown in any of the above method embodiments.

[0247] An embodiment of the present application provides an electronic device, and the electronic device in the embodiment of the present application includes: one or more processors; a memory; one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more programs are executed by the processor, an image to be evaluated is obtained; a parameter value corresponding to each image quality evaluation parameter of the image to be evaluated is determined; an evaluation threshold of each image quality evaluation parameter is obtained; and a quality evaluation result of the image to be evaluated is determined based on each parameter value and each evaluation threshold; wherein, for any image quality evaluation parameter, the evaluation threshold corresponding to the image quality evaluation parameter is determined in the following manner: a test data set corresponding to the image quality evaluation parameter is obtained, the test data set including each test sample image and an image quality category label of each test sample image; a parameter value corresponding to the image quality evaluation parameter of each test sample image is obtained; and the evaluation threshold of the image quality evaluation parameter is determined based on the parameter value corresponding to the image quality evaluation parameter of each test sample image and each image quality category label.

[0248] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program runs on a processor, the processor can execute the corresponding contents of the aforementioned method embodiment.

[0249] According to one aspect of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the aforementioned image quality assessment method.

[0250] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0251] The above descriptions are only partial embodiments of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for image quality assessment, characterized in that: The method comprises: Obtain the image to be evaluated; Determining parameter values of the image to be evaluated corresponding to each image quality evaluation parameter; Obtaining an evaluation threshold value of each image quality evaluation parameter; Determining a quality assessment result of the image to be assessed based on the parameter values and the assessment thresholds; Wherein, for any of the image quality assessment parameters, the assessment threshold corresponding to the image quality assessment parameter is determined by: Acquire a test data set corresponding to the image quality assessment parameter, the test data set including each test sample image and an image quality category label of each test sample image, the image quality category label representing a true quality category of the test sample image corresponding to the quality assessment parameter; Obtaining a parameter value corresponding to the image quality assessment parameter for each of the test sample images; If it is determined based on the parameter values and image quality category labels of each of the test sample images that the image quality of each positive sample image in the test sample images is higher than the image quality of each negative sample image, then the quality classification results corresponding to each of the test sample images are determined to be linearly separable; otherwise, they are determined to be linearly inseparable; If the quality classification result is linearly inseparable, determining the classification intersection area of the positive sample image and the negative sample image based on the parameter value of each of the test sample images; An evaluation threshold of the image quality evaluation parameter is determined based on the parameter value corresponding to the image quality evaluation parameter and the image quality category label of each test sample image corresponding to the classification intersection area.

2. The method according to claim 1, characterized in that The method further comprises: If the quality classification results corresponding to each of the test sample images are linearly separable, the evaluation threshold is determined based on the parameter value of each target test sample image, wherein each target test sample image includes a set of sample images with adjacent parameter values in the test sample image, and the set of sample images with adjacent parameter values includes at least one positive sample image and at least one negative sample image.

3. The method according to claim 1 or 2, characterized in that For any image quality assessment parameter, if the image quality scores of the positive sample image and the negative sample image meet any of the following conditions, the quality classification result corresponding to the test sample image is linearly separable; otherwise, the quality classification result corresponding to the test sample image is linearly inseparable: In a case where a larger parameter value of the image quality assessment parameter indicates better image quality, a minimum parameter value of the image quality assessment parameter of the positive sample image is greater than a maximum parameter value of the image quality assessment parameter of the negative sample image; In the case where the smaller the parameter value of the image quality evaluation parameter, the better the image quality, the maximum parameter value of the image quality evaluation parameter of the positive sample image is smaller than the minimum parameter value of the image quality evaluation parameter of the negative sample image.

4. The method according to claim 2, characterized in that The determining the evaluation threshold based on the parameter value of each target sample image includes: An evaluation threshold of the image quality evaluation parameter is obtained by fusing the parameter values of the target test sample images corresponding to the image quality evaluation parameter.

5. The method according to claim 2, characterized in that The determining, based on the parameter value of the image quality assessment parameter corresponding to the classification intersection area and each of the image quality category labels, an assessment threshold of the image quality assessment parameter includes: taking the parameter values corresponding to the test sample images whose quality classification results are located in the classification intersection area as first candidate evaluation thresholds; For each of the first candidate evaluation thresholds, determining a first classification result for each of the test sample images based on the parameter value of each test sample image in the test data set and the first candidate evaluation threshold; For each of the first candidate evaluation thresholds, determining a classification loss value corresponding to the first candidate evaluation threshold based on the first classification result and the image quality category label corresponding to each of the test sample images; The first candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the first candidate evaluation thresholds is used as the evaluation threshold.

6. The method according to claim 2, characterized in that The determining, based on the parameter value of the image quality assessment parameter corresponding to the classification intersection area and each of the image quality category labels, an assessment threshold of the image quality assessment parameter includes: Determining a parameter value range of the classification intersection area corresponding to the image quality assessment parameter; Selecting at least one value within the parameter value range as a second candidate evaluation threshold; For each of the second candidate evaluation thresholds, determining a second classification result for each of the test sample images based on the parameter value of each test sample image in the test data set and the second candidate evaluation threshold; For each of the second candidate evaluation thresholds, determining a classification loss value corresponding to the second candidate evaluation threshold based on the second classification result and the image quality category label corresponding to each of the test sample images; The second candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the second candidate evaluation thresholds is used as the evaluation threshold.

7. The method according to claim 1, characterized in that The image to be evaluated includes a face image, and the image quality evaluation parameter includes at least one of the following: Face confidence, face tilt, face offset, proportion of face area in the face image, resolution of the face image, clarity of the face image, and brightness of the face image.

8. The method according to claim 7, characterized in that The clarity of the facial image is determined by: Acquiring the global clarity of the facial image and the local clarity of the facial region in the facial image; Obtaining the clarity of the face image by fusing the global clarity and the local clarity; The brightness of the face image is determined by: Acquire the global brightness of the facial image and the local brightness of the facial area in the facial image; The brightness of the face image is obtained by fusing the global brightness and the local brightness.

9. An image quality assessment device, characterized in that: The device comprises: An image acquisition module, used to acquire an image to be evaluated; A parameter determination module, configured to determine parameter values corresponding to various image quality assessment parameters of the image to be assessed; A threshold acquisition module, used to obtain the evaluation threshold of each image quality evaluation parameter; A result determination module, configured to determine a quality assessment result of the image to be assessed based on each of the parameter values and each of the assessment thresholds; Wherein, for any of the image quality assessment parameters, the assessment threshold corresponding to the image quality assessment parameter is determined by the threshold determination module in the following manner: Acquire a test data set corresponding to the image quality assessment parameter, the test data set including each test sample image and an image quality category label of each test sample image; the image quality category label represents a true quality category of the test sample image corresponding to the quality assessment parameter; Obtaining a parameter value corresponding to the image quality assessment parameter for each of the test sample images; If it is determined based on the parameter values and image quality category labels of each of the test sample images that the image quality of each positive sample image in the test sample images is higher than the image quality of each negative sample image, then the quality classification results corresponding to each of the test sample images are determined to be linearly separable; otherwise, they are determined to be linearly inseparable; If the quality classification result is linearly inseparable, determining the classification intersection area of the positive sample image and the negative sample image based on the parameter value of each of the test sample images; An evaluation threshold of the image quality evaluation parameter is determined based on the parameter value of each test sample image corresponding to the classification intersection area and each image quality category label corresponding to the image quality evaluation parameter.

10. The device according to claim 9, characterized in that The threshold determination module is further configured to: If the quality classification results corresponding to each of the test sample images are linearly separable, the evaluation threshold is determined based on the parameter value of each target test sample image, wherein each target test sample image includes a set of sample images with adjacent parameter values in the test sample image, and the set of sample images with adjacent parameter values includes at least one positive sample image and at least one negative sample image.

11. The device according to claim 9 or 10, characterized in that For any image quality assessment parameter, if the image quality scores of the positive sample image and the negative sample image meet any of the following conditions, the quality classification result corresponding to the test sample image is linearly separable; otherwise, the quality classification result corresponding to the test sample image is linearly inseparable: In a case where a larger parameter value of the image quality assessment parameter indicates better image quality, a minimum parameter value of the image quality assessment parameter of the positive sample image is greater than a maximum parameter value of the image quality assessment parameter of the negative sample image; In the case where the smaller the parameter value of the image quality evaluation parameter, the better the image quality, the maximum parameter value of the image quality evaluation parameter of the positive sample image is smaller than the minimum parameter value of the image quality evaluation parameter of the negative sample image.

12. The device according to claim 10, characterized in that The threshold acquisition module is specifically used to determine the evaluation threshold based on the parameter value of each target test sample image: An evaluation threshold of the image quality evaluation parameter is obtained by fusing the parameter values of the target test sample images corresponding to the image quality evaluation parameter.

13. The device according to claim 9, characterized in that The threshold acquisition module determines an evaluation threshold of the image quality evaluation parameter based on the parameter value corresponding to the image quality evaluation parameter and the image quality category label of each test sample image corresponding to the classification intersection area, and is specifically used to: taking the parameter values corresponding to the test sample images whose quality classification results are located in the classification intersection area as first candidate evaluation thresholds; For each of the first candidate evaluation thresholds, determining a first classification result for each of the test sample images based on the parameter value of each test sample image in the test data set and the first candidate evaluation threshold; For each of the first candidate evaluation thresholds, determining a classification loss value corresponding to the first candidate evaluation threshold based on the first classification result and the image quality category label corresponding to each of the test sample images; The first candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the first candidate evaluation thresholds is used as the evaluation threshold.

14. The device according to claim 9, characterized in that The threshold acquisition module determines an evaluation threshold of the image quality evaluation parameter based on the parameter value corresponding to the image quality evaluation parameter and the image quality category label of each test sample image corresponding to the classification intersection area, and is specifically used to: Determining a parameter value range of the classification intersection area corresponding to the image quality assessment parameter; Selecting at least one value within the parameter value range as a second candidate evaluation threshold; For each of the second candidate evaluation thresholds, determining a second classification result for each of the test sample images based on the parameter value of each test sample image in the test data set and the second candidate evaluation threshold; For each of the second candidate evaluation thresholds, determining a classification loss value corresponding to the second candidate evaluation threshold based on the second classification result and the image quality category label corresponding to each of the test sample images; The second candidate evaluation threshold with the smallest classification loss value among the classification loss values corresponding to the second candidate evaluation thresholds is used as the evaluation threshold.

15. The device according to claim 9, characterized in that The image to be evaluated includes a face image, and the image quality evaluation parameter includes at least one of the following: Face confidence, face tilt, face offset, proportion of face area in the face image, resolution of the face image, clarity of the face image, and brightness of the face image.

16. The device according to claim 15, characterized in that The clarity of the facial image is determined by: Acquiring the global clarity of the facial image and the local clarity of the facial region in the facial image; Obtaining the clarity of the face image by fusing the global clarity and the local clarity; The brightness of the face image is determined by: Acquire the global brightness of the facial image and the local brightness of the facial area in the facial image; The brightness of the face image is obtained by fusing the global brightness and the local brightness.

17. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and the one or more computer programs are configured to perform the method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and when the computer program runs on a processor, the processor can execute the method according to any one of claims 1 to 8.

Images