An image quality evaluation method and apparatus

Abstract

This specification discloses an image quality evaluation method and apparatus. Obtain an image to be evaluated and a number of semantic labels corresponding to the image to be evaluated, and then determine the similarity between the image to be evaluated and each of the semantic labels according to the image features of the image to be evaluated and the label features of each semantic label. Furthermore, determine the fusion features of the image to be evaluated according to each similarity, image features, and label features, identify the fusion features, determine the recognition result representing the probability that the image to be evaluated belongs to each preset low-quality image type, and then determine the quality score of the image to be evaluated according to the recognition result. While determining the image quality score, this method can simultaneously obtain the probability that the image to be evaluated corresponds to each low-quality image type, enabling improvement of the image based on the low-quality reasons and enhancing the accuracy of image quality evaluation.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image quality evaluation method and device. Background Art

[0002] Image quality assessment is one of the basic techniques in image processing. It mainly analyzes the characteristics of the image and then evaluates the quality of the image.

[0003] A commonly used image quality assessment method is based on a pre-trained image quality assessment model. Specifically, the image to be assessed is first identified as the image to be assessed. Then, the image to be assessed is fed into the pre-trained image quality assessment model to obtain a quality score for each pixel in the image. Finally, based on the quality scores of each pixel, an overall quality score for the image is obtained.

[0004] However, in the existing technology, when training image quality evaluation models, the quality scores determined based on factors such as image clarity and completeness are usually used as annotations for training. Therefore, when determining the image quality based on the trained image quality evaluation model, if the quality score of the image to be evaluated is low, the reason for the low quality of the image cannot be known, resulting in low evaluation efficiency. Summary of the Invention

[0005] The embodiments of this specification provide an image quality evaluation method and apparatus to partially solve the problems in the prior art.

[0006] The embodiments of this specification adopt the following technical solutions:

[0007] This specification provides an image quality evaluation method, including:

[0008] Obtaining an image to be evaluated and a plurality of semantic labels corresponding to the image to be evaluated;

[0009] Determining, based on the image features of the image to be evaluated and the label features of each semantic label, the similarity between the image to be evaluated and each semantic label;

[0010] Determining, based on the similarities, the image features, and the label features, a fusion feature of the image to be evaluated, identifying the fusion feature, and determining a recognition result, wherein the recognition result is a probability that the image to be evaluated belongs to each preset low-quality image type;

[0011] A quality score of the image to be evaluated is determined according to the recognition result.

[0012] Optionally, determining a number of semantic labels corresponding to the image to be evaluated specifically includes:

[0013] Input the image to be evaluated as input into a pre-trained object classification model to obtain the classification results output by the label determination model;

[0014] According to the classification results and a preset label dictionary, the semantic labels corresponding to the image to be evaluated are determined.

[0015] Optionally, the target object classification model is trained in the following manner:

[0016] Acquire a plurality of images as first training samples, and determine, for each training sample, labels of the first training sample according to the semantic label dictionary;

[0017] The first training sample is input into the target object classification model to be trained to obtain classification results of the first training sample;

[0018] A first loss is determined according to the classification results and labels of the first training samples, and a model parameter of the target object classification model is adjusted according to the first loss.

[0019] Optionally, determining a fusion feature of the image to be evaluated based on the similarities, the image features, and the label features, and identifying the fusion feature to determine an identification result, specifically includes:

[0020] The image features and each label feature are input into the fusion layer of a pre-trained image quality assessment model, and the fusion features of the image to be evaluated are determined according to each similarity;

[0021] The fused features are input into the recognition layer of the image quality assessment model, the fused features are identified, and a recognition result of the image to be evaluated outputted by the recognition layer is determined.

[0022] Optionally, the image quality assessment model is trained in the following manner:

[0023] Determining, based on the acquired images, each second training sample, each semantic label corresponding to each second training sample, and a label of each second training sample corresponding to each semantic label;

[0024] For each second training sample, determining the similarity between the second training sample and each semantic label according to the image feature of the second training sample and the label feature of each semantic label;

[0025] The image features and label features of the second training sample are input into a fusion layer of the image quality assessment model to be trained, and the fusion features of the second training sample are determined according to the similarities;

[0026] Inputting the fused features into the recognition layer of the image quality assessment model, recognizing the fused features, and determining a recognition result of the second training sample output by the recognition layer;

[0027] The image quality assessment model is trained according to the labeling and recognition results of the second training samples.

[0028] Optionally, determining each second training sample according to the acquired images specifically includes:

[0029] Acquire several images;

[0030] For each image, preprocessing is performed on the image, and the preprocessing result is used as a second training sample, wherein the preprocessing at least includes affine transformation.

[0031] Optionally, determining the image features of the image to be evaluated and the label features of each semantic label specifically includes:

[0032] Segmenting the image to be evaluated to determine a number of unit images, and determining, for each unit image, an image feature of the unit image based on a similarity between the unit image and other unit images;

[0033] Determining the image features of the image to be evaluated according to the image features of each unit image;

[0034] For each semantic tag corresponding to the image to be evaluated, a tag feature of the semantic tag is determined according to the correlation between the semantic tag and other semantic tags.

[0035] This specification provides an image quality evaluation device, comprising:

[0036] An acquisition module, configured to acquire an image to be evaluated and a plurality of semantic labels corresponding to the image to be evaluated;

[0037] A similarity determination module is used to determine the similarity between the image to be evaluated and each semantic tag according to the image features of the image to be evaluated and the tag features of each semantic tag;

[0038] an identification module, configured to determine, based on the similarities, the image features, and the label features, a fusion feature of the image to be evaluated, identify the fusion feature, and determine an identification result, wherein the identification result is a probability that the image to be evaluated belongs to each preset low-quality image type;

[0039] The scoring module is used to determine the quality score of the image to be evaluated based on the recognition result.

[0040] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image quality evaluation method is implemented.

[0041] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned image quality evaluation method is implemented.

[0042] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:

[0043] In this specification, an image to be evaluated and several semantic labels corresponding to the image to be evaluated are obtained, and then based on the image features of the image to be evaluated and the label features of each semantic label, the similarity of the image to be evaluated corresponding to each of the semantic labels is determined, and then based on each similarity, image features and each label feature, the fusion features of the image to be evaluated are determined, the fusion features are identified, and an identification result representing the probability that the image to be evaluated belongs to each preset low-quality image type is determined, and then based on the identification result, the quality score of the image to be evaluated is determined.

[0044] According to the above content, it can be seen that this method can simultaneously obtain the probability of the image to be evaluated corresponding to each low-quality type of image while determining the image quality score, so that the image can be improved based on the cause of low quality, thereby improving the accuracy of image quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 A flowchart of the image quality evaluation method provided in this manual;

[0047] Figure 2 A flowchart of the image quality evaluation method provided in this manual;

[0048] Figure 3 This is a schematic diagram of the structure of the image quality evaluation device provided in this manual;

[0049] Figure 4 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0050] To make the purpose, technical solutions, and advantages of this specification more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0052] Figure 1 A flowchart of an image quality evaluation method provided in an embodiment of this specification may include the following steps:

[0053] S100: Obtain an image to be evaluated and several semantic labels corresponding to the image to be evaluated.

[0054] Unlike existing techniques, which determine a quality score for each pixel of an image to be evaluated and then determine the quality score of the image to be evaluated based on the quality score of each pixel, and in the case of a low score, only the low quality of the image to be evaluated is known, but the reason for the low quality is unknown, this specification provides a new image quality evaluation method that can determine multiple semantic labels for the image to be evaluated, and enhance the semantics of the image to be evaluated based on each semantic label. Based on the enhanced results, the score of the image to be evaluated and the reason for the low quality are determined. This improves evaluation efficiency.

[0055] Based on this, we can first determine the image to be evaluated and its corresponding semantic labels.

[0056] In one or more embodiments provided herein, the image quality assessment method is typically applied to scenarios involving the quality assessment of key frames of an image or video, and is executed by a service provider's server. Similarly, the image quality assessment method provided herein may also be executed by a server and applied to scenarios involving the quality assessment of image or video frames. The server may be a single server or a system consisting of multiple servers, such as a distributed server. This specification does not impose any restrictions on this, and configuration may be performed as needed.

[0057] Specifically, when an image needs to be evaluated, the server may receive an image to be evaluated. The image to be evaluated may be sent by a user or by another server.

[0058] Then, the server may determine a semantic tag of the image to be evaluated. The tag may be predetermined, or may be determined by the server inputting the image to be evaluated into a predetermined object classification model.

[0059] Specifically, the server may first take the image to be evaluated as input into a pre-trained object classification model to obtain classification results output by the object classification model.

[0060] Then, the semantic labels corresponding to the image to be evaluated are determined according to the classification results and a preset label dictionary.

[0061] Of course, if a preset label dictionary is used to determine the annotations when training the target object classification model, then when determining the semantic label corresponding to the image to be evaluated, there is no need to determine it based on the label dictionary.

[0062] In addition, the target object classification model can be trained in the following ways:

[0063] The server may first obtain a number of images as first training samples, and determine, for each training sample, labels of the first training sample according to a preset label dictionary.

[0064] Then, the server may use the first training sample as input into the target object classification model to be trained to obtain various classification results of the first training sample.

[0065] Finally, the server may determine a first loss based on the classification results and labels of the first training samples, and adjust the model parameters of the target object classification model based on the first loss.

[0066] The server for training the target object classification model and the server for executing the image quality assessment method may be the same server or different servers, and may be specifically configured as needed, which is not limited in this specification.

[0067] S102: Determine, based on the image features of the image to be evaluated and the label features of the semantic labels, the similarity between the image to be evaluated and the semantic labels.

[0068] In one or more embodiments provided herein, as previously described, semantic enhancement can be performed on an image to be evaluated based on its semantic labels. Different semantic labels have different weights within the image. Therefore, the similarity between the image to be evaluated and the semantic labels can be determined, and then semantic enhancement can be performed on the image to be evaluated based on the similarity between the semantic labels.

[0069] Based on this, the server can determine the similarity between the images to be evaluated and corresponding to each semantic tag.

[0070] Specifically, the server may first perform feature extraction on the image to be evaluated to determine the image features of the image to be evaluated. The image features may be determined by inputting the image to be evaluated into a pre-trained image feature extraction module, or by multiplying the pixel values of the image to be evaluated by a pre-determined feature matrix to obtain the image features of the image to be evaluated. The specific method for determining the image features can be configured as needed and is not limited in this specification.

[0071] The server can then determine the tag features corresponding to each tag based on the preset correspondence between each tag and each tag feature. The method for determining the tag features can be the same as the method for determining the image features described above. The specific method for determining the tag features can be set as needed and is not limited in this specification.

[0072] Finally, the server can determine, for each tag, the similarity between the tag features of the tag and the image features of the image to be evaluated. The similarity can be a vector product, Euclidean distance, etc. The specific method of determining similarity can be set as needed and is not limited in this specification.

[0073] S104: Determine the fusion features of the image to be evaluated based on the similarities, the image features, and the label features, identify the fusion features, and determine a recognition result, where the recognition result is the probability that the image to be evaluated belongs to each preset low-quality image type.

[0074] In one or more embodiments provided herein, as previously described, semantic enhancement may be performed on the image to be evaluated based on its semantic tags. Therefore, the server may determine the fusion features of the image to be evaluated based on the similarities, the image features, and the tag features.

[0075] Specifically, the server may first determine the weight corresponding to each tag feature according to the similarity between each tag feature and the image feature, then perform weighted summation on each tag feature, and fuse the weighted result with the image feature to obtain the image enhancement feature.

[0076] Then, the server may determine, for each tag feature, a weight of the image to be evaluated corresponding to the tag feature according to the similarity between the tag feature and the image feature, and determine a tag enhancement feature of the tag according to the weight.

[0077] Finally, the server may determine the fusion features of the image to be evaluated based on the image enhancement features and the label enhancement features corresponding to each label.

[0078] Furthermore, the image quality assessment method provided herein aims to determine the reasons for the image's low quality while also determining the image's corresponding quality. Based on this, the server can determine the probability of the image belonging to each of the preset low-quality image categories and determine a score for the image to be assessed.

[0079] Specifically, the server can use the determined fusion features as input to a pre-trained recognition model, and obtain a recognition result for the image to be evaluated, output by the recognition model. The recognition result is the probability of the image being classified as low-quality for each preset image quality category. The model result of the recognition model can be a fully connected layer.

[0080] Furthermore, the above steps of determining the fusion features and the recognition results of the fusion features can also be performed by inputting the image features and label features into a pre-trained image quality model to determine,

[0081] Specifically, the server may first take the image features and label features as input, input them into a preferably trained fusion layer for image quality evaluation, and then determine the fusion features of the image to be evaluated based on the similarities.

[0082] Then, the server may input the fused features into the recognition layer of the image quality evaluation model, identify the fused features, and obtain a recognition result of the image to be evaluated output by the recognition layer.

[0083] The above image quality assessment model can be trained using the following method:

[0084] Specifically, the training model server may first obtain a number of images, and determine each training sample based on each image.

[0085] Secondly, for each training sample, the server may determine the semantic labels of the second training sample and the annotation of the second training sample, wherein the annotation is the probability that the second training sample belongs to each preset low-quality image type.

[0086] Then, the server can input the image features of the second training sample and the label features of each semantic label into the fusion layer of the image quality evaluation model to be trained, determine the fusion features of the second training sample, and then input the fusion features into the recognition layer of the image quality evaluation model to identify the fusion features and determine the recognition result of the second training sample output by the recognition layer.

[0087] Finally, the server may determine the loss according to the labeling and recognition results of the second training samples, and adjust the model parameters of the image quality assessment model according to the loss to train the image quality assessment model.

[0088] Furthermore, to ensure the robustness of the image quality assessment, when determining training samples, the server may also preprocess each acquired image and use the preprocessed result as the second training sample. The preprocessing method may include, for example, an affine transformation. The specific type of preprocessing method can be set as needed and is not limited in this specification.

[0089] Furthermore, before determining the fusion feature, the server may first perform semantic enhancement on the image feature and the label feature respectively, and then perform semantic enhancement when determining the fusion feature to obtain a fusion feature with richer semantics.

[0090] For image features:

[0091] Specifically, the server may first segment the image to be evaluated to determine a number of unit images.

[0092] Secondly, the server may determine, for each unit image, the weights of other unit images relative to the unit image according to the similarities between the unit image and other unit images.

[0093] Then, the image features of the unit image are determined through weighted summation.

[0094] Finally, the server determines the image features of the image to be evaluated based on the image features of each unit image.

[0095] For the label feature:

[0096] Specifically, the server may determine, for each semantic tag, the weight of other semantic tags relative to the semantic tag according to the similarity between the semantic tag and other semantic tags.

[0097] Then, through weighted summation, the server can determine the tag feature of the semantic tag.

[0098] Of course, in order to ensure the input of the above-mentioned image quality evaluation model, the server can also fuse the label features of the above-mentioned labels to obtain the label features of the image to be evaluated with the same dimension as the image features.

[0099] S106: Determine a quality score of the image to be evaluated based on the recognition result.

[0100] In one or more embodiments of the present specification, after determining the recognition result, the server may determine a quality score of the image to be evaluated based on the recognition result.

[0101] Specifically, the higher the probability of the image being of low quality, the lower the image quality score. Since the above recognition result is the probability that the image to be evaluated belongs to each preset low-quality image type, the server can determine the quality score of the image to be evaluated based on the value of the above recognition result.

[0102] The quality score is negatively correlated with the value in the recognition result.

[0103] The quality score can be determined based on the maximum value corresponding to each probability in the above recognition results, or it can be determined based on the mean value. The specific method of determining the quality score based on the recognition results can be set as needed, and this manual does not impose any restrictions on this.

[0104] Finally, after determining the quality scores of the images, images with higher quality scores may be displayed based on the quality scores.

[0105] based on Figure 1 The image quality assessment method shown in the figure obtains an image to be evaluated and several semantic labels corresponding to the image to be evaluated. Based on the image features of the image to be evaluated and the label features of each semantic label, the method determines the similarity of the image to be evaluated with respect to each of the semantic labels. Furthermore, based on the similarities, image features, and label features, a fusion feature of the image to be evaluated is determined. The fusion feature is then identified to determine a recognition result representing the probability that the image to be evaluated belongs to each preset low-quality image category. Based on the recognition result, a quality score for the image to be evaluated is determined. While determining the image quality score, this method also obtains the probability that the image to be evaluated belongs to each low-quality image category, allowing improvements to be made to the image based on the cause of the low quality, thereby improving the accuracy of image quality assessment.

[0106] Furthermore, the image quality evaluation model can be applied to the scenario of displaying search results to users, that is, receiving each image to be evaluated determined according to the user's recommendation request, and determining the score of each image to be evaluated, and then determining the image to be displayed to the user based on each quality score. However, there may be a situation where the user particularly hates a certain target object. The server can also train the image quality evaluation model based on user data. Specifically, the server can determine the user data based on the user's user information, and determine the target object that the user hates based on the user data. Then, based on the determined target object, the type of low-quality image type in the recognition result is increased, and the annotation of the second training sample containing the target object for the newly added low-quality image type is set to 1. The image quality evaluation model corresponding to the user can be trained.

[0107] Based on the above image quality evaluation method, this specification also provides a flow chart of an image quality evaluation method, such as Figure 2 shown.

[0108] Figure 2 A flow chart of the image quality evaluation method provided in this specification. In the figure, the server can first obtain the image to be evaluated, then determine the image features of the image to be evaluated, and input the model to be evaluated into a pre-trained target object classification model to obtain the target objects contained in the image to be evaluated as various semantic labels. Then, the image features of the image to be evaluated and the label features of each semantic label are used as input to the fusion layer of the pre-trained image quality evaluation model to determine the fusion features, and the fusion features are used as input to the recognition layer of the image quality evaluation model to obtain the recognition result, and then the quality score of the image to be evaluated is determined based on the recognition result.

[0109] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0110] based on Figure 1 The image quality evaluation method shown in the embodiment of this specification also provides a structural schematic diagram of an image quality evaluation device, as shown in FIG. Figure 3 shown.

[0111] Figure 3 A schematic diagram of the structure of an image quality evaluation device provided in an embodiment of this specification includes:

[0112] The acquisition module 200 is used to acquire an image to be evaluated and a plurality of semantic labels corresponding to the image to be evaluated.

[0113] The similarity determination module 202 is configured to determine the similarity between the image to be evaluated and each semantic tag according to the image features of the image to be evaluated and the tag features of each semantic tag.

[0114] The recognition module 204 is used to determine the fusion features of the image to be evaluated based on the similarities, the image features and the label features, and to identify the fusion features to determine a recognition result, which is the probability that the image to be evaluated belongs to each preset low-quality image type.

[0115] The scoring module 206 is configured to determine a quality score of the image to be evaluated based on the recognition result.

[0116] Optionally, the device further includes:

[0117] The training module 208 is used to train the object classification model in the following manner: obtain a number of images as first training samples, and for each training sample, determine the labels of the first training sample according to the semantic label dictionary, use the first training sample as input, input it into the object classification model to be trained, obtain the classification results of the first training sample, determine the first loss according to the classification results and labels of each first training sample, and adjust the model parameters of the object classification model according to the first loss.

[0118] Optionally, the acquisition module 200 is used to take the image to be evaluated as input into a pre-trained target object classification model, obtain the classification results output by the target object classification model, and determine the semantic labels corresponding to the image to be evaluated based on the classification results and a preset label dictionary.

[0119] Optionally, the acquisition module 200 is used to take the image features and each label feature as input, input them into the fusion layer of a pre-trained image quality assessment model, determine the fusion features of the image to be evaluated based on each similarity, input the fusion features into the recognition layer of the image quality assessment model, identify the fusion features, and determine the recognition result of the image to be evaluated output by the recognition layer.

[0120] Optionally, the training module 208 is used to train the image quality assessment model in the following manner: based on the acquired images, determine each second training sample, each semantic label corresponding to the second training sample, and each label corresponding to the second training sample; for each second training sample, determine the similarity between the second training sample and each semantic label based on the image features of the second training sample and the label features of each semantic label; use the image features and each label feature of the second training sample as input and input them into the fusion layer of the image quality assessment model to be trained; determine the fusion features of the second training sample based on the similarities; input the fusion features into the recognition layer of the image quality assessment model; identify the fusion features; determine the recognition result of the second training sample output by the recognition layer; and train the image quality assessment model based on the labels and recognition results of each second training sample.

[0121] Optionally, the training module 208 is configured to obtain a plurality of images, perform preprocessing on each image, and use the preprocessing result as a second training sample, wherein the preprocessing includes at least affine transformation.

[0122] Optionally, the similarity determination module 202 is used to segment the image to be evaluated, determine a number of unit images, and for each unit image, determine the image features of the unit image based on the similarity between the unit image and other unit images, determine the image features of the image to be evaluated based on the image features of each unit image, and for each semantic label corresponding to the image to be evaluated, determine the label features of the semantic label based on the similarity between the semantic label and other semantic labels.

[0123] The embodiment of this specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1 Provided image quality evaluation method.

[0124] based on Figure 1 The image quality evaluation method shown in the embodiment of this specification also proposes Figure 4 The schematic structure diagram of the electronic device shown in FIG. Figure 4 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The image quality evaluation method shown.

[0125] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0126] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0127] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0128] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0129] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0130] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0135] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0136] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0138] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0140] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0141] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. An image quality evaluation method, characterized in that, it includes: obtaining an image to be evaluated and a plurality of semantic labels corresponding to the image to be evaluated; determining the similarity of the image to be evaluated corresponding to each semantic label according to the image features of the image to be evaluated and the label features of each semantic label; determining the fusion features of the image to be evaluated according to each similarity, the image features and each label feature, and identifying the fusion features to determine an identification result, where the identification result is the probability that the image to be evaluated belongs to each preset low-quality image type; determining the quality score of the image to be evaluated according to the identification result; The determination of the image features of the image to be evaluated and the label features of each semantic label specifically includes: segmenting the image to be evaluated to determine a plurality of unit images, and for each unit image, determining the image features of the unit image according to the similarity between the unit image and other unit images; determining the image features of the image to be evaluated according to the image features of each unit image; for each semantic label corresponding to the image to be evaluated, determining the label features of the semantic label according to the similarity between the semantic label and other semantic labels; determining the fusion features of the image to be evaluated according to each similarity, the image features and each label feature, and identifying the fusion features to determine an identification result, specifically including: taking the image features and each label feature as inputs and inputting them into the fusion layer of a pre-trained image quality evaluation model, and determining the fusion features of the image to be evaluated according to each similarity; inputting the fusion features into the identification layer of the image quality evaluation model, identifying the fusion features, and determining the identification result of the image to be evaluated output by the identification layer.

2. The method according to claim 1, characterized in that, determining a plurality of semantic labels corresponding to the image to be evaluated specifically includes: taking the image to be evaluated as an input and inputting it into a pre-trained target object classification model to obtain each classification result output by the target object classification model; determining each semantic label corresponding to the image to be evaluated according to each classification result and a preset label dictionary.

3. The method according to claim 2, characterized in that, the target object classification model is trained in the following manner: obtaining a plurality of images as each first training sample, and for each training sample, determining each annotation of the first training sample according to the semantic label dictionary; taking the first training sample as an input and inputting it into the target object classification model to be trained to obtain each classification result of the first training sample; determining a first loss according to each classification result and its annotation of each first training sample, and adjusting the model parameters of the target object classification model according to the first loss.

4. The method according to claim 1, characterized in that, the image quality evaluation model is trained in the following manner: determining each second training sample, each semantic label corresponding to each second training sample, and the annotation corresponding to each second training sample according to the obtained plurality of images; For each second training sample, determine the similarity of the second training sample corresponding to each semantic label according to the image features of the second training sample and the label features of each semantic label. Take the image features of the second training sample and each label feature as inputs, input them into the fusion layer of the image quality evaluation model to be trained, and determine the fusion features of the second training sample according to each similarity. Input the fusion features into the recognition layer of the image quality evaluation model, recognize the fusion features, and determine the recognition result of the second training sample output by the recognition layer. Train the image quality evaluation model according to the annotations and recognition results of each second training sample.

5. The method according to claim 4, wherein, determining each second training sample according to the obtained several images specifically includes: obtain several images; For each image, perform preprocessing on the image, and use the preprocessing result as the second training sample, where the preprocessing at least includes affine transformation.

6. An image quality evaluation device, wherein, comprising: an acquisition module, configured to acquire an image to be evaluated and several semantic labels corresponding to the image to be evaluated; a similarity determination module, configured to determine the similarity of the image to be evaluated corresponding to each semantic label according to the image features of the image to be evaluated and the label features of each semantic label. Determining the image features of the image to be evaluated and the label features of each semantic label specifically includes: Segment the image to be evaluated to determine several unit images, and for each unit image, determine the image features of the unit image according to the similarity between the unit image and other unit images; Determine the image features of the image to be evaluated according to the image features of each unit image; For each semantic label corresponding to the image to be evaluated, determine the label features of the semantic label according to the similarity between the semantic label and other semantic labels; a recognition module, configured to determine the fusion features of the image to be evaluated according to each similarity, the image features and each label feature, and recognize the fusion features to determine the recognition result. The recognition result is the probability that the image to be evaluated belongs to each preset low-quality image type. Determining the fusion features of the image to be evaluated according to each similarity, the image features and each label feature, and recognizing the fusion features to determine the recognition result specifically includes: Take the image features and each label feature as inputs, input them into the fusion layer of the pre-trained image quality evaluation model, and determine the fusion features of the image to be evaluated according to each similarity; Input the fusion features into the recognition layer of the image quality evaluation model, recognize the fusion features, and determine the recognition result of the image to be evaluated output by the recognition layer; a scoring module, configured to determine the quality score of the image to be evaluated according to the recognition result.

7. A computer-readable storage medium, wherein, the storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1-5 above is implemented.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, when the processor executes the program, the method described in any one of claims 1-5 above is implemented.

Images