Image quality assessment method and apparatus

Abstract

A method and device for image quality assessment. The method comprises: obtaining a first image; performing quality assessment on the first image to obtain a comprehensive assessment result and a problem type assessment result; wherein the comprehensive assessment result is used to indicate a comprehensive quality level of the first image, and the problem type assessment result is used to indicate a type of problem existing on the first image, the problem type comprising at least one of overexposure, underexposure, blur, color cast, noise, glare, reflection or tree shadow; and determining a first quality level of the first image according to the comprehensive assessment result and the problem type assessment result. In this way, the accuracy of image quality assessment can be improved.

Description

Technical Field

[0001] This application relates to the fields of connected vehicles and mapping, and in particular to an image quality assessment method and apparatus. Background Technology

[0002] In machine vision tasks such as image classification, object detection, image segmentation, and 3D reconstruction, image quality is a key factor affecting task performance. Poor-quality images can severely impact task execution.

[0003] However, it is obviously unrealistic to rely on manual evaluation of image quality. Moreover, image quality is related to many factors such as shooting hardware, shooting environment, and shooting techniques. Therefore, how to automatically and efficiently judge the quality of images is a question that needs to be considered. Summary of the Invention

[0004] The purpose of this application is to provide an image quality assessment method and apparatus to improve the accuracy of image quality assessment.

[0005] Firstly, an image quality assessment method is provided. The method includes: acquiring a first image; performing a quality assessment on the first image to obtain a comprehensive assessment result and a problem type assessment result; wherein the comprehensive assessment result indicates the overall quality level of the first image, and the problem type assessment result indicates the type of problem present in the first image, the problem type including at least one of overexposure, underexposure, blur, color cast, noise, glare, reflection, or tree shadow; and determining a first quality level of the first image based on the comprehensive assessment result and the problem type assessment result.

[0006] In this embodiment, the image quality assessment device can obtain not only the comprehensive assessment result of the first image, but also the problem type assessment result of the first image. The problem type assessment result can indicate the type of problem existing in the first image. The two assessment results (comprehensive assessment result and problem type assessment result) are fused to obtain the final quality level (i.e., the first quality level) of the first image, which helps to improve the accuracy of image quality assessment.

[0007] The following section presents the comprehensive evaluation results and the problem type evaluation results.

[0008] In one possible design, the overall evaluation result is related to the user's liking of the first image. For example, the higher the user's liking of the first image, the higher the overall evaluation result. For instance, an AI algorithm (such as a convolutional neural network) can be used to calculate the overall evaluation result. Generally, the AI ​​algorithm is trained before calculation. The training process includes: Step 1, acquiring training images and manually assigning quality scores to them. Step 2, using the AI ​​algorithm to calculate the training results on the training images, which are the quality scores calculated by the AI ​​algorithm. If the training results differ significantly from the manually labeled quality scores, the calculation parameters of the AI ​​algorithm are adjusted until the difference between the training results obtained by the AI ​​algorithm and the manually labeled quality scores is small, thus ending the training. Therefore, the training process can be understood as adjusting the calculation parameters of the AI ​​algorithm to make the training results obtained by the AI ​​algorithm closer to the manually labeled results. After training is complete, the AI ​​algorithm (the trained AI algorithm) is used to evaluate the image quality to obtain the overall evaluation result. It is understandable that because the training process makes the calculation results of the AI ​​algorithm biased towards the manually labeled results, the overall evaluation result obtained using the AI ​​algorithm is more in line with user preferences.

[0009] In one possible design, the problem type evaluation result includes at least one of problem confidence, problem quality level, or problem weight; wherein the problem confidence is used to indicate the probability that the problem exists on the first image, the problem quality level is used to indicate the severity of the problem, and the weight is used to indicate the proportion of the problem on the first image.

[0010] In other words, for problems present in the first image, their type, confidence level, quality level, and weight can be determined, enabling a more accurate quantitative assessment of the problems in the image. For example, if there are N problems in the first image, the problem type assessment result includes N problem types, corresponding to N problem confidence levels, N problem quality levels, and N weights. Assuming N = 2, and the two problems correspond to overexposure and reflection, then the overexposure problem corresponds to a problem confidence level of 1 (the probability of overexposure), a problem quality level of 1 (the severity of overexposure), and a weight of 1 (the proportion of overexposure problems in the first image). The reflection problem corresponds to a problem confidence level of 2 (the probability of reflection), a problem quality level of 2 (the severity of reflection), and a weight of 2 (the proportion of reflection problems in the first image).

[0011] It should be noted that currently, image problems are mainly identified manually. However, manual identification is inefficient. Moreover, for more difficult image problems such as glare, reflections, and tree shadows, strong professional skills are required for identification, making it quite challenging. In the embodiments of this application, not only can image problems be automatically identified, but the problems can also be quantitatively evaluated, that is, parameters such as the type of problem, the confidence level of the problem, the quality level of the problem, and the weight of the problem can be determined.

[0012] In one possible design, the method further includes determining the weight based on at least one of the following: the problem quality level, the area occupied by the problem on the first image, or the location of the problem on the first image. That is, in this embodiment, the weight corresponding to a problem existing in the image can be automatically determined without manual assignment. Moreover, determining the weight based on at least one of the following—the problem quality level, the area occupied by the problem on the first image, or the location of the problem on the first image—results in high accuracy.

[0013] In this embodiment of the application, the final evaluation result obtained by quality assessment of the image is not simply equal to the comprehensive evaluation result, but is obtained by fusing the comprehensive evaluation result and the image type evaluation result to obtain the final evaluation result (i.e., the first quality level), which has high accuracy.

[0014] Specifically, determining the first quality level of the first image based on the comprehensive evaluation result and the problem type evaluation result includes: determining the second quality level of the first image based on the problem type evaluation result, and then determining the first quality level based on the second quality level and the comprehensive quality level.

[0015] The method for determining the second quality level of the first image based on the evaluation results of the problem type includes at least one of the following methods one to three:

[0016] Method 1: Determine the number of questions with a confidence level greater than a first threshold; based on this number, generate a second quality level for the first image. For example, if the question type evaluation results include N questions, then there are N corresponding question confidence levels. Assuming that M of the N question confidence levels have a confidence level greater than the first threshold (e.g., 0.5), the second quality level is generated based on M. For example, the larger M is, the higher the second quality level. This method, which generates the second quality level based on the number of questions (questions with a probability greater than the first threshold) present in the first image, is relatively simple and efficient.

[0017] Method 2: Calculate the second quality level of the first image based on the average or weighted average of the confidence scores of multiple questions. For example, if the question type assessment results include N questions, then there are N corresponding question confidence scores. Calculate the average or weighted average of the N question confidence scores, and generate the second quality level based on the average or weighted average. For example, the higher the average or weighted average, the higher the second quality level. This method generates the second quality level by averaging and weighting the probabilities of multiple questions existing in the first image, resulting in a relatively accurate second quality level.

[0018] Method 3: Calculate the second quality level of the first image based on the question quality level and its weight. For example, if the question type evaluation results include M image questions, corresponding to M image question confidence levels, then the second quality level satisfies the following formula:

[0019]

[0020] Among them, S i Let be the quality level of the i-th question in the first image, wi be the weight corresponding to the i-th question, and S be the second quality level. This method uses the weight and severity (S) of the questions present in the first image to determine the quality level. i The second quality level is generated, which is more accurate.

[0021] After obtaining the second quality level, the final quality level (i.e., the first quality level) of the first image is obtained based on the second quality level and the overall quality level. Specifically, the method for obtaining the first quality level of the first image based on the second quality level and the overall quality level includes at least one of the following:

[0022] Method A: The first quality level is the intermediate level between the overall quality level and the second quality level. For example, if the overall quality level is level 3 and the second quality level is level 1, then the first quality level is determined to be level 2.

[0023] Method B: Determining the first quality level based on the comprehensive quality level and the second quality level includes: determining the difference between the comprehensive quality level and the second quality level; and using the difference to correct the comprehensive quality level or the second quality level to obtain the first quality level. For example, the first quality level satisfies α ± (η). i ×Δ), η i This is a preset value, where Δ = |α - α′|, α is the overall quality level, and α′ is the second quality level. This method obtains the first quality level by correcting the overall quality result through the difference, i.e., Δ. Alternatively, the first quality level satisfies α′ ± (η). i ×Δ), ηi The value is a preset value, where Δ = |α - α′|, α is the overall quality level, and α′ is the second quality assessment result. This method obtains the first quality level by correcting the second quality result based on the difference, i.e., Δ.

[0024] In one possible design, the problem type is related to at least one of the following: the type of object captured in the first image, environmental information when the first image was acquired, geographical location when the first image was acquired, time information when the first image was acquired, or user-specified operation.

[0025] Let's take the issue type as an example, relating it to the type of object in the first image. If the object in the first image is a person, the issue types include exposure, underexposure, and reflection. If the object is a plant, the issue types include blur, color cast, and glare. If the object is a building (such as a road), the issue types include tree shadows and reflection. In other words,

[0026] Taking the issue type as an example, which is related to the environmental information when the first image was acquired. For instance, if the environmental information when the first image was acquired was cloudy, the issue type would include underexposure, blur, etc. If the environmental information when the first image was acquired was sunny, the issue type would include overexposure, reflection, etc.

[0027] Taking the issue type as an example, if the geographical location when the first image was acquired is a first location (e.g., a northern city), then the identified issue types include reflection, overexposure, etc. If the geographical location when the first image was acquired is a second location (e.g., a southern city), then the identified issue types include tree shadows, blur, etc.

[0028] Taking the issue type as an example, which is related to the time information when the first image was acquired. For instance, if the first image was acquired at noon, the issue type would be identified as overexposure, reflection, etc. If the first image was acquired at night, the issue type would be identified as blur, tree shadows, etc.

[0029] Let's take the example of a question type related to a user-specified action. For instance, suppose a user sets question types 1 through 3 through input. Then, the system will identify whether any questions belonging to question types 1 through 3 exist in the first image. In this way, users can set which question types they want to identify in the first image based on their needs.

[0030] In one possible design, the quality assessment of the first image to obtain a comprehensive assessment result and a problem type assessment result includes: obtaining a training image and the corresponding real assessment result;

[0031] The quality of the training image is predicted using an algorithm model to obtain a prediction evaluation result. The algorithm model includes a preprocessing module, a problem evaluation module, an attention mechanism module, a comprehensive evaluation module, and a fusion module. The preprocessing module performs at least one preprocessing step on the training image, including image feature extraction, image resizing, or multidimensional array adjustment. The problem evaluation module identifies the types of problems present in the training image, and the attention mechanism module determines the weights corresponding to the problems. The comprehensive evaluation module determines the comprehensive quality level of the training image, and the fusion module fuses the comprehensive quality level and a third quality level of the training image to obtain the prediction evaluation result. The third quality level is calculated based on the weights. When the difference between the prediction evaluation result and the true evaluation result exceeds a threshold, the parameters of the algorithm model are adjusted until the difference between the prediction evaluation result obtained by predicting the quality of the training image again using the adjusted parameters and the true evaluation result is less than the threshold, at which point training ends. The trained algorithm model is then used to evaluate the quality of the first image to obtain a comprehensive evaluation result and a problem type evaluation result.

[0032] In this embodiment, the image quality assessment device includes an algorithm model. This algorithm model is an algorithmic unit used to assess the quality of an image and obtain a comprehensive assessment result and a problem type assessment result. It can be a software module or a hardware module. The algorithm model includes a preprocessing module, a problem assessment module, an attention mechanism module, a comprehensive assessment module, and a fusion module, etc. These modules can be software modules or hardware modules, and they can be integrated on the same or different hardware devices. The algorithm model can be trained as a whole, or the individual modules within the algorithm model can be trained separately; this embodiment does not impose any limitations.

[0033] Secondly, an image quality assessment device is provided, comprising:

[0034] Image acquisition unit, used to acquire the first image;

[0035] An image quality assessment unit is used to assess the quality of the first image and obtain a comprehensive assessment result and a problem type assessment result; wherein, the comprehensive assessment result is used to indicate the overall quality level of the first image, and the problem type assessment result is used to indicate the type of problem existing in the first image, the problem type including at least one of overexposure, underexposure, blur, color cast, noise, glare, reflection or tree shadow; and a first quality level of the first image is determined based on the comprehensive assessment result and the problem type assessment result.

[0036] In one possible design, the problem type evaluation result includes at least one of problem confidence, problem quality level, or image weight;

[0037] Wherein, the problem confidence level is used to indicate the probability that the image exists in the first image, the problem quality level is used to indicate the severity of the image, and the weight is used to indicate the proportion of the problem in the first image.

[0038] In one possible design, the image quality assessment unit, when determining the first quality level of the first image based on the comprehensive assessment result and the problem type assessment result, specifically performs the following:

[0039] Determine the number of questions whose confidence level is greater than the first threshold;

[0040] Based on the quantity, a second quality level is generated for the first image;

[0041] The first quality level is determined based on the second quality level and the overall quality level.

[0042] In one possible design, the image quality assessment unit, when determining the first quality level of the first image based on the comprehensive assessment result and the problem type assessment result, specifically performs the following:

[0043] The second quality level of the first image is calculated based on the average or weighted average of the question confidence scores of multiple questions;

[0044] The first quality level is determined based on the second quality level and the overall quality level.

[0045] In one possible design, the image quality assessment unit, when determining the first quality level of the first image based on the comprehensive assessment result and the problem type assessment result, specifically performs the following:

[0046] Calculate the second quality level of the first image based on the problem quality level and weight;

[0047] The first quality level is determined based on the second quality level and the overall quality level.

[0048] In one possible design, the second quality level is an intermediate level between the first quality level and the combined quality level.

[0049] In one possible design, the image quality assessment unit, when determining the first quality level based on the second quality level and the overall quality level, specifically performs the following functions:

[0050] Determine the difference between the overall quality level and the second quality level;

[0051] The difference is used to correct the second quality level or the overall quality level to obtain the first quality level.

[0052] In one possible design, the overall evaluation result is related to the user's level of liking for the first image.

[0053] In one possible design, the problem type is related to at least one of the following: the type of object captured in the first image, environmental information when the first image was acquired, geographical location when the first image was acquired, time information when the first image was acquired, or user-specified operation.

[0054] In one possible design, the image quality assessment unit is also used for:

[0055] The weight corresponding to the problem is determined based on at least one of the following: the problem quality level, the area occupied by the problem on the first image, or the location of the problem on the first image.

[0056] In one possible design, the image quality assessment unit is specifically used for: acquiring training images and corresponding real assessment results; using an algorithm model to predict the quality of the training images to obtain predicted assessment results; wherein, the algorithm model includes a preprocessing module, a problem assessment module, an attention mechanism module, a comprehensive assessment module, and a fusion module; the preprocessing module is used to perform at least one preprocessing procedure on the training images, such as image feature extraction, image size adjustment, or multidimensional array adjustment; the problem assessment module is used to identify the types of problems existing in the training images, and the attention mechanism module is used to determine the weights corresponding to the problems; the comprehensive assessment module is used to determine the comprehensive quality level of the training images, and the fusion module is used to fuse the comprehensive quality level and a third quality level of the training images to obtain the predicted assessment results, wherein the third quality level is calculated based on the weights; when the difference between the predicted assessment results and the real assessment results is greater than a threshold, the parameters of the algorithm model are adjusted until the difference between the predicted assessment results obtained by predicting the quality of the training images again using the adjusted parameters and the real assessment results is less than the threshold, thus ending the training; the trained algorithm model is used to perform quality assessment on the first image to obtain a comprehensive assessment result and a problem type assessment result.

[0057] Thirdly, an image quality assessment apparatus is also provided. This image quality assessment apparatus includes a processor for implementing the method described in the first aspect. The apparatus may further include a memory for storing programs and instructions. The memory is coupled to the processor, and when the processor executes the program instructions stored in the memory, it can implement the method described in the first aspect. The apparatus may also include an interface circuit, through which the processor is coupled to the memory, and the processor can execute the program code in the memory to implement the method described in the first aspect. Alternatively, the interface circuit is also used for communication between the apparatus and other devices; for example, the interface circuit may be a transceiver, circuit, bus module, or other type of interface circuit.

[0058] For example, the image quality assessment device can be a server (such as a map server) or a chip installed in a server. The transceiver is implemented, for example, through an antenna, feeder, and codec in the server. Alternatively, if the image quality assessment device is a chip installed in a server, the interface circuit is, for example, a communication interface in the chip, which is connected to a radio frequency transceiver component in the server to transmit and receive information via the radio frequency transceiver component. Alternatively, the image quality assessment device can also be a terminal device, a vehicle, or an in-vehicle device, or a chip installed in a terminal device, vehicle, or in-vehicle device. For example, the transceiver is implemented, for example, through an antenna, feeder, and codec in the in-vehicle device; or, if the image quality assessment device is a chip installed in an in-vehicle device, the interface circuit is, for example, a communication interface in the chip, which is connected to a radio frequency transceiver component in the in-vehicle device to transmit and receive information via the radio frequency transceiver component.

[0059] Fourthly, a computer-readable storage medium is also provided, including computer instructions that, when executed on a computer, cause the computer to perform the method provided in the first aspect above.

[0060] Fifthly, a computer program product is also provided, the computer program product comprising a computer program that, when run on a computer, causes the computer to perform the method provided in the first aspect above.

[0061] For the beneficial effects of the second to fifth aspects mentioned above, please refer to the beneficial effects of the first aspect, which will not be repeated here. Attached Figure Description

[0062] Figure 1 A schematic diagram of a convolution operation provided in an embodiment of this application;

[0063] Figure 2 A schematic diagram illustrating the principle of the attention mechanism provided in one embodiment of this application;

[0064] Figure 3A A schematic diagram illustrating an application scenario provided in one embodiment of this application;

[0065] Figure 3B A schematic diagram illustrating another application scenario provided by an embodiment of this application;

[0066] Figure 4A This is a functional block diagram of an image quality assessment device provided in an embodiment of this application;

[0067] Figure 4B Another functional block diagram of an image quality assessment device provided in an embodiment of this application;

[0068] Figure 5 A schematic flowchart illustrating an image quality assessment method provided in an embodiment of this application;

[0069] Figure 6 A schematic diagram illustrating the calculation of the comprehensive evaluation result using a convolutional neural network in the first branch of an embodiment of this application;

[0070] Figure 7 A flowchart illustrating the training process of an attention mechanism provided in an embodiment of this application;

[0071] Figure 8 A schematic diagram of an image quality assessment device provided in an embodiment of this application;

[0072] Figure 9 Another schematic diagram of an image quality assessment device provided in an embodiment of this application. Detailed Implementation

[0073] To facilitate understanding, the relevant terms involved in the embodiments of this application will be introduced below.

[0074] (1) Artificial intelligence (AI) algorithms

[0075] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence. It aims to acquire knowledge and use that knowledge to achieve optimal results. AI seeks to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. In essence, AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0076] Generally speaking, AI algorithms include decision trees, logistic regression (LR), naive bayes (NB) classification algorithms, random forest (RF) algorithms, support vector machines (SVM) classification algorithms, histogram of oriented gradients (HOG), neural networks, deep neural networks, convolutional neural networks (CNN), and so on.

[0077] The following section provides a brief introduction using AI algorithms such as neural networks, deep neural networks, and convolutional neural networks as examples.

[0078] (2) Neural Network

[0079] Neural networks can be composed of neural units. Simply put, a neural unit is a computational unit with computational capabilities, and the output of the computational unit can be:

[0080]

[0081] Where, x s The input signal is s = 1, 2, ..., n, where n is a natural number greater than 1, and W s For x s The weights are denoted by b, the bias of the neural unit is denoted by b, and the activation functions are denoted by f, which convert the input signal into the neural unit into the output signal. As an example, the activation function can be the sigmoid function, the hyperbolic tangent function (Tanh() function), the rectified linear unit (ReLU) function, etc., and no restrictions are placed on the activation function here.

[0082] A neural network can be a network formed by connecting many of the above-mentioned single neural units together. Different neural units in a neural network can be connected to each other. For example, the output of one neural unit can be the input of another neural unit. In this way, the final output signal is obtained by passing through all the neural units one by one.

[0083] (3) Deep Neural Networks

[0084] Deep Neural Networks (DNNs), also known as multi-layer neural networks, generally consist of three layers: input layer, hidden layer, and output layer. The first layer is the input layer, which receives input data, and the last layer is the output layer, which outputs data. The layers between the first and last layers are hidden layers. All layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer. Although DNNs appear complex, the operation of each layer is actually not complicated; it can be simply described by the following linear relationship expression:

[0085]

[0086] in, It is the input vector. It is the output vector. α is the offset vector, W is the weight matrix (also called coefficients), and α() is the activation function. Each layer is simply an adjustment of the input vector. The output vector is obtained after the above linear transformation. It's important to note that the input layer does not have a W parameter. Because DNNs have many layers, the weight matrix W and the offset vector... The number of hidden layers is thus large, and the process of training a deep neural network is essentially the process of learning the weight matrix W, making the output of the trained deep neural network closer to the ideal result. In deep neural networks, more hidden layers allow the network to better depict complex situations in the real world. Theoretically, the more layers a model has, the higher its complexity and the greater its "capacity," meaning it can complete more complex learning tasks.

[0087] (4) Convolutional Neural Network

[0088] A Convolutional Neural Network (CNN) is a deep neural network with a convolutional structure. The calculation method for each layer in a Deep Neural Network (DNN) described in (3) above is based on the formula (2) above. Unlike the DNN, the weights W in a CNN are replaced by convolutional kernels. A convolutional kernel can be a two-dimensional or three-dimensional matrix used to perform convolution operations with input data (such as an output image) to extract feature information from the input data. If the input data is an image, the extracted feature information can be color, brightness, exposure, grayscale, and other features of the image. For ease of understanding, a simple example is used to illustrate the convolution operation process.

[0089] Please see Figure 1 The diagram shown is a schematic representation of the convolution operation provided in an embodiment of this application. Figure 1As shown, the input image and the convolution kernel are convolved to obtain the output image. Figure 1 In this example, the input image is a 4x4 matrix, and the convolution kernel is a 3x3 matrix. The convolution kernel moves from left to right and from top to bottom on the input image to complete the convolution operation. Specifically, the convolution operation process is as follows:

[0090] v1=p1*k1+p2*k2+p3*k3+p5*k4+p6*k5+p7*k6+p9*k7+p10*k8+p11*k9;

[0091] v2=p3*k1+p4*k2+0*k3+p7*k4+p8*k5+0*k6+p11*k7+p12*k8+0*k9;

[0092] v3=p9*k1+p10*k2+p11*k3+p13*k4+p14*k5+p15*k6+0*k7+0*k8+0*k9;

[0093] v4=p11*k1+p12*k2+0*k3+p15*k4+p16*k5+0*k6+0*k7+0*k8+0*k9;

[0094] In simple terms, convolving p1, p2, p3, p5, p6, p7, p9, p10, and p11 in the input image with the convolution kernel yields v1 in the output image; convolving p2, p3, p4, p6, p7, p8, p10, p11, and p12 in the input image with the convolution kernel yields v2 in the output image, and so on.

[0095] Generally, the neural network inside a CNN can be divided into three categories: input layer, hidden layer, and output layer. Each layer corresponds to a feature extractor, which includes a convolutional kernel. The same layer can correspond to one or more feature extractors, i.e., one or more convolutional kernels. The feature extractors for different layers can be the same or different. Optionally, the input layer may not have a feature extractor. Each layer can use its corresponding feature extractor to extract features from the input image (i.e., perform a convolution operation between the convolutional kernel in the feature extractor and the input image; see [link to implementation details]). Figure 1 ).

[0096] pass Figure 1As shown in the convolution process, different locations on the input image within the same layer can share a single feature extractor, i.e., share the same convolution kernel (also known as shared weights). Generally, the more convolution kernels there are, the richer the image information reflected by the convolution operation, which can improve the accuracy of image recognition in subsequent processes. Moreover, a specific feature extractor (i.e., a specific convolution kernel) can extract specific feature information from the input data. Therefore, the design of the feature extractor is particularly important, and the training process of a CNN is essentially the process of training the convolution kernels in the feature extractor.

[0097] (5) Loss Function

[0098] During CNN training, to ensure the CNN's output is as close as possible to the desired value, we compare the current output with the target value and update the weights of each layer based on the difference. (Of course, there's usually an initialization process before the first update, where parameters are pre-configured for each layer). For example, if the output value is too high, the weights are adjusted to lower it, and this adjustment continues until the desired target value or a value very close to it is obtained. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the output and target values. Taking the loss function as an example, a higher loss value indicates a greater difference, so the training process becomes a process of minimizing this loss as much as possible.

[0099] (6) Attention mechanism

[0100] In cognitive science, selectively focusing on a subset of all information while ignoring other visible information is often referred to as an attention mechanism. For example, when reading, people typically only pay attention to and process a small number of words. Therefore, one approach is to suggest that when an attention mechanism is introduced into a neural network, it enables the network to focus on a subset of its input (or features), such as selecting specific inputs. Thus, the attention mechanism is a resource allocation scheme that addresses information overload by allocating computational resources to more important tasks when computational power is limited.

[0101] In neural network learning, generally speaking, the more parameters a model has, the stronger its expressive power and the greater the amount of information it stores. However, this can lead to information overload, and excessive information can result in a lower focus on key information. Therefore, by introducing an attention mechanism, we can focus on information more critical to the current task from a large amount of input information, reducing attention to other information and even filtering out irrelevant information. This can solve the information overload problem and improve the efficiency and accuracy of task processing.

[0102] Attention mechanisms include soft attention and hard attention. Hard attention can be understood as focusing on a point or region, which is either selected or not. Soft attention, on the other hand, focuses on all regions, but the degree of importance given to different regions varies. This article primarily uses soft attention as an example.

[0103] Soft attention mechanisms include various methods such as global attention, local attention, and non-local blocking. The following section will use global attention as an example.

[0104] Please see Figure 2 As shown, X = [x1,…,xN] represents N sets of input information, where each vector xi, i∈[1,N], represents a set of input information. In order to select information related to a specific task from the N input vectors [x1,…,xN], a task-related representation, called a query vector, needs to be introduced, and the correlation between each input vector and the query vector needs to be calculated. For example, it can be calculated using a scoring function. For example, the probability αi of selecting the i-th input vector given q and X is calculated, and the probability αi satisfies the following formula (1):

[0105]

[0106] Where αi is called the attention distribution (attention distribution αi can be interpreted as the degree of attention received by the i-th input vector when given a task-related query q), and s(xi,q) is the attention scoring function, which can be calculated in various ways, such as algebraic model, dot product model, scaled dot product model, bilinear model, etc., which are not limited in the embodiments of this application.

[0107] Therefore, the correlation between each input vector and the query vector (given task) can be determined through the attention mechanism, i.e., the probability αi mentioned above. The higher the probability αi, the greater the correlation.

[0108] The image quality assessment method provided in the embodiments of this application is described below.

[0109] There are currently two image quality assessment schemes.

[0110] The first image quality assessment scheme uses an image quality assessment algorithm to evaluate image quality. This algorithm is an AI algorithm. The calculation principle involves first training the AI ​​algorithm, and then using the trained AI algorithm to assess image quality. The training process includes: Step 1, acquiring training images and manually assessing their quality, for example, manually assigning quality scores. Step 2, using the algorithm to calculate the training results on the training images, which are the quality scores calculated by the AI ​​algorithm. If the training results are inconsistent with the manually assigned quality scores, the calculation parameters of the AI ​​algorithm are adjusted until the training results obtained by the AI ​​algorithm are consistent with the manually assigned quality scores, thus ending the training. Therefore, the training process can be understood as adjusting the calculation parameters of the AI ​​algorithm to make the training results obtained by the AI ​​algorithm consistent with the manually assigned results. After training is complete, the AI ​​algorithm (the trained AI algorithm) is used to assess image quality. It is understandable that because the training process causes the AI ​​algorithm's calculation results to be biased towards the manually assigned results, and human labeling results vary from person to person and often reflect user preferences, this scheme yields image quality assessment results that align with user preferences.

[0111] This approach has certain drawbacks: Firstly, the manually labeled images used for training are not objective. Since everyone has different preferences, the labeling results for the same image may differ. Therefore, using manually labeled images to train the algorithm can make it biased towards human preferences. Secondly, AI algorithms can only provide a comprehensive evaluation of the image, such as a quality score, without knowing the image details (e.g., what image problems exist). Therefore, this approach has relatively low accuracy.

[0112] The second image quality assessment method involves manually identifying image problems and assessing image quality based on these problems. For example, more image problems result in lower image quality, while fewer problems result in higher image quality. However, relying on manual identification is inefficient. Moreover, identifying more difficult image problems such as glare, reflections, and tree shadows requires strong professional skills, making this method less applicable and unable to achieve quantification.

[0113] In view of this, this application provides an image quality assessment method. This method is applicable to image quality assessment devices. Specifically, the image quality assessment device acquires a first image (which can be understood as the image to be assessed), performs a quality assessment on the first image, and obtains a comprehensive assessment result and a problem type assessment result. The comprehensive assessment result indicates the overall quality level of the first image, and the problem type assessment result indicates the type of image problem (which can be simply referred to as a problem) existing in the first image. The image problem type includes at least one of overexposure, underexposure, blur, noise, glare, color cast, reflection, or tree shadow. Based on the comprehensive assessment result and the problem type assessment result, a first quality level of the first image is determined. That is, the final assessment result (i.e., the first quality level) obtained by the image quality assessment method provided in this application is not simply equal to the comprehensive assessment result, but rather is obtained by fusing the comprehensive assessment result and the image type assessment result, resulting in higher accuracy.

[0114] The method provided in this application can be applied to image quality assessment devices (or image assessment devices, image detection devices, image recognition devices, etc., which are not limited herein). The image quality assessment device can be a mobile terminal, such as a mobile phone, tablet computer, laptop computer, PDA, mobile internet device (MID), wearable device, such as a smartwatch, smart bracelet, pedometer, etc. It can also be a vehicle or vehicle device, road test unit, cloud (such as a server), etc.

[0115] The following describes several application scenarios applicable to the embodiments of this application.

[0116] The first scenario is the autonomous driving scenario.

[0117] Please see Figure 3AThe vehicle 101 is equipped with an image acquisition module for acquiring images, including objects ahead of the road. The vehicle 101 can identify objects on the road ahead through image recognition. To improve accuracy, before image recognition, the image quality assessment method provided in this embodiment can be used to assess the image quality. If the assessment result is good, the image is recognized; otherwise, it is not recognized, avoiding dangerous driving due to inaccurate recognition results. Alternatively, the vehicle 101 can send the acquired images to the cloud, where the cloud uses the image quality assessment method provided in this embodiment to assess the image quality and returns the assessment result to the vehicle, reducing the vehicle's computational burden. In other words, the image quality assessment method provided in this embodiment can filter numerous acquired images, retaining and recognizing images with higher quality, improving the accuracy of the recognition results and ensuring driving safety.

[0118] The second scenario is the roadside unit scenario.

[0119] Please see Figure 3B An image acquisition module can be installed on the roadside unit 104. This image acquisition module is used to acquire images in real time, such as... Figure 3B The images acquired by the roadside unit 104 include the road surrounding the roadside unit and the vehicles 101 traveling on the road. The roadside unit 104 can determine information about the vehicles 101 on the road (such as license plate numbers) through image recognition, or it can determine whether the vehicles 101 are driving legally (such as whether the vehicles are crossing lane lines or speeding). Before recognizing an image, the roadside unit 104 can also perform a quality assessment on the image, for example, using the image quality assessment method provided in this application. Alternatively, the roadside unit 104 can send the acquired images to other devices, such as the cloud, where the other devices can perform an image quality assessment using the lane line attribute detection method provided in this application and transmit the assessment results back to the roadside unit. If the assessment result is good, the image is recognized (e.g., to determine whether the vehicle is driving legally); otherwise, the image is not recognized to avoid inaccurate recognition results.

[0120] The two scenarios mentioned above are merely examples. The image quality assessment method provided in this application can be applied to many more scenarios, which will not be listed here.

[0121] The image quality assessment method provided in this application will be described in detail below with reference to the accompanying drawings.

[0122] Please see Figure 4A This is a functional block diagram of an image quality assessment device provided in an embodiment of this application. Figure 4AAs shown, the image quality assessment device includes an image acquisition module, a first branch, a second branch, and a fusion module. The image acquisition module refers to a device capable of acquiring images, such as a camera or camcorder. The image quality assessment device may or may not include an image acquisition module; this embodiment does not limit this. In the case where the image quality assessment device does not include an image acquisition module, images can be acquired from other devices, and then the two branches can be used to perform image quality assessment.

[0123] The first branch can be understood as a first algorithm unit in the image quality assessment device. It can be a software module or a collection of software modules in the image quality assessment device, or an integration of a hardware module or multiple hardware modules in the image quality assessment device (such as a chip system composed of one or more chips), etc., which is not limited in the embodiments of this application. The second branch can be understood as a second algorithm unit in the image quality assessment device. It can be a software module or a collection of software modules in the image quality assessment device, or an integration of a hardware module or multiple hardware modules in the image quality assessment device (such as a chip system composed of one or more chips), etc. The first branch and the second branch can be integrated on the same software or hardware module, or the two branches can be independently arranged in different software or hardware modules. The fusion module is used to determine the final evaluation result of the image based on the evaluation results of the two branches. The fusion module can be understood as an algorithm unit in the image quality assessment device. It can be a software module or a collection of software modules in the image quality assessment device, or an integration of one or more hardware modules. The fusion module, the first branch, and the second branch can be integrated on the same software or hardware module, or they can be independently arranged in different software or hardware modules.

[0124] In this embodiment, to improve the accuracy of image quality assessment, different branches can be used to perform quality assessments on the image from different dimensions. For example, the first branch performs a quality assessment on the image to obtain a comprehensive assessment result, while the second branch performs a quality assessment on the image to obtain a problem type assessment result. The comprehensive assessment result indicates the overall quality level of the image, while the problem type assessment result indicates the type of image problem present, such as overexposure, underexposure, overexposure, underexposure, blur, noise, color cast, glare, reflection, tree shadows, etc. The fusion of the comprehensive assessment result and the problem type assessment result yields the final image quality assessment result, thus improving the accuracy of image quality assessment.

[0125] Please see Figure 4B This is another functional block diagram of the image quality assessment device provided in the embodiments of this application. Figure 4B It can be understood as Figure 4A The refinement, for example, comparison Figure 4A and Figure 4B As can be seen, the first branch includes an image problem evaluation module and an attention mechanism module, while the second branch includes an image comprehensive evaluation module. The image problem evaluation module is used to determine the types of image problems present in the image, and the attention mechanism is used to determine the weights corresponding to the image problems. The attention mechanism and weights will be introduced later.

[0126] The image quality assessment device also includes a preprocessing module for preprocessing images acquired by the image acquisition module. For example, the preprocessing module includes three modules: a CNN backbone network, adaptive average pooling, and tensor flattening. The CNN backbone network is used to extract features from the image (such as pixel grayscale, brightness, and color information), such as ResNet models. The adaptive average pooling module ensures that the output size remains unchanged for different input images. The tensor flattening module expands a multidimensional array into a one-dimensional array, which is composed of the features extracted from the image. This one-dimensional array is the input data for the image problem assessment module and the image comprehensive assessment module; this array dimensionality reduction simplifies the processing flow of these modules. The preprocessing module and the image acquisition module are optional (so...). Figure 4B (The dashed lines are used to indicate the meaning), but this application does not limit the scope of the embodiments.

[0127] Please see Figure 5 This is a flowchart illustrating an image quality assessment method provided in an embodiment of this application. Figure 5 As shown, the process of the method includes:

[0128] S100, the image acquisition module acquires the first image and sends the first image to the first branch and the second branch.

[0129] An image quality assessment device may include an image acquisition module, such as a camera. In this case, acquiring the first image means that the image acquisition module acquires the first image. Alternatively, the image quality assessment device may not include an image acquisition module. In this case, acquiring the first image may mean receiving the first image from another device.

[0130] S200, the first branch performs a quality assessment on the first image to obtain a comprehensive assessment result. This comprehensive assessment result indicates the overall quality level of the first image.

[0131] For example, image quality is divided into N levels, with higher levels representing higher image quality. For instance, N=5, representing levels one through five, with level one being the worst and level five the best. Optionally, different levels can correspond to different identifiers. For example, the overall evaluation result can be described using an integer value from 1 to -5, such as 5 indicating level five; or, the overall evaluation result can be described using a score from 1 to 50, such as a score between 40 and 50 indicating level five; or, the overall evaluation result can be described using a five-digit number, such as 00001 indicating level five, while 10000 indicates level one. Of course, other level representation methods are also possible, and this application does not limit the specific representation.

[0132] Optionally, the overall evaluation result can also indicate the confidence level of the overall quality level of the first image, that is, the probability of the overall quality level of the first image. For example, if the overall quality level is level 5, the corresponding confidence level is 0.6, which means that the probability of the first image being level 5 is 0.6.

[0133] In some embodiments, the overall evaluation result of the first image is related to the user's degree of liking for the first image. For example, the higher the user's degree of liking for the first image, the higher the first quality level. Therefore, S200 can be refined to: analyzing the user's degree of liking for the first image and determining the overall evaluation result based on the degree of liking.

[0134] Example 1: Analyzing a user's preference for a first image and determining a comprehensive evaluation result based on that preference may include: determining the difference between the shooting parameters of the first image and the recorded shooting parameters preferred by the user, and determining the comprehensive evaluation result based on that difference. If the difference between the shooting parameters of the first image and the recorded shooting parameters preferred by the user is large, it indicates that the first image does not meet the user's preferences, and the comprehensive quality level indicated by the comprehensive evaluation result is low. If the difference between the shooting parameters of the first image and the recorded shooting parameters preferred by the user is small, it indicates that the first image meets the user's preferences, and the comprehensive quality level indicated by the comprehensive evaluation result is high. It is understood that, prior to this, the user's preferred shooting parameters can be recorded. For example, shooting parameters frequently used by the user when taking images can be recorded as the user's preferred shooting parameters. These shooting parameters include, for example, aperture size, exposure value, white balance value, etc.

[0135] Example 2, analyzing the user's preference for the first image and determining the overall evaluation result based on the preference level, may include: determining whether the subject included in the first image is a subject the user likes; if so, the first image matches the user's preferences, and the overall quality level indicated by the overall evaluation result is high. If not, the first image does not match the user's preferences, and the overall quality level indicated by the overall evaluation result is low. It is understood that prior to this, the process may also include the step of recording the user's favorite subjects, where the subject can be one or more objects or object types. For example, if it is determined that there are many images in the image library containing a certain subject (such as a cat), then that subject is determined to be a favorite subject of the user, or the object type to which that subject belongs (such as an animal type) is determined to be a favorite subject of the user.

[0136] In some embodiments, the first branch can use a first algorithm to perform quality assessment on the first image to obtain a comprehensive evaluation result. The first algorithm can be an AI algorithm, such as decision tree, logistic regression (LR), naive Bayes (NB) classification algorithm, random forest (RF) algorithm, support vector machine (SVM) classification algorithm, histogram of oriented gradients (HOG), neural network, deep neural network, convolutional neural network, etc. The following describes the process of using a convolutional neural network as the first algorithm to perform quality assessment on the first image.

[0137] First, let's introduce the input and output data of the first algorithm.

[0138] Input data includes a first image or a feature vector extracted from the first image. A feature vector is a vector composed of one or more feature information extracted from the first image. These feature information may include, for example, color, brightness, grayscale, etc.

[0139] Output data and a comprehensive evaluation result. The comprehensive evaluation result indicates the overall quality level of the first image. As an example, the comprehensive evaluation result can be an image quality label, which can be represented as: {S1, S2, S3, S4, ..., S5}. That is, five levels from S1 to S5. If the overall quality level is the first level, then the quality label can be represented as {1, 0, 0, 0, ..., 0}, representing the first level. If the overall quality level is the second level, then the quality label can be represented as {0, 1, 0, 0, ..., 0}, representing the second level.

[0140] For details, please see Figure 6 The diagram shown is an exemplary structural schematic of a convolutional neural network provided in an embodiment of this application. Figure 6 As shown, a Convolutional Neural Network (CNN) can include N processing layers, where N is an integer greater than or equal to 3. The first layer of the CNN is the input layer 201, responsible for receiving input data. The last layer of the CNN is the output layer 203, used to output the processing result of the CNN. The other layers besides the first and last layers are hidden layers 202, which can include one or more layers. Figure 6 In the middle, hidden layer 202 includes multiple layers (such as...) Figure 6 (As shown in the dashed box in the image) Taking this as an example, each layer in hidden layer 202 can both receive input data and output calculation results. Figure 6 Taking the CNN shown as an example, the process by which a CNN processes input data to obtain output data includes:

[0141] Input layer 201 receives input data, such as an image, and propagates it forward to the first hidden layer. The first hidden layer corresponds to the first convolutional kernel. The input data is convolved using the first convolutional kernel to obtain the first convolution result. The process of using the convolutional kernel to perform convolution calculations on the input data is explained in the glossary section above. The value of the first convolutional kernel can be predefined or obtained through training. The first hidden layer continues to propagate the first convolution result forward to the second hidden layer. The second hidden layer corresponds to the second convolutional kernel. The input data passed to the second hidden layer is convolved using the second convolutional kernel to obtain the second convolution result. This second convolution result serves as the input to the next hidden layer, and so on. After calculations through multiple hidden layers, the image information is richer, which is more helpful for further image recognition. For example, the last hidden layer passes its convolution result to output layer 203, which performs image quality evaluation to obtain a comprehensive evaluation result, i.e., the image quality label. The convolution processing of the image by hidden layer 202 enriches the image information and improves the accuracy of image quality evaluation in the output layer.

[0142] The first algorithm employs training methods including supervised training and unsupervised training. Taking supervised training as an example, the training set includes training images and corresponding comprehensive evaluation results. These comprehensive evaluation results can be manually labeled and used to indicate the overall quality level of the training images. After the input layer 201 acquires the training set, it propagates forward to the output layer 203. The output layer 203 performs quality evaluation on the training images to obtain the training results. The output layer 203 contains a loss function. The loss function calculates the difference between the training results and the manually labeled results. If the difference is large, backpropagation (e.g., ...) is performed. Figure 6The propagation from the output layer 203 to the input layer 201 shown is backpropagation. The convolutional kernels corresponding to each layer in the hidden layer 202 are updated. The updated CNN recalculates on the training set and obtains the training result of the training image again. If the difference between the obtained training result and the manually labeled result of the training image is small, training stops. Otherwise, backpropagation continues to update the convolutional kernels corresponding to each layer in the hidden layer 202 until the error between the training result obtained by the updated CNN and the manually labeled result is small.

[0143] Therefore, if images from all scenes and all weather conditions are used as the training set to train a CNN, the resulting CNN will be suitable for image quality assessment in all scenes and all weather conditions.

[0144] S300, the first branch sends the comprehensive evaluation results to the fusion module.

[0145] S400, the second branch performs a quality assessment on the image to obtain a problem type assessment result. This problem type assessment result indicates the type of image problem present in the first image. The image problem type includes at least one of the following: overexposure, underexposure, overexposure, underexposure, blur, noise, color cast, glare, reflection, or tree shadows. It should be noted that the image problem types listed here are merely examples; in practical applications, many more types of image problems may be included, and this embodiment does not limit the scope of the application.

[0146] Optionally, the image problem type is related to at least one of the following: the type of object in the first image, the environmental information when the first image was acquired, the geographical location when the first image was acquired, the time information when the first image was acquired, or the user-specified operation.

[0147] For example, consider the relationship between image problem type and the type of object in the first image. This can be understood as follows: prior to S400, the process could include determining the corresponding image problem type based on the type of object in the first image. For instance, if the object in the first image is a person, the determined image problem types include overexposure, underexposure, and reflections, indicating that for images of people, more attention is paid to image problems such as overexposure, underexposure, and reflections. If the object in the first image is a plant, the determined image problem types include blur, color cast, and glare. If the object in the first image is a building (such as a road), the determined image problem types include tree shadows and reflections.

[0148] For example, consider the image problem type being related to the environmental information at the time of acquiring the first image. This can be understood as follows: prior to S400, the process could also include the step of determining the corresponding image problem type based on the environmental information at the time of acquiring the first image. For instance, if the environmental information at the time of acquiring the first image is cloudy, then the determined image problem types include underexposure, blur, etc. This indicates that for images taken on cloudy days, more attention is paid to image problems such as underexposure and blur. If the environmental information at the time of acquiring the first image is sunny, then the determined image problem types include overexposure, reflection, etc. This indicates that for images taken on sunny days, more attention is paid to image problems such as overexposure and reflection.

[0149] For example, consider the image problem type being related to the geographical location when the first image was acquired. This can be understood as follows: before step S400, a step could be included: determining the corresponding image problem type based on the geographical location when the first image was acquired. The geographical location could be a GPS location, such as different cities or countries. For instance, if the geographical location when the first image was acquired was a first location (e.g., a northern city), then the determined image problem types would include reflections, overexposure, etc., indicating that for images taken at the first location, more attention would be paid to image problems such as reflections and overexposure. If the geographical location when the first image was acquired was a second location (e.g., a southern city), then the determined image problem types would include tree shadows, blur, etc., indicating that for images taken at the second location, more attention would be paid to image problems such as tree shadows and blur.

[0150] For example, consider an image problem type related to the time information when the first image was acquired. This can be understood as follows: before step S400, a step could be included: determining the corresponding image problem type based on the time information when the first image was acquired. For instance, if the first image was acquired at noon, the determined image problem types would include overexposure, reflection, etc. If the first image was acquired at night, the determined image problem types would include blur, tree shadows, etc.

[0151] For example, consider an image problem type related to a user-specified operation. This can be understood as follows: before S400, the process could include the step of determining the corresponding image problem type based on the user-specified operation. For instance, suppose the user sets image problem type 1 to image problem type 3 through an input operation. Then, the image quality assessment device identifies whether image problems of type 1 to type 3 exist in the first image. In this way, the user can set which image problem types they want to identify in the first image according to their needs.

[0152] In some embodiments, the second branch can use a second algorithm to perform quality assessment on the first image to obtain an image problem type assessment result. The second algorithm can be an AI algorithm, such as decision tree, logistic regression (LR), naive Bayes (NB) classification algorithm, random forest (RF) algorithm, support vector machine (SVM) classification algorithm, histogram of oriented gradients (HOG), neural network, deep neural network, convolutional neural network, etc. The following describes the process of using a convolutional neural network as the second algorithm to perform quality assessment on the first image.

[0153] First, let's introduce the input and output data of the second algorithm.

[0154] Input data includes a first image or a feature vector extracted from the first image. A feature vector is a vector composed of one or more feature information extracted from the first image. These feature information may include, for example, color, brightness, grayscale, etc.

[0155] Output data: Image problem evaluation results. These results indicate the types of image problems present in the first image.

[0156] by Figure 6 Taking the CNN shown as an example, the process of processing input data to obtain output data includes: Input layer 201 receives input data, such as an image, and propagates the input data forward to the first hidden layer. The first hidden layer corresponds to the first convolutional kernel, and performs a convolution operation on the input data corresponding to the first convolutional kernel to obtain the first convolution result. The first hidden layer continues to propagate the first convolution result forward to the second hidden layer, which corresponds to the second convolutional kernel. Performs a convolution operation on the input data passed to the second hidden layer to obtain the second convolution result, which serves as the input to the next hidden layer, and so on. The last hidden layer passes its convolution result to the output layer 203, which performs image problem identification. For example, the feature information (such as color, brightness, grayscale, etc.) in the convolution result of the last hidden layer is compared with a threshold to identify the type of image problem present in the image. For example, if the detected brightness is higher than threshold 1, an overexposure problem is identified; if the brightness is lower than threshold 2 (threshold 2 is less than threshold 1), an underexposure problem is identified. Because the image information (the convolution result of the last hidden layer) is richer after multiple hidden layers of computation, the output layer has higher accuracy in recognizing image problems.

[0157] Optionally, the problem type evaluation result includes at least one of image problem confidence, image problem quality level, or image problem weight. The image problem confidence indicates the probability of an image problem existing in the first image, the image problem quality level indicates the severity of the image problem, and the weight indicates the proportion of the image problem in the first image. That is, one image problem corresponds to a set of parameters, which includes the type of image problem, and at least one of image problem confidence, image problem quality level, and image problem weight. Assume there are M image problems in the first image, corresponding to M sets of parameters.

[0158] (1) Image problem confidence; used to indicate the probability that the image problem exists in the first image.

[0159] Suppose there are M image problems on the first image, corresponding to M image problem confidence levels (i.e., one image problem corresponds to one image problem confidence level). The image problem confidence level can take values ​​within the range [0,1]. For example, an image problem confidence level of 1 indicates the existence of the image problem; an image problem confidence level of 0 indicates the non-existence of the image problem; and an image problem confidence level of 0.6 indicates that the probability of the image problem existing is 0.6.

[0160] For ease of description, image problem labels can be generated based on the confidence scores of M image problems, as shown in Table 1 below, which is an exemplary representation of image problem labels:

[0161] Table 1: Image Problem Tags

[0162] S1 S2 S3 … SM

[0163] In this context, confidence level 1 (S1) indicates the probability that image problem 1 exists in the first image; confidence level 2 (S2) indicates the probability that image problem 2 exists in the first image; and so on, confidence level M (SM) indicates the probability that image problem M exists in the first image. For example, assuming M = 5, then the image problem label has 5 digits. The image problem labels for the first image are shown in Table 2 below:

[0164] Table 2

[0165] 0 0 0 1 1

[0166] That is, the image problem label is 00011, which means that image problems 4 to 5 exist in the first image, and image problems 1 to 3 do not exist.

[0167] (2) Image problem quality level, which indicates the severity of the image problem.

[0168] Suppose there are M image problems in the first image, corresponding to M image problem quality levels (i.e., one image problem corresponds to one image problem quality level). The image problem quality level can be a numerical score, such as 0-100, with higher scores indicating more severe image problems.

[0169] (3) Image problem weights, used to indicate the proportion of the image problems in the first image.

[0170] The weights of image problems can be determined in several ways, such as: Method A, determining the weight based on the image problem quality level. For example, establishing a mapping relationship between image problem quality level and weight, and determining the weight based on this mapping relationship. For instance, a higher image problem quality level indicates a more severe image problem, and therefore a higher weight. Method B, determining the weight based on the area occupied by the image problem on the first image. For instance, the larger the area of ​​an image problem on the first image, the higher the weight corresponding to that image problem. Method C, determining the weight based on the location of the image problem on the first image. For instance, if the image problem is in the center area or near the center of the first image, then the image problem has a higher weight; if the image problem is in the edge area or near the edge of the first image, then the image problem has a lower weight. It should be understood that Methods A to C can be used individually or in combination of any two or more, and this application embodiment does not limit this.

[0171] The following section uses the attention mechanism as an example to introduce the process of automatically determining the weights of an image problem.

[0172] First, let's introduce the input parameters and output data of the attention mechanism.

[0173] The input data can be a first image and image problem labels identified from the first image. Please refer to the previous section for an introduction to image problem labels. Assuming there are M image problems in the first image, the image problem labels are {S1, S2, ..., SM}. Here, S1 indicates the probability of image problem 1 existing in the first image; S2 indicates the probability of image problem 2 existing in the first image; and so on, SM indicates the probability of image problem M existing in the first image.

[0174] The output data can be the weights corresponding to the image problem.

[0175] Taking the global attention algorithm as an example, as described above, in order to select information related to a specific task from M input vectors [x1,…,xN], a task-related representation, called a query vector, needs to be introduced. Therefore, to obtain the correlation between the first image and image problem 1, a query vector q = q1 can be introduced. Using the formula (1) above, α1 can be obtained, where α1 is the weight 1 corresponding to image problem 1. Similarly, to obtain the correlation between the first image and image problem 2, a query vector q = q2 is introduced. Using the formula (1) above, α2 can be obtained, where α2 is the weight 2 corresponding to image problem 2. Likewise, to obtain the correlation between the first image and image problem n, a query vector q = qn is introduced. Using the formula (1) above, αn can be obtained, where αn is the weight n corresponding to image problem n.

[0176] The query vector Q can be pre-set (e.g., the default setting before shipment) or it can be obtained through training. For the training process, please refer to [link to training documentation]. Figure 7 As shown, it includes:

[0177] Step 1: Set the learning parameters for the attention mechanism, such as Q.

[0178] Step 2: Obtain the training images and the corresponding image problem labels for the training images. The image problem labels may include one or more manually specified image problems.

[0179] Step 3: Input the training images and their corresponding image question labels into the attention mechanism to obtain the weights corresponding to the image questions. For example, if the image question labels include M image questions, then M weights are obtained.

[0180] Step 4: Based on the determined weights and image problem labels, predict the quality score.

[0181] Assuming there are M images in the problem, the predicted quality scores satisfy the formula:

[0182]

[0183] Among them, S i is the score of the i-th image problem existing on the training image (e.g., manually assigned), wi is the weight corresponding to the i-th image problem, and S is the predicted quality score.

[0184] Step 5: Determine whether the difference between the predicted quality score and the manually labeled image quality label exceeds a threshold. The image quality label refers to the quality level or quality score of the manually labeled training image.

[0185] If the difference is greater than the threshold, repeat step 2, that is, readjust the parameters of the attention mechanism until the difference between the quality score predicted by the attention mechanism using the adjusted parameters and the manually labeled image quality label is small, and the training ends.

[0186] It should be noted that there are various algorithms for attention mechanisms. In addition to those mentioned above, there are other algorithms, such as non-local blocks, and other algorithms can also be used to determine the weights corresponding to the image problem.

[0187] In this embodiment, the image quality assessment device can automatically identify image problems and automatically identify the weights corresponding to the image problems, eliminating the need for manual identification of image problems and manual assignment of weights, thus achieving high efficiency.

[0188] S500, the second branch sends the problem type assessment results to the fusion module.

[0189] S600, the fusion module determines the first quality level (i.e. the final quality assessment result) based on the comprehensive assessment results and the problem type assessment results.

[0190] As mentioned above, the comprehensive evaluation result is used to indicate the overall quality level of the first image. Therefore, S600 can be further refined as follows: the fusion module generates a second quality level based on the problem type evaluation result, and determines the first quality level based on the comprehensive quality level and the second quality level.

[0191] There are several ways to generate a second quality level based on the evaluation results of the problem type.

[0192] Method A: The problem type evaluation result includes image problem confidence. The number of image problems with an image problem confidence greater than a first threshold is determined, and a second quality level is generated based on the number of such problems.

[0193] For example, if there are M image problems in the first image, and corresponding to M image problem confidence levels, the number of image problems with confidence levels greater than a first threshold is determined. Based on a pre-stored mapping relationship between the number of image problems and quality levels, a second quality level is determined. For instance, the higher the number of image problems, the lower the second quality level.

[0194] Method B involves evaluating the problem type results, including the confidence level of the image problem, determining the average or weighted average of the confidence levels of multiple image problems, and generating a second quality level based on the average or weighted average.

[0195] For example, if there are M image problems in the first image, and M image problem confidence levels, determine the average or weighted average of the M image problem confidence levels, and determine the second quality level corresponding to the average or weighted average based on the pre-stored mapping relationship between confidence levels and quality levels.

[0196] Method C: The problem type evaluation results include the image problem quality level and weight, and a second quality level is generated based on the image problem quality level and weight.

[0197] For example, if there are M image problems in the first image, corresponding to M image problem confidence levels, then the second quality level satisfies the following formula:

[0198]

[0199] Among them, S i Let be the quality level of the i-th image problem existing in the first image, wi be the weight corresponding to the i-th image problem, and S be the second quality level.

[0200] After obtaining the second quality level, the first quality level is determined based on the second quality level and the overall quality level (i.e., the overall evaluation result), including but not limited to the following methods.

[0201] Method 1: The first quality level is the intermediate level between the overall quality level and the second quality level. For example, if the overall quality level is the second level and the second quality level is the fourth level, then the first quality level is determined to be the intermediate level between the second and fourth levels, namely the third level.

[0202] Method 2 involves obtaining the first confidence level corresponding to the overall quality level and the second confidence level corresponding to the second quality level, and determining the first quality level as the one with the higher confidence level between the overall quality level and the second quality level. The first confidence level is obtained by including the overall quality level in the comprehensive evaluation result calculated by the first branch, and may also include the confidence level of the overall quality level, which is the first confidence level. The second confidence level is obtained by generating the second quality level based on the average or weighted average of the confidence levels of the image problems, as described above in Method B, where the average or weighted average is the second confidence level.

[0203] Method 3: Obtain the first confidence level corresponding to the overall quality level, the second confidence level corresponding to the second quality level, calculate the third confidence level based on the first and second confidence levels, and determine the first quality level based on the third confidence level. For example, the third confidence level is the average or weighted average of the first and second confidence levels, and then the first quality level corresponding to the average or weighted average is determined according to the mapping relationship between confidence levels and quality levels.

[0204] Method 4: Determine the difference between the overall quality level and the second quality level; use the difference to verify (or correct) the overall quality level or the second quality level to obtain the first quality level. The first quality level satisfies α ± (η) i ×Δ), η i This is a preset value, where Δ = |α - α′|, α is the overall quality level, and α′ is the second quality level. This method obtains the first quality level by correcting the overall quality result through the difference, i.e., Δ. Alternatively, the first quality level satisfies α′ ± (η). i ×Δ), η i The value is a preset value, where Δ = |α - α′|, α is the overall quality level, and α′ is the second quality assessment result. This method obtains the first quality level by correcting the second quality result based on the difference, i.e., Δ.

[0205] It should be noted that the image quality assessment device may randomly select one of the above methods 1 to 4, or the user may specify a method; this application embodiment does not limit the method.

[0206] It should be noted that in the above embodiments, the attention mechanism is trained separately and is optional. Figure 4B The model shown can be trained as a whole. For example, see... Figure 4BThe training images are input into a preprocessing module, which processes them before inputting them into an image problem evaluation module and an overall image evaluation module. The overall image evaluation module evaluates the training images to obtain a quality level (or quality score). The image problem evaluation module identifies image problems in the training images and inputs both the image problems and the training images into an attention mechanism. The attention mechanism automatically determines the weights corresponding to the image problems and then uses these weights and the image problems to determine another quality level (or quality score). The fusion module uses the two quality levels (or two quality scores) to determine the final quality level (or final score). If the final quality level differs significantly from the quality level of the manually labeled training images, the model parameters are adjusted—for example, the parameters in the image problem evaluation module, the overall image evaluation module, or the attention mechanism—until the final quality level obtained using the adjusted parameters is significantly different from the manually labeled quality level, at which point training stops.

[0207] In summary, the image evaluation method provided in this application can obtain a comprehensive evaluation result of an image, as well as an evaluation result of the types of image problems, indicating the types of image problems present. Then, based on the problem type evaluation results and the comprehensive evaluation, the final evaluation result of the image is determined. The final evaluation result is not simply equal to the comprehensive evaluation result. This approach provides better evaluation results and eliminates the need for manual identification of image problems, thus increasing efficiency.

[0208] The apparatus used to implement the above method in the embodiments of this application is described below with reference to the accompanying drawings. Therefore, the content above can be used in subsequent embodiments, and repeated content will not be described again.

[0209] Figure 8 This is a structural block diagram of an image quality assessment device 800 provided in an embodiment of this application. Exemplarily, the image quality assessment device 800 may be a server (such as a map server), a terminal device, a vehicle or in-vehicle device, a roadside unit, etc. The image quality assessment device 800 includes an image acquisition unit 801 and an image quality assessment unit 802. The image acquisition unit 801 is used to acquire a first image. The image quality assessment unit 802 is used to perform a quality assessment on the first image to obtain a comprehensive assessment result and a problem type assessment result; wherein the comprehensive assessment result indicates the overall quality level of the first image, and the problem type assessment result indicates the type of problem present in the first image, the problem type including at least one of overexposure, underexposure, blur, color cast, noise, glare, reflection, or tree shadow; the image quality assessment unit 802 is also used to determine a first quality level of the first image based on the comprehensive assessment result and the problem type assessment result.

[0210] For example, the image quality assessment device 800 can be an image quality assessment device (e.g., a server, terminal device, vehicle or vehicle-mounted device, roadside unit, etc.), a chip or chip system applied in the image quality assessment device, or other combined devices or components with image quality assessment functions in the image quality assessment device. When the image quality assessment device 800 is a component with image quality assessment functions, the image acquisition unit 801 and the image quality assessment unit 802 can be processors, such as baseband processors, which may include one or more central processing units (CPUs). When the image quality assessment device 800 is a chip system, the image acquisition unit 1001 and the image quality assessment unit 802 can be processors in the chip system, which may include one or more central processing units.

[0211] Optionally, the image quality assessment device 800 may also include a communication unit. The communication unit may include a receiving unit and a transmitting unit. The transmitting unit may be a functional module for performing a transmitting operation; the receiving unit may be a functional module for performing a receiving operation.

[0212] The image acquisition unit 801 can be used to perform... Figure 5 The image acquisition module performs the operations shown in the illustrated embodiment. The image quality evaluation unit 802 can be used to perform... Figure 5 The embodiments shown include all operations of the first branch, the second branch, and the fusion module, and / or other processes used to support the techniques described herein.

[0213] In one possible design, the problem type evaluation result includes at least one of problem confidence, problem quality level, or image weight; wherein the problem confidence is used to indicate the probability that the image exists on the first image, the problem quality level is used to indicate the severity of the image, and the weight is used to indicate the proportion of the problem on the first image.

[0214] In one possible design, when the image quality assessment unit determines a first quality level of the first image based on the comprehensive assessment result and the problem type assessment result, it is specifically used to: determine the number of problems with a problem confidence level greater than a first threshold; generate a second quality level of the first image based on the number; and determine the first quality level based on the second quality level and the comprehensive quality level.

[0215] In one possible design, the image quality assessment unit, when determining the first quality level of the first image based on the comprehensive assessment result and the problem type assessment result, specifically performs the following:

[0216] The second quality level of the first image is calculated based on the average or weighted average of the question confidence levels of multiple questions; the first quality level is determined based on the second quality level and the overall quality level.

[0217] In one possible design, when the image quality assessment unit determines the first quality level of the first image based on the comprehensive assessment result and the problem type assessment result, it is specifically used to: calculate the second quality level of the first image based on the problem quality levels and weights of multiple problems; and determine the first quality level based on the second quality level and the comprehensive quality level.

[0218] In one possible design, the second quality level is an intermediate level between the first quality level and the combined quality level.

[0219] In one possible design, when the image quality assessment unit determines the first quality level based on the second quality level and the overall quality level, it is specifically configured to: determine the difference between the overall quality level and the second quality level; and use the difference to correct the second quality level or the overall quality level to obtain the first quality level.

[0220] In one possible design, the overall evaluation result is related to the user's level of liking for the first image.

[0221] In one possible design, the problem type is related to at least one of the following: the type of object captured in the first image, environmental information when the first image was acquired, geographical location when the first image was acquired, time information when the first image was acquired, or user-specified operation.

[0222] In one possible design, the image quality assessment unit is further configured to: determine the weight corresponding to the problem based on at least one of the problem quality level, the area occupied by the problem in the first image, or the location of the problem in the first image.

[0223] In one possible design, the image quality assessment unit is specifically used for: acquiring training images and corresponding real assessment results; using an algorithm model to predict the quality of the training images to obtain predicted assessment results; wherein, the algorithm model includes a preprocessing module, a problem assessment module, an attention mechanism module, a comprehensive assessment module, and a fusion module; the preprocessing module is used to perform at least one preprocessing procedure on the training images, such as image feature extraction, image size adjustment, or multidimensional array adjustment; the problem assessment module is used to identify the types of problems existing in the training images, and the attention mechanism module is used to determine the weights corresponding to the problems; the comprehensive assessment module is used to determine the comprehensive quality level of the training images, and the fusion module is used to fuse the comprehensive quality level and a third quality level of the training images to obtain the predicted assessment results, wherein the third quality level is calculated based on the weights; when the difference between the predicted assessment results and the real assessment results is greater than a threshold, the parameters of the algorithm model are adjusted until the difference between the predicted assessment results obtained by predicting the quality of the training images again using the adjusted parameters and the real assessment results is less than the threshold, thus ending the training; the trained algorithm model is used to perform quality assessment on the first image to obtain a comprehensive assessment result and a problem type assessment result.

[0224] The unit division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into a single processor, exist as separate physical units, or be integrated into a single unit. The integrated units described above can be implemented in hardware or as software functional units.

[0225] Figure 8 One or more of the various units within can be implemented using software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various integrated circuits, such as a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).

[0226] Figure 9 This is a structural block diagram of the image quality assessment device 900 provided in an embodiment of this application. Figure 9The image quality assessment device 900 shown includes at least one processor 901. The image quality assessment device 900 also includes at least one memory 902 for storing program instructions and / or data. The memory 902 and the processor 901 are coupled. The coupling in this embodiment is an indirect coupling or communication connection between devices, units, or modules, which can be electrical, mechanical, or other forms, used for information interaction between devices, units, or modules. The processor 901 can operate collaboratively with the memory 902, and the processor 901 can execute the program instructions stored in the memory 902. At least one of the at least one memory 902 can be included in the processor 901. The image quality assessment device 900 may also include an interface circuit (not shown in the figure), through which the processor 901 is coupled to the memory 902. The processor 901 can execute the program code in the memory 902 to implement the image quality assessment method provided in this embodiment.

[0227] Optionally, the image quality assessment device 900 may also include an image acquisition module (not shown in the figure), such as a camera, and the number of cameras may be one or more, for acquiring the first image.

[0228] The image quality assessment device 900 may further include a communication interface 903 for communicating with other devices via a transmission medium, thereby enabling the image quality assessment device 900 to communicate with other devices, such as receiving a first image from other devices. In this embodiment, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface. In this embodiment, when the communication interface is a transceiver, the transceiver may include an independent receiver, an independent transmitter, or a transceiver integrating transceiver functions, or an interface circuit, etc.

[0229] It should be understood that the connection medium between the processor 901, memory 902, and communication interface 903 described above is not limited in the embodiments of this application. The embodiments of this application... Figure 9 The memory 902, processor 901, and communication interface 903 are connected via a communication bus 904. Figure 9 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus may include an address bus, data bus, control bus, etc. For ease of illustration, Figure 9 The symbol is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0230] In one example, when the image quality assessment device 900 is used to achieve the above... Figure 5In the illustrated process, when the image quality assessment device performs certain steps, the communication interface 903 is used to perform the relevant operations for sending and receiving data in the above embodiment, and the processor 901 is used to perform the relevant processing operations of the image quality assessment device in the above method embodiment (e.g., ...). Figure 5 (The relevant steps of the first branch, the second branch, and the fusion module).

[0231] For example, processor 901 is used to perform a quality assessment on the first image to obtain a comprehensive assessment result and a problem type assessment result; wherein, the comprehensive assessment result is used to indicate the overall quality level of the first image, and the problem type assessment result is used to indicate the type of problem present in the first image, the problem type including at least one of overexposure, underexposure, blur, color cast, noise, glare, reflection, or tree shadow; and a first quality level of the first image is determined based on the comprehensive assessment result and the problem type assessment result.

[0232] This application provides a computer-readable storage medium including computer instructions that, when executed by a processor, cause the lane line attribute detection device to perform the lane line attribute detection method described in the embodiments of this application.

[0233] This application provides a computer program product, which includes a computer program that, when run on a processor, causes the lane line attribute detection device to execute the lane line attribute detection method described in the embodiments of this application.

[0234] In the embodiments of this application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0235] In the embodiments of this application, the memory can be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as random-access memory (RAM). Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in the embodiments of this application can also be a circuit or any other device capable of implementing storage functions, used to store program instructions and / or data.

[0236] The methods provided in this application can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, magnetic tape), optical media (e.g., digital video disc (DVD), or semiconductor media (e.g., SSD), etc.

[0237] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0238] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0239] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0240] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0241] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0242] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0243] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways without exceeding the scope of this application. For example, the embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0244] Furthermore, the described apparatus and methods, as well as the schematic diagrams of different embodiments, can be combined or integrated with other systems, modules, technologies, or methods without departing from the scope of this application. Additionally, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through interfaces, devices, or units, and may be electronic, mechanical, or other forms.

[0245] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image quality assessment method, characterized in that, include: Get the first image; Based on the first information, the problem type is determined. The first information includes at least one of the following: the type of object in the first image, environmental information when the first image was acquired, geographical location when the first image was acquired, time information when the first image was acquired, and user-specified operation. The problem type includes at least one of overexposure, underexposure, blur, color cast, noise, glare, reflection, or tree shadow. Based on the user's preference for the first image, a quality assessment is performed on the first image to obtain a comprehensive assessment result, which is used to indicate the overall quality level of the first image. Based on the determined problem type, the first image is subjected to a quality assessment to obtain a problem type assessment result, which is used to determine the second quality level of the first image; A first quality level is determined for the first image based on the overall quality level and the second quality level.

2. The method according to claim 1, characterized in that, The problem type evaluation results include at least one of the following: problem confidence level, problem quality level, or problem weight; Wherein, the problem confidence level is used to indicate the probability that the problem type exists on the first image, the problem quality level is used to indicate the severity of the problem type, and the weight is used to indicate the proportion of the problem type on the first image.

3. The method according to claim 2, characterized in that, The method further includes: Determine the number of questions whose confidence level is greater than the first threshold; Based on the quantity, a second quality level is generated for the first image.

4. The method according to claim 2, characterized in that, The method further includes: The second quality level of the first image is calculated based on the average or weighted average of the question confidence levels of multiple questions.

5. The method according to claim 2, characterized in that, The method further includes: Calculate the second quality level of the first image based on the problem quality level and weight.

6. The method according to claim 1, characterized in that, The first quality level is an intermediate level between the overall quality level and the second quality level.

7. The method according to claim 1, characterized in that, Determining the first quality level based on the comprehensive quality level and the second quality level includes: Determine the difference between the overall quality level and the second quality level; The difference is used to correct the overall quality level or the second quality level to obtain the first quality level.

8. The method according to claim 2, characterized in that, The method further includes: The weight is determined based on at least one of the following: the quality level of the problem, the area occupied by the problem on the first image, or the location of the problem on the first image.

9. The method according to any one of claims 1-8, characterized in that, The quality assessment of the first image, yielding a comprehensive assessment result and a problem type assessment result, includes: Obtain the training images and the corresponding real evaluation results; The quality of the training images is predicted using an algorithm model to obtain a prediction evaluation result. The algorithm model includes a preprocessing module, a problem evaluation module, an attention mechanism module, a comprehensive evaluation module, and a fusion module. The preprocessing module performs at least one preprocessing operation on the training images, including image feature extraction, image resizing, or multidimensional array adjustment. The problem evaluation module identifies the types of problems present in the training images, and the attention mechanism module determines the weights corresponding to the problems. The comprehensive evaluation module determines the overall quality level of the training images, and the fusion module fuses the overall quality level and a third quality level of the training images to obtain the prediction evaluation result. The third quality level is calculated based on the weights. When the difference between the predicted evaluation result and the true evaluation result is greater than a threshold, the parameters of the algorithm model are adjusted until the difference between the predicted evaluation result obtained by predicting the quality of the training image again using the adjusted parameters and the true evaluation result is less than the threshold, thus ending the training. The trained algorithm model is used to evaluate the quality of the first image, resulting in a comprehensive evaluation result and a problem type evaluation result.

10. An image quality assessment device, characterized in that, include: Image acquisition unit, used to acquire the first image; An image quality assessment unit is used to determine the problem type based on first information, the first information including at least one of the following: the type of object in the first image, environmental information when the first image was acquired, geographical location when the first image was acquired, time information when the first image was acquired, and user-specified operation. The problem type includes at least one of overexposure, underexposure, blur, color cast, noise, glare, reflection, or tree shadow. The image quality assessment unit is further configured to assess the quality of the first image based on the user's preference for the first image, and obtain a comprehensive assessment result, which is used to indicate the comprehensive quality level of the first image. The image quality assessment unit is further configured to assess the quality of the first image according to the determined problem type, and obtain a problem type assessment result, wherein the problem type assessment result is used to determine a second quality level of the first image; and to determine a first quality level of the first image based on the comprehensive quality level and the second quality level.

11. The apparatus according to claim 10, characterized in that, The problem type evaluation result includes at least one of the following: problem confidence level, problem quality level, or image weight; Wherein, the problem confidence level is used to indicate the probability that the problem type exists on the first image, the problem quality level is used to indicate the severity of the problem type, and the weight is used to indicate the proportion of the problem type on the first image.

12. The apparatus according to claim 11, characterized in that, The image quality assessment unit is specifically used for: Determine the number of questions whose confidence level is greater than the first threshold; Based on the quantity, a second quality level is generated for the first image.

13. The apparatus according to claim 11, characterized in that, The image quality assessment unit is specifically used for: The second quality level of the first image is calculated based on the average or weighted average of the question confidence levels of multiple questions.

14. The apparatus according to claim 11, characterized in that, The image quality assessment unit is specifically used for: Calculate the second quality level of the first image based on the problem quality level and weight.

15. The apparatus according to claim 10, characterized in that, The first quality level is an intermediate level between the second quality level and the overall quality level.

16. The apparatus according to claim 10, characterized in that, The image quality assessment unit is specifically used for: Determine the difference between the overall quality level and the second quality level; The difference is used to correct the second quality level or the overall quality level to obtain the first quality level.

17. The apparatus according to claim 11, characterized in that, The image quality assessment unit is also used for: The weight is determined based on at least one of the following: the quality level of the problem, the area occupied by the problem on the first image, or the location of the problem on the first image.

18. The apparatus according to any one of claims 10-17, characterized in that, The image quality assessment unit is specifically used for: Obtain the training images and the corresponding real evaluation results; The quality of the training images is predicted using an algorithm model to obtain a prediction evaluation result. The algorithm model includes a preprocessing module, a problem evaluation module, an attention mechanism module, a comprehensive evaluation module, and a fusion module. The preprocessing module performs at least one preprocessing operation on the training images, including image feature extraction, image resizing, or multidimensional array adjustment. The problem evaluation module identifies the types of problems present in the training images, and the attention mechanism module determines the weights corresponding to the problems. The comprehensive evaluation module determines the overall quality level of the training images, and the fusion module fuses the overall quality level and a third quality level of the training images to obtain the prediction evaluation result. The third quality level is calculated based on the weights. When the difference between the predicted evaluation result and the true evaluation result is greater than a threshold, the parameters of the algorithm model are adjusted until the difference between the predicted evaluation result obtained by predicting the quality of the training image again using the adjusted parameters and the true evaluation result is less than the threshold, thus ending the training. The trained algorithm model is used to evaluate the quality of the first image, resulting in a comprehensive evaluation result and a problem type evaluation result.

19. An image quality assessment device, characterized in that, include: Processor and interface circuitry; The processor is coupled to the memory via the interface circuit, and the processor is used to execute program code in the memory to implement the method as described in any one of claims 1-9.

20. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-9.

21. A computer program product, characterized in that, The computer program product includes a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 9.

Images