An image quality evaluation method, device, equipment, medium and product

Abstract

The application provides an image quality evaluation method, device, equipment, medium and product, which comprises the following steps: obtaining a to-be-evaluated image and inputting a pre-trained segmentation model to obtain an initial segmentation result; performing dynamic mask optimization on the initial result to form an optimized segmentation mask; then performing semantic guidance to output the final segmentation mask and pixel confidence corresponding to each object. For each object, the mask and the confidence are combined to calculate the dimension score in multiple evaluation dimensions and are fused to obtain the object quality score; the object attribute is extracted from the mask to determine the weight, and the scores of the objects are weighted and aggregated to obtain the overall image quality score; finally, the image quality evaluation result is generated and output according to the overall image score and the object score. Thus, the accuracy, the degree of refinement and the semantic interpretability of the image quality evaluation are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image quality evaluation method, apparatus, device, medium, and product. Background Technology

[0002] With the widespread application of digital images in fields such as surveillance, autonomous driving, and content generation, how to automatically and accurately assess image quality has become a key issue in computer vision. Image Quality Assessment (IQA) aims to use algorithms to simulate human visual perception and quantify image quality, thereby providing a basis for system optimization, content selection, and other tasks.

[0003] Traditional image quality assessment methods are mainly divided into two categories: subjective evaluation relying on human scoring, which yields results consistent with human perception but is costly and inefficient; and objective evaluation calculated through mathematical models, including full-reference, semi-reference, and no-reference methods. Full-reference methods require the complete original image for comparison, such as the commonly used Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). Semi-reference methods require only partial original information; while no-reference methods do not require the original image at all, directly assessing the quality of distorted images, making them more suitable for real-world scenarios. In recent years, deep learning-based no-reference methods have gradually become mainstream, capable of automatically learning features from images and predicting quality scores, but they still face significant challenges under complex real-world conditions.

[0004] In summary, existing methods have significant shortcomings: subjective evaluations are greatly influenced by human factors and are difficult to scale; traditional objective methods often do not align with human perception, especially when dealing with mixed distortions (such as the simultaneous presence of blur, noise, and compression); and unreferenced deep learning models are easily limited by training data, have weak generalization ability in cross-device and cross-scene applications, and fail to fully incorporate the semantic information of key objects in the image (such as whether faces are clear and whether text is legible), resulting in evaluation results that often deviate from human visual experience in complex real-world scenarios.

[0005] Overall, existing image quality assessment methods suffer from technical problems such as over-reliance on overall statistical features and neglect of local details and semantic information, leading to discrepancies between the assessment results and actual human perception, resulting in inaccurate assessment results. Summary of the Invention

[0006] This application provides an image quality evaluation method, apparatus, device, medium, and product to address the technical problem that existing image quality evaluation methods rely too heavily on overall statistical features and ignore local details and semantic information, resulting in evaluation results that deviate from actual human perception and are inaccurate.

[0007] To solve the above-mentioned technical problems, this application is implemented as follows:

[0008] In a first aspect, embodiments of this application provide an image quality evaluation method, the method comprising:

[0009] The image to be evaluated is acquired, and the image to be evaluated is input into a pre-trained segmentation model for initial segmentation processing to obtain the initial segmentation result output by the segmentation model.

[0010] The initial segmentation result is subjected to dynamic mask optimization processing to obtain the optimized segmentation mask;

[0011] The optimized segmentation mask is subjected to semantic guidance processing to obtain the final segmentation mask corresponding to each object in the image to be evaluated, and the confidence information of each pixel in the final segmentation mask;

[0012] For each object, based on the final segmentation mask of the object and the confidence information of each pixel in the final segmentation mask, the scores of the object in each of the preset multiple evaluation dimensions are calculated respectively, and the scores of the object in each dimension are fused to obtain the object quality score.

[0013] For each object, the attribute information of the object is determined according to the final segmentation mask of the object, the weight information of the object is determined according to the attribute information, and the object quality scores of each object are weighted and aggregated according to the weight information of each object to obtain the overall image quality score of the image to be evaluated.

[0014] The quality evaluation result of the image to be evaluated is generated and output based on the overall image quality score and the object quality scores of each object.

[0015] Optionally, the image to be evaluated is input into a pre-trained segmentation model for initial segmentation processing to obtain an initial segmentation result output by the segmentation model; the initial segmentation result is then subjected to dynamic mask optimization processing to obtain an optimized segmentation mask, including:

[0016] The image to be evaluated is input into the segmentation model to obtain the initial mask of each object in the image to be evaluated and the initial confidence of each pixel in the initial mask. The initial mask and the initial confidence are used as the initial segmentation result.

[0017] The image quality of the image to be evaluated is pre-evaluated, the image quality level of the image to be evaluated is determined based on the result of the pre-evaluation, and a corresponding threshold is determined based on the image quality level, wherein the lower the image quality level, the lower the threshold.

[0018] Perform iterative optimization processing, wherein the iterative optimization processing includes:

[0019] For each pixel in the initial mask, the region containing pixels whose initial confidence is less than the threshold is defined as a low-confidence region.

[0020] Extract the edge pixels of the low confidence region, generate local cue information based on the edge pixels, input the local cue information into the segmentation model, and obtain the thinning mask of the low confidence region and the thinning confidence of each pixel in the thinning mask output by the segmentation model;

[0021] The fusion weights are determined based on the initial confidence level and the refined confidence level. The initial mask and the refined mask are then weighted and fused according to the fusion weights to obtain the fused mask.

[0022] Calculate the intersection-union ratio (IUGR) between the fused mask obtained in the current iteration and the fused mask obtained in the previous iteration, and determine whether the IUGR reaches a preset convergence threshold. If yes, stop the iteration and use the current fused mask as the optimized segmentation mask; otherwise, use the current fused mask as the initial mask for the next iteration, use the confidence score of each pixel in the current fused mask as the initial confidence score for the next iteration, and return to execute the iterative optimization process.

[0023] Optionally, semantic guidance processing is performed on the optimized segmentation mask to obtain the final segmentation mask corresponding to each object in the image to be evaluated, and the confidence information of each pixel in the final segmentation mask, including:

[0024] Feature extraction is performed on the optimized segmentation mask to obtain mask features;

[0025] Obtain a predefined semantic template library, wherein the semantic template library includes multiple object categories, feature templates corresponding to each object category, and semantic priorities corresponding to each object category;

[0026] For each object category, calculate the cosine similarity between the mask feature and the feature template corresponding to the object category; compare the cosine similarities corresponding to each object category, and determine the object category with the largest cosine similarity as the semantic label of the object;

[0027] The semantic priority is obtained from the semantic template library according to the semantic tag, and the attention weight of the object is determined according to the semantic priority, wherein the higher the semantic priority, the greater the attention weight;

[0028] The optimized segmentation mask is associated with and stored with the semantic label and the attention weight to obtain the final segmentation mask of the object, and the confidence of each pixel in the optimized segmentation mask is used as the confidence information of each pixel in the final segmentation mask.

[0029] Optionally, for each object, the attribute information of the object is determined based on the final segmentation mask of the object, and the weight information of the object is determined based on the attribute information, including:

[0030] For each of the objects, calculate the mask area of ​​the final segmentation mask of the object, calculate the total image area of ​​the image to be evaluated, calculate the ratio of the mask area to the total image area, and use the ratio as the area factor;

[0031] Calculate the center position of the final segmentation mask of the object, calculate the center position of the image to be evaluated, calculate the distance between the center position of the mask and the center position of the image, calculate the diagonal length of the image to be evaluated, and determine the position factor based on the ratio of the distance to the diagonal length, wherein the smaller the ratio of the distance to the diagonal length, the larger the position factor;

[0032] The final segmentation mask of the object is input into a pre-trained saliency model to obtain the saliency factor of the object output by the saliency model.

[0033] Extract the edge region of the final segmentation mask of the object, calculate the gradient value of each pixel in the edge region, calculate the mean of each gradient value, and use the mean as the boundary sharpness factor;

[0034] The area factor, the position factor, the saliency factor, and the boundary clarity factor are multiplied together to obtain the fusion weight of the object;

[0035] Calculate the sum of the fusion weights of each object, calculate the ratio of the fusion weight of each object to the sum of the fusion weights, and use the ratio as the weight information of the corresponding object.

[0036] Optionally, based on the final segmentation mask of the object and the confidence information of each pixel in the final segmentation mask, the scores of the object in each of the preset multiple evaluation dimensions are calculated, including:

[0037] The image to be evaluated and the final segmentation mask of each object in the image to be evaluated are input into a pre-trained cross-domain transfer model to obtain the scores of each object in each of the preset multiple evaluation dimensions output by the cross-domain transfer model.

[0038] Optionally, the pre-trained cross-domain transfer model is trained in the following manner:

[0039] Acquire source domain data and target domain data, wherein the source domain data includes high-quality images and corresponding subjective ratings, and the target domain data includes low-quality scene images and corresponding subjective ratings;

[0040] Scene type identification is performed on each image in the target domain data to obtain the scene type of each image; wherein, the scene type includes at least one of the following: low light scene, rain and fog scene, and motion blur scene;

[0041] The corresponding image enhancement method is determined according to the scene type of each image, and the image is enhanced according to the image enhancement method to obtain source domain stylized target domain data; wherein, the dark light scene corresponds to the illumination enhancement method, the rain and fog scene corresponds to the defogging and deraining method, and the motion blur scene corresponds to the deblurring method.

[0042] Training a preset quality knowledge encoder and a preset quality regression head based on the source domain data includes: inputting high-quality images from the source domain data into the quality knowledge encoder to obtain high-quality features output by the quality knowledge encoder; inputting the high-quality features into the quality regression head to obtain a predicted score output by the quality regression head; calculating a quality prediction loss based on the difference between the predicted score and the subjective score; and optimizing the parameters of the quality knowledge encoder and the quality regression head based on the quality prediction loss.

[0043] The knowledge from the trained quality knowledge encoder and the quality regression head is distilled into a lightweight network to obtain a lightweight quality encoder;

[0044] A domain adaptive migration architecture is constructed, wherein the domain adaptive migration architecture includes a shared feature extractor, a domain classifier, and a quality predictor; the shared feature extractor is initialized based on the lightweight quality encoder;

[0045] The source domain data is input into the shared feature extractor to obtain the source domain features output by the shared feature extractor;

[0046] The source domain stylized target domain data is input into the shared feature extractor to obtain the target domain features output by the shared feature extractor;

[0047] The source domain features and the target domain features are respectively input into the domain classifier to obtain the domain classification result output by the domain classifier;

[0048] Calculate the domain adversarial loss based on the domain classification result, and optimize the parameters of the shared feature extractor and the quality predictor based on the quality prediction loss and the domain adversarial loss to reduce the difference between the source domain features and the target domain features;

[0049] The optimized shared feature extractor and the quality predictor are used as the cross-domain migration model.

[0050] Optionally, after obtaining the object's mass score, the method further includes:

[0051] The image to be evaluated is input into a pre-trained scene classifier to obtain the scene label of the image to be evaluated output by the scene classifier;

[0052] Obtain a preset scene correction parameter table, wherein the scene correction parameter table includes the correspondence between each scene label, correction coefficient, and scene-specific offset;

[0053] Based on the scene label of the image to be evaluated, the corresponding target correction coefficient and the corresponding target scene-specific offset are retrieved from the scene correction parameter table;

[0054] The final object quality score is determined based on the corrected object quality score, the target correction coefficient, and the target scene-specific offset.

[0055] Optionally, determining the object's attribute information based on the final segmentation mask, and determining the object's weight information based on the attribute information, includes:

[0056] Obtain the final segmentation mask for each of the objects, and calculate the area of ​​each final segmentation mask;

[0057] The sum of the areas of each final segmentation mask is calculated to obtain the total mask area. The ratio of the area of ​​each final segmentation mask to the total mask area is then calculated, and the ratio is used as the weight information of each object.

[0058] Optionally, the quality evaluation result of the image to be evaluated is generated and output based on the overall image quality score and the object quality scores of each object, including:

[0059] Obtain a preset grid size, and divide the image to be evaluated into multiple grid units according to the preset grid size;

[0060] For each of the grid cells, identify the objects whose final segmentation mask overlaps with the grid cell, obtain the object quality score of each identified object, calculate the average of the object quality scores, and use the average as the cell quality score of the grid cell.

[0061] Obtain a preset color mapping rule, map the unit quality score of each grid cell to the corresponding display color according to the color mapping rule, and generate a quality heatmap based on each grid cell and the corresponding display color.

[0062] Obtain a preset scoring threshold. For each object, compare the object quality score with the preset scoring threshold. If the object quality score is lower than the preset scoring threshold, then the object is identified as a low-quality object.

[0063] For each low-quality object, obtain the scores of each dimension of the low-quality object, compare the scores of each dimension, determine the dimension with the lowest score as the target dimension, and generate corresponding attribution prompt information based on the target dimension;

[0064] Output the overall image quality score, the quality heatmap, and the attribution information.

[0065] Optionally, the method further includes:

[0066] In response to the user's region selection operation, the target region is determined based on the region selection operation;

[0067] For each object, calculate the intersection of the final segmentation mask of the object and the target region; determine whether the intersection is empty; if the intersection is not empty, then identify the object as the target object.

[0068] For each target object, calculate the intersection area of ​​the intersection of the target objects; calculate the sum of the intersection areas of all the target objects to obtain the total intersection area; calculate the ratio of the intersection area of ​​all the target objects to the total intersection area, and use the ratio as the target region weight of each target object.

[0069] The region quality score of the target region is determined based on the target region weight of each target object and the corresponding object quality score;

[0070] Generate a quality heatmap of the target area based on the regional quality score, and output the quality heatmap.

[0071] Optionally, the plurality of evaluation dimensions include at least one of the following: segmentation integrity dimension, feature stability dimension, edge accuracy dimension, occlusion robustness dimension, and quality consistency dimension.

[0072] Secondly, embodiments of this application provide an image quality evaluation device, the device comprising:

[0073] The acquisition module is used to acquire the image to be evaluated, input the image to be evaluated into a pre-trained segmentation model for initial segmentation processing, and obtain the initial segmentation result output by the segmentation model.

[0074] The execution module is used to perform dynamic mask optimization processing on the initial segmentation result to obtain the optimized segmentation mask;

[0075] The optimized segmentation mask is subjected to semantic guidance processing to obtain the final segmentation mask corresponding to each object in the image to be evaluated, and the confidence information of each pixel in the final segmentation mask;

[0076] For each object, based on the final segmentation mask of the object and the confidence information of each pixel in the final segmentation mask, the scores of the object in each of the preset multiple evaluation dimensions are calculated respectively, and the scores of the object in each dimension are fused to obtain the object quality score.

[0077] For each object, the attribute information of the object is determined according to the final segmentation mask of the object, the weight information of the object is determined according to the attribute information, and the object quality scores of each object are weighted and aggregated according to the weight information of each object to obtain the overall image quality score of the image to be evaluated.

[0078] The quality evaluation result of the image to be evaluated is generated and output based on the overall image quality score and the object quality scores of each object.

[0079] Thirdly, embodiments of this application provide a network device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of an image quality evaluation method as described in the first aspect.

[0080] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of an image quality assessment method as described in the first aspect.

[0081] Fifthly, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of an image quality evaluation method as described in the first aspect.

[0082] In this embodiment, the initial segmentation result is obtained by inputting the image to be evaluated into a pre-trained segmentation model, and dynamic mask optimization is performed on it, which significantly improves the contour accuracy and robustness of the object mask in scenes with blurred edges, complex backgrounds, and partial occlusion. Then, based on semantic guidance processing, each object obtains a final segmentation mask with clear semantic information and its pixel-level confidence, thereby establishing a structured representation from pixels to semantic objects, providing a fine-grained analytical basis for quality evaluation. On this basis, for each object, scores are calculated and fused on multiple preset dimensions according to its final segmentation mask and internal confidence information, realizing a multi-angle and comprehensive quantification of the visual quality of objects, overcoming the problem of incomplete evaluation by a single indicator. Furthermore, weights are determined according to the attribute information of each object and weighted aggregation is performed, so that the overall image quality score can simultaneously reflect the overall quality level and the importance differences of different semantic regions, enhancing the consistency between the evaluation results and human subjective perception. The final output is the quality evaluation result.

[0083] This achieves a leap from whole-image statistics to semantic object-level evaluation, improving the accuracy, refinement, and semantic interpretability of image quality assessment. Attached Figure Description

[0084] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0085] Figure 1 A flowchart illustrating an image quality assessment method provided in this application embodiment;

[0086] Figure 2 This is a structural block diagram of an image quality assessment method provided in an embodiment of this application;

[0087] Figure 3 A schematic diagram illustrating the object-level quality score and the overall quality score of the entire image provided in the embodiments of this application;

[0088] Figure 4 A structural block diagram of an image quality evaluation device provided in an embodiment of this application;

[0089] Figure 5 This is a structural block diagram of a network device provided in an embodiment of this application. Detailed Implementation

[0090] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0091] Figure 1 An image quality assessment method according to an embodiment of this application is shown, such as... Figure 1 As shown, the method includes:

[0092] Step S101: Obtain the image to be evaluated, input the image to be evaluated into the pre-trained segmentation model for initial segmentation processing, and obtain the initial segmentation result output by the segmentation model;

[0093] Step S102: Perform dynamic mask optimization on the initial segmentation result to obtain the optimized segmentation mask;

[0094] Step S103: Perform semantic guidance processing on the optimized segmentation mask to obtain the final segmentation mask corresponding to each object in the image to be evaluated, and the confidence information of each pixel in the final segmentation mask.

[0095] Step S104: For each object, based on the final segmentation mask of the object and the confidence information of each pixel in the final segmentation mask, calculate the score of each dimension of the object under multiple preset evaluation dimensions, and fuse the scores of each dimension of the object to obtain the object quality score.

[0096] Step S105: For each object, determine the object's attribute information based on the object's final segmentation mask, determine the object's weight information based on the attribute information, and perform weighted aggregation of the object quality scores of each object based on the weight information of each object to obtain the overall image quality score of the image to be evaluated.

[0097] Step S106: Generate and output the quality evaluation results of the image to be evaluated based on the overall image quality score and the object quality scores of each object.

[0098] It should be noted that, Figure 1The core of the method described lies in using "objects" as the basic evaluation unit. The image to be evaluated is first processed through a pre-trained segmentation model to obtain an initial segmentation result. Then, dynamic mask optimization is performed on the initial segmentation result to obtain an optimized segmentation mask, thereby improving the fit of the segmentation mask to object boundaries and regions in complex scenes. Based on this, semantic guidance processing is applied to the optimized segmentation mask to further clarify the final segmentation mask corresponding to each object in the image to be evaluated, while retaining the confidence information of each pixel in the final segmentation mask. This allows subsequent evaluation to utilize both region range and pixel reliability information simultaneously. Subsequently, for each object, based on the object's final segmentation mask and the confidence information of each pixel in the final segmentation mask, a preset... The system obtains scores for each of the multiple evaluation dimensions, and then merges these scores to form an object quality score, thereby achieving a comprehensive quantification of different quality attributes. Simultaneously, for each object, attribute information is determined based on its final segmentation mask, and weight information is determined from this attribute information. This weighted aggregation of object quality scores yields a weighted overall image quality score, reflecting the differences in the contribution of different objects to the overall image quality. Finally, the system generates and outputs the quality evaluation results of the image under evaluation based on the overall image quality score and the object quality scores of each object, thus providing clear, usable, and consistent quality evaluation output in terms of both "global conclusions" and "object-level interpretations."

[0099] In one possible implementation, multiple evaluation dimensions include at least one of the following: segmentation integrity dimension, feature stability dimension, edge accuracy dimension, occlusion robustness dimension, and quality persistence dimension.

[0100] It should be noted that the segmentation integrity dimension reflects the sufficiency of the final segmentation mask in covering the object region; the feature stability dimension reflects the consistency of the object-related feature performance under different quality perturbations; the edge precision dimension reflects the fitting accuracy of the final segmentation mask in the edge region; the occlusion robustness dimension reflects the reliability of the evaluation results in occluded scenarios; and the quality persistence dimension reflects the stable and persistent performance of the evaluation results under similar conditions or adjacent samples.

[0101] Under the constraints of this set of dimensions, the process of fusing the scores of each dimension to obtain the object quality score, and the subsequent weighted aggregation of the object quality scores to obtain the overall image quality score, can be supported by a clear, attributable, and reproducible dimensional system, thereby improving the interpretability of the quality evaluation results.

[0102] In one possible implementation, the image to be evaluated is input into a pre-trained segmentation model for initial segmentation processing to obtain the initial segmentation result output by the segmentation model; the initial segmentation result is then subjected to dynamic mask optimization processing to obtain an optimized segmentation mask, including:

[0103] Input the image to be evaluated into the segmentation model to obtain the initial mask of each object in the image to be evaluated and the initial confidence of each pixel in the initial mask. Use the initial mask and the initial confidence as the initial segmentation result.

[0104] A pre-assessment of image quality is performed on the image to be evaluated. Based on the results of the pre-assessment, the image quality level of the image to be evaluated is determined, and a corresponding threshold is determined based on the image quality level. The lower the image quality level, the lower the threshold.

[0105] Perform iterative optimization processing, which includes:

[0106] For each pixel in the initial mask, the region containing pixels whose initial confidence is less than a threshold is defined as a low-confidence region.

[0107] Extract edge pixels from low-confidence regions, generate local cue information based on edge pixels, input the local cue information into the segmentation model, and obtain the thinned mask of the low-confidence region output by the segmentation model and the thinning confidence of each pixel in the thinned mask;

[0108] The fusion weights are determined based on the initial confidence and the refined confidence. The initial mask and the refined mask are then weighted and fused according to the fusion weights to obtain the fused mask.

[0109] Calculate the intersection-union ratio (IUGR) between the fused mask obtained in the current iteration and the fused mask obtained in the previous iteration. Determine whether the IUGR reaches the preset convergence threshold. If yes, stop the iteration and use the current fused mask as the optimized segmentation mask. If no, use the current fused mask as the initial mask for the next iteration, use the confidence score of each pixel in the current fused mask as the initial confidence score for the next iteration, and return to perform iterative optimization processing.

[0110] It should be noted that this possible implementation method is in Figure 1 Based on the method shown, the implementation mechanism of "inputting the image to be evaluated into the pre-trained segmentation model for initial segmentation processing" and "performing dynamic mask optimization processing on the initial segmentation result to obtain the optimized segmentation mask" is further refined and constrained, so that the dynamic mask optimization processing can adaptively adjust according to the image quality pre-evaluation result of the image to be evaluated.

[0111] Specifically, the initial segmentation result consists of the initial mask of each object output by the segmentation model and the initial confidence of each pixel in the initial mask, so that subsequent optimization has a clear basis for pixel-level uncertainty; and the image quality level is determined by performing image quality pre-evaluation on the image to be evaluated, and then the corresponding threshold is determined by the image quality level, and the lower the image quality level, the lower the threshold, so that low-quality images are more likely to trigger targeted refinement under a more relaxed threshold condition. Furthermore, during iterative optimization, regions containing pixels with initial confidence scores below a threshold are defined as low-confidence regions. Edge pixels are extracted from these low-confidence regions to generate local cue information. This local cue information is then input into the segmentation model to obtain a refined mask and refined confidence score for the low-confidence regions. This concentrates the optimization efforts on the boundaries and local structures of the "most unreliable and most in need of correction" regions. Subsequently, fusion weights are determined based on the initial and refined confidence scores. The initial and refined masks are then weighted and fused to obtain a fused mask, ensuring overall stability while absorbing effective improvements from the refined results. The cross-intersection over union (CUI) ratio between the fused mask obtained in the current iteration and the fused mask obtained in the previous iteration is calculated to determine if a preset convergence threshold has been reached. If it has, the iteration stops and the optimized segmentation mask is output. If not, the current fused mask and its confidence score are fed back into the initial mask and initial confidence score for the next iteration, allowing for continued iteration. This ensures stable convergence and controllable quality improvement of the dynamic mask optimization process under consistent convergence criterion constraints.

[0112] In one possible implementation, the optimized segmentation mask undergoes semantic guidance processing to obtain the final segmentation mask corresponding to each object in the image to be evaluated, and the confidence information of each pixel in the final segmentation mask, including:

[0113] Feature extraction is performed on the optimized segmentation mask to obtain the mask features;

[0114] Obtain a predefined semantic template library, which includes multiple object categories, feature templates corresponding to each object category, and semantic priorities corresponding to each object category;

[0115] For each object category, calculate the cosine similarity between the mask features and the feature template corresponding to the object category; compare the cosine similarities corresponding to each object category, and determine the object category with the highest cosine similarity as the semantic label of the object.

[0116] The semantic priority is obtained from the semantic template library based on the semantic tag, and the attention weight of the object is determined based on the semantic priority. The higher the semantic priority, the greater the attention weight.

[0117] The optimized segmentation mask is associated with semantic labels and attention weights and stored to obtain the final segmentation mask of the object. The confidence of each pixel in the optimized segmentation mask is used as the confidence information of each pixel in the final segmentation mask.

[0118] It should be noted that this possible implementation method clearly defines the data organization and judgment criteria for "semantic guidance processing", so that the final segmentation mask not only has the ability to depict regions at the geometric level, but also has semantic information that can be used for attention scheduling.

[0119] It extracts mask features from the optimized segmentation mask and introduces a predefined semantic template library. This library contains multiple object categories, corresponding feature templates for each category, and semantic priorities for each category, providing a unified and standardized reference for semantic label determination and attention weight allocation. During the determination process, the cosine similarity between the mask features and the corresponding feature templates for each object category is calculated. The cosine similarities for each object category are compared, and the object category with the highest cosine similarity is determined as the object's semantic label, ensuring that the determination of semantic labels has a calculable and reproducible similarity basis. Subsequently, based on the linguistic... Semantic tags obtain corresponding semantic priorities from the semantic template library, and determine the attention weight of objects based on semantic priorities. The higher the semantic priority, the greater the attention weight, so that objects with different semantic tags have differentiated attention intensities in the subsequent evaluation and aggregation stages. Finally, the optimized segmentation mask is associated and stored with semantic tags and attention weights to obtain the final segmentation mask. The confidence of each pixel in the optimized segmentation mask is used as the confidence information of each pixel in the final segmentation mask. This ensures that the final segmentation mask maintains the continuity of pixel confidence while completing the consistent binding of "mask-semantics-weight", improving the interpretability and usability of the quality evaluation results.

[0120] In one possible implementation, for each object, the object's attribute information is determined based on the object's final segmentation mask, and the object's weight information is determined based on the attribute information, including:

[0121] For each object, calculate the mask area of ​​the final segmentation mask of the object, calculate the total image area of ​​the image to be evaluated, calculate the ratio of the mask area to the total image area, and use the ratio as the area factor.

[0122] Calculate the center position of the final segmentation mask of the object, calculate the center position of the image to be evaluated, calculate the distance between the center position of the mask and the center position of the image, calculate the diagonal length of the image to be evaluated, and determine the position factor based on the ratio of distance to diagonal length. The smaller the ratio of distance to diagonal length, the larger the position factor.

[0123] The final segmentation mask of the object is input into the pre-trained saliency model to obtain the saliency factor of the object output by the saliency model.

[0124] Extract the edge region of the final segmentation mask of the object, calculate the gradient value of each pixel in the edge region, calculate the mean of each gradient value, and use the mean as the boundary sharpness factor.

[0125] The area factor, position factor, salience factor, and boundary clarity factor are multiplied together to obtain the object's fusion weight.

[0126] Calculate the sum of the fusion weights of each object, and calculate the ratio of the fusion weight of each object to the sum of the fusion weights. Use this ratio as the weight information of the corresponding object.

[0127] It should be noted that this possible implementation defines the composition of "determining attribute information based on the final segmentation mask and determining weight information based on the attribute information" in a computable manner, so that the weight information can be directly derived from the final segmentation mask and has a clear physical meaning.

[0128] First, the mask area is calculated based on the final segmentation mask. The ratio of the mask area to the total image area is then used as an area factor to characterize the object's influence on the image's proportion. Simultaneously, the distance between the center of the final segmentation mask and the center of the image under evaluation is calculated, and the ratio of this distance to the diagonal length of the image under evaluation is used to determine the position factor. A smaller ratio indicates a larger position factor, reflecting the principle that objects located in the center of the image are more likely to affect the overall visual quality. Further, the final segmentation mask is input into a pre-trained saliency model to obtain a saliency factor, reflecting the object's quality influence weight in terms of visual saliency. Finally, the edge regions of the final segmentation mask are extracted, and the gradient values ​​of each pixel within these regions are calculated and averaged as a boundary sharpness factor, ensuring that the weight information reflects the clarity and recognizability of the boundary regions. Finally, the area factor, location factor, saliency factor, and boundary clarity factor are multiplied to obtain the fusion weight. The fusion weight of each object is summed and the ratio of the fusion weight to the sum of the fusion weights is taken as the weight information of the corresponding object. This yields normalized weight information that can be directly used for weighted aggregation, enabling the overall image quality score to be more closely aligned with the actual focus under the constraints of multiple factors such as area, location, saliency, and boundary clarity.

[0129] In one possible implementation, based on the final segmentation mask of the object and the confidence information of each pixel in the final segmentation mask, the scores of the object in each of the preset evaluation dimensions are calculated, including:

[0130] The image to be evaluated and the final segmentation mask of each object in the image to be evaluated are input into the pre-trained cross-domain transfer model to obtain the scores of each object in each of the preset multiple evaluation dimensions output by the cross-domain transfer model.

[0131] It should be noted that this possible implementation further clarifies the generation method of scores for each dimension, so that the scores for each dimension under the multiple preset evaluation dimensions for each object are directly output by a unified pre-trained cross-domain transfer model, thereby ensuring the consistency of multi-dimensional scores and cross-scene adaptability.

[0132] Specifically, the image to be evaluated and the final segmentation masks of each object in the image are input into a pre-trained cross-domain transfer model. This allows the cross-domain transfer model to simultaneously utilize the appearance information of the image to be evaluated and the object spatial constraints provided by the final segmentation masks. Under the same inference framework, it can provide scores for each of the multiple evaluation dimensions for each object. This approach avoids scale inconsistencies or accumulated biases caused by independent modeling of different evaluation dimensions. It makes it easier to obtain object quality scores through subsequent fusion processing. Furthermore, it supports the weighted aggregation of object quality scores using weight information to obtain the overall image quality score, thereby improving the stability and transferability of quality evaluation results under diverse data distributions.

[0133] In one possible implementation, the pre-trained cross-domain transfer model is trained in the following way:

[0134] Acquire source domain data and target domain data. The source domain data includes high-quality images and corresponding subjective ratings, while the target domain data includes low-quality scene images and corresponding subjective ratings.

[0135] Scene type identification is performed on each image in the target domain data to obtain the scene type of each image. The scene type includes at least one of the following: low light scene, rain and fog scene, and motion blur scene.

[0136] The corresponding image enhancement method is determined based on the scene type of each image, and the image is enhanced according to the image enhancement method to obtain source domain stylized target domain data; among them, low light scene corresponds to illumination enhancement method, rain and fog scene corresponds to defogging and deraining method, and motion blur scene corresponds to deblurring method.

[0137] The process involves training a pre-defined quality knowledge encoder and a pre-defined quality regression head based on source domain data. This includes: inputting high-quality images from the source domain data into the quality knowledge encoder to obtain high-quality features output by the quality knowledge encoder; inputting the high-quality features into the quality regression head to obtain the predicted score output by the quality regression head; calculating the quality prediction loss based on the difference between the predicted score and the subjective score; and optimizing the parameters of the quality knowledge encoder and the quality regression head based on the quality prediction loss.

[0138] The knowledge from the trained quality knowledge encoder and quality regression head is distilled into a lightweight network to obtain a lightweight quality encoder;

[0139] A domain-adaptive transfer architecture is constructed, which includes a shared feature extractor, a domain classifier, and a quality predictor; the shared feature extractor is initialized based on a lightweight quality encoder.

[0140] The source domain data is input into the shared feature extractor to obtain the source domain features output by the shared feature extractor;

[0141] The source domain stylized target domain data is input into the shared feature extractor to obtain the target domain features output by the shared feature extractor;

[0142] The source domain features and target domain features are input into the domain classifier respectively to obtain the domain classification result output by the domain classifier;

[0143] Calculate the domain adversarial loss based on the domain classification results, and optimize the parameters of the shared feature extractor and the quality predictor based on the quality prediction loss and the domain adversarial loss to reduce the difference between the source domain features and the target domain features.

[0144] The optimized shared feature extractor and quality predictor are used as cross-domain transfer models.

[0145] It should be noted that this possible implementation method systematically limits the training method of the pre-trained cross-domain transfer model, enabling the cross-domain transfer model to maintain effective learning and transfer of subjective scoring patterns even when there are distribution differences between the source domain data and the target domain data.

[0146] This approach acquires source and target domain data. The source domain data includes high-quality images and corresponding subjective ratings, while the target domain data includes low-quality scene images and corresponding subjective ratings. First, scene type identification is performed on each image in the target domain data to determine the scene type. Then, based on the scene type, corresponding image enhancement methods are determined and applied to obtain source-domain stylized target domain data. Specifically, low-light scenes correspond to illumination enhancement methods, rain and fog scenes to defogging and deraining methods, and motion-blurred scenes to deblurring methods. This stylization alignment of the target domain data with the source domain representation space is performed before training. Subsequently, a quality knowledge encoder and a quality regression head are trained based on the source domain data. High-quality features are obtained by inputting high-quality images into the quality knowledge encoder and then input into the quality regression head to output predicted scores. The quality prediction loss is calculated based on the difference between the predicted scores and subjective ratings, and the parameters of the quality knowledge encoder and quality regression head are optimized to achieve stable quality representation and regression capabilities. Finally, the knowledge from the trained quality knowledge encoder and quality regression head is distilled into a lightweight network to obtain a lightweight quality encoder, enabling a more adaptable deployment structure while maintaining capabilities.

[0147] A domain-adaptive transfer architecture is further constructed, which includes a shared feature extractor, a domain classifier, and a quality predictor. The shared feature extractor is initialized based on a lightweight quality encoder. Source domain data and source-domain stylized target domain data are respectively input into the shared feature extractor to obtain source domain features and target domain features, which are then respectively input into the domain classifier to obtain domain classification results. The parameters of the shared feature extractor and the quality predictor are jointly optimized by domain adversarial loss and quality prediction loss to reduce the difference between source domain features and target domain features. Finally, the optimized shared feature extractor and quality predictor are used as a cross-domain transfer model, so that the cross-domain transfer model can still output reliable scores in various dimensions under different scenarios and different quality distributions.

[0148] In one possible implementation, after obtaining the object's mass score, the method further includes:

[0149] The image to be evaluated is input into a pre-trained scene classifier to obtain the scene label of the image to be evaluated output by the scene classifier. The scene label includes at least one of the following: low light scene, rain and fog scene, occlusion scene and motion blur scene.

[0150] Obtain the preset scene correction parameter table, which includes the correspondence between each scene label, correction coefficient, and scene-specific offset;

[0151] Based on the scene label of the image to be evaluated, the corresponding target correction coefficient and the corresponding target scene-specific offset are retrieved from the scene correction parameter table;

[0152] The final object quality score is determined based on the corrected object quality score, the target correction factor, and the target scene-specific offset.

[0153] It should be noted that after obtaining the object quality score, a correction mechanism based on scene labels is further introduced, so that the object quality score can be calibrated consistently according to different scene conditions such as low light scene, rain and fog scene, occlusion scene and motion blur scene in which the image to be evaluated is located.

[0154] Scene labels are obtained by inputting the image to be evaluated into a pre-trained scene classifier. The correction coefficients and scene-specific offsets corresponding to the scene labels are determined by combining the preset scene correction parameter table. Thus, the score is compensated for in the form of "correction coefficients + scene-specific offsets". On this basis, the final object quality score is determined according to the corrected object quality score, the target correction coefficient, and the target scene-specific offset. This reduces the systematic bias introduced by scene factors in the final object quality score. In the subsequent weighted aggregation based on weight information to obtain the overall image quality score and generate the quality evaluation result, the comparability and stability of the scores between different scenes are guaranteed, and the practicality of the quality evaluation result under complex shooting conditions is improved.

[0155] In one possible implementation, the attribute information of the objects is determined based on the final segmentation mask of the objects, and the weight information of the objects is determined based on the attribute information. This includes: obtaining the final segmentation mask of each object, calculating the area of ​​each final segmentation mask respectively; calculating the sum of the areas of each final segmentation mask to obtain the total mask area, calculating the ratio of the area of ​​each final segmentation mask to the total mask area respectively, and using the ratio as the weight information of each object.

[0156] It should be noted that the final segmentation mask of each object can be obtained, and the area of ​​each final segmentation mask can be calculated separately. Then, the sum of the areas of each final segmentation mask can be calculated to obtain the total mask area. Based on this, the ratio of the area of ​​each final segmentation mask to the total mask area can be calculated, and the ratio can be used as the weight information of the object. In this way, the relative share of each object in the image can be described by the area to determine the weight information. This can concisely and efficiently determine the weight information of the object and reduce the computational complexity of the weight information.

[0157] In one possible implementation, the quality assessment results of the image to be evaluated are generated and output based on the overall image quality score and the object quality scores of each object, including:

[0158] Obtain the preset grid size and divide the image to be evaluated into multiple grid units according to the preset grid size;

[0159] For each grid cell, identify the objects whose final segmentation mask overlaps with the grid cell, obtain the object quality score of each identified object, calculate the mean of each object quality score, and use the mean as the cell quality score of the grid cell.

[0160] Obtain the preset color mapping rules, map the unit quality score of each grid cell to the corresponding display color according to the color mapping rules, and generate a quality heatmap based on each grid cell and the corresponding display color;

[0161] Obtain a preset scoring threshold. For each object, compare the object's quality score with the preset scoring threshold. If the object's quality score is lower than the preset scoring threshold, then the object is identified as a low-quality object.

[0162] For each low-quality object, obtain the score of each dimension of the low-quality object, compare the scores of each dimension, determine the dimension with the lowest score as the target dimension, and generate corresponding attribution prompt information based on the target dimension.

[0163] Output the overall image quality score, quality heatmap, and attribution suggestions.

[0164] It should be noted that this possible implementation method further limits the generation and output format of the quality assessment results, enabling the quality assessment results to simultaneously possess both the visualization expression at the whole-map level and the attribution explanation capability at the object level.

[0165] Specifically, the image to be evaluated is divided into multiple grid units by obtaining a preset grid size. Within each grid unit, objects whose final segmentation mask overlaps with the grid unit are identified. The object quality score of each identified object is obtained and the average value is calculated as the unit quality score of that grid unit. Thus, the object quality score is mapped to a gridded score with spatial distribution meaning. Then, according to the preset color mapping rules, the unit quality score of each grid unit is mapped to the corresponding display color and a quality heatmap is generated, so that the quality evaluation results can intuitively present the clustered areas of quality problems in the image space. Meanwhile, by obtaining a preset scoring threshold and comparing the object quality score of each object with the preset scoring threshold, objects below the preset scoring threshold are identified as low-quality objects. For each low-quality object, the scores of each dimension are compared to determine the dimension with the lowest score as the target dimension. Then, corresponding attribution prompts are generated based on the target dimension, thus clearly outputting the reason why "low-quality objects are low" in the form of the target dimension. Finally, the overall image quality score, quality heatmap, and attribution prompts are uniformly output, so that the quality evaluation results have three types of information: overall conclusion, spatial location, and cause prompts, improving the efficiency and consistency of subsequent quality inspection decisions and problem review.

[0166] In one possible implementation, the method further includes: responding to a user's region selection operation, determining a target region based on the region selection operation; for each object, calculating the intersection of the object's final segmentation mask and the target region; determining whether the intersection is empty, and if the intersection is not empty, identifying the object as a target object; for each target object, calculating the intersection area of ​​the intersections of the target objects; calculating the sum of the intersection areas of all target objects to obtain the total intersection area; calculating the ratio of the intersection area of ​​each target object to the total intersection area, and using the ratio as the target region weight of each target object; determining the region quality score of the target region based on the target region weight of each target object and the corresponding object quality score; generating a quality heatmap of the target region based on the region quality score, and outputting the quality heatmap.

[0167] It should be noted that after determining the target region, the intersection of the final segmentation mask of each object and the target region is calculated. The presence or absence of an empty intersection is used as the criterion; objects with non-empty intersections are identified as target objects, ensuring that region evaluation only covers objects actually related to the target region. Subsequently, the intersection area of ​​each target object is calculated, and the intersection areas of all target objects are summed to obtain the total intersection area. The ratio of the intersection area to the total intersection area is then used as the target region weight, allowing the target region weight to reflect the relative proportion of each target object within the target region. Finally, the region quality score of the target region is determined based on the target region weight of each target object and the corresponding object quality score. A quality heatmap of the target region is generated based on the region quality score and output. This allows users to perform consistent, interpretable, and visualized quality evaluation and review of local regions under the same object quality scoring system.

[0168] In this embodiment, the initial segmentation result is obtained by inputting the image to be evaluated into a pre-trained segmentation model, and dynamic mask optimization is performed on it, which significantly improves the contour accuracy and robustness of the object mask in scenes with blurred edges, complex backgrounds, and partial occlusion. Then, based on semantic guidance processing, each object obtains a final segmentation mask with clear semantic information and its pixel-level confidence, thereby establishing a structured representation from pixels to semantic objects, providing a fine-grained analytical basis for quality evaluation. On this basis, for each object, scores are calculated and fused on multiple preset dimensions according to its final segmentation mask and internal confidence information, realizing a multi-angle and comprehensive quantification of the visual quality of objects, overcoming the problem of incomplete evaluation by a single indicator. Furthermore, weights are determined according to the attribute information of each object and weighted aggregation is performed, so that the overall image quality score can simultaneously reflect the overall quality level and the importance differences of different semantic regions, enhancing the consistency between the evaluation results and human subjective perception. The final output is the quality evaluation result.

[0169] This achieves a leap from whole-image statistics to semantic object-level evaluation, improving the accuracy, refinement, and semantic interpretability of image quality assessment.

[0170] This paper presents an overall description of an image quality assessment method proposed in the embodiments of this application (for reference). Figure 2 Or, to put it another way, this application proposes a deep learning-based target-level image quality assessment method. It introduces the zero-shot segmentation capability of Segment Anything Model 2 (SAM2) into the image quality assessment system, achieving a paradigm shift from "whole-image scoring" to "object-level quantization + full-image aggregation." The method described in this application achieves a key improvement in image quality assessment: by upgrading from "indiscriminate whole-image evaluation" to "semantically aware object-level scoring," it lays the foundation for refined quality analysis. The specific implementation steps are as follows:

[0171] Step 1: SAM2 basic segmentation and enhancement strategy to achieve high-precision target segmentation.

[0172] 1.1 Dynamic Mask Refinement (DMR):

[0173] The core issue is that initial segmentation is prone to mask offsets or gaps in areas with blurred boundaries (such as glass reflections), complex backgrounds (such as crowds), and low-quality images (such as low light).

[0174] Implementation process: Generating the initial mask: Input the image into SAM2, and generate the initial target mask using point / box prompts. And output the confidence score of each pixel. (x, y are pixel coordinates); Confidence threshold determination: Set a dynamic threshold. (Adaptive image quality, low-quality image) (Reduce by 10%-20%), filter low-confidence areas. Iterative refinement: for Add local hints (such as edge pixels) to the region and regenerate the local mask. The initial and refined masks are then fused using a confidence-weighted method (according to the following formula):

[0175] ;

[0176] in:

[0177] ;

[0178] in, The confidence weights ensure that high-confidence regions dominate the final mask. Convergence is determined when the mask IoU (Intersection over Union) of two iterations is ≥ 0.95, and the final mask is output. The advantages include a 30%+ improvement in edge fit (experimentally verified on a reflective object dataset) and the resolution of segmentation errors for transparent / occluded objects.

[0179] 1.2 Semantic Segmentation Guidance:

[0180] Core principle: Utilize the semantic prior knowledge (such as feature templates for "face" and "vehicle") pre-trained by SAM2 to assign semantic labels to the segmentation results, avoiding confusion between similar-looking regions such as "face vs. doll".

[0181] Implementation process: Semantic label matching: Extracting SAM2 mask features (Dimensions C×H×W, where C is the number of feature channels), and a predefined semantic template library. ( Calculate the cosine similarity (using the feature template of the i-th object class):

[0182] ;

[0183] Take the largest Sim As a mask semantic label Key area guidance: Assign attention weights to high-priority semantic tags (such as "face" and "license plate"). (Ordinary area) This guides subsequent IQA models to focus on key regions; interactive segmentation: supports user input point prompts. SAM2 is based on Update mask: This enables targeted target segmentation (e.g., focusing only on the "text" region in an image).

[0184] Step 2: Construct a five-dimensional object quality index based on the SAM2 output to quantify the visual quality of a single object.

[0185] Final mask for a single target and its characteristics Using this as input, construct quality indicators across five dimensions. (Subsequently merged into the total mass score Q of the object), the calculation formulas for each dimension are shown in Table 1 below.

[0186] Table 1

[0187]

[0188] The parameters in the formula are explained below:

[0189] K: Number of prompts (default K=3, meaning the mask is generated through 3 different prompts) G: Target real area (annotated data); (Intersection over Union ratio, a measure of the degree of overlap between two regions); N: The total number of pixels in; : The average logits of all pixels in the region; Edge(A): the edge of region A extracted by Canny edge detection; mIoU: the intersection-union ratio of multiple targets (when the target is partially occluded, the mean IoU of the unoccluded region is calculated); In a video sequence, the target mask The number of consecutive frames that maintain an "active state" (IoU ≥ 0.7).

[0190] Furthermore, experimental verification showed that on the LIVE and TID2013 subjective datasets, the Pearson correlation coefficient (PCC) between the logits score of SAM2 and human subjective ratings was ≥0.7, proving that the above indicators can effectively model visual quality.

[0191] Step 3: Intelligent quality assessment (to achieve "object-level scoring → personalized / cross-domain adaptation").

[0192] 3.1 Adaptive Weight Assignment Mechanism (AWAM):

[0193] The core idea is that different objects contribute differently to the overall image quality (e.g., a face in the center is more important than the background at the edge), and weights need to be dynamically allocated according to object attributes.

[0194] Implementation process: Weight factor quantification: Calculate object weight factors from 4 dimensions: area factor ( (total image area); location factor (Central area) (≈1.0, marginal ≈0.5); significance factor (Output from a pre-trained saliency model, range [0,1]);

[0195] Boundary sharpness factor: (The higher the gradient value, the clearer the boundary.) (The larger the value); Weight normalization: Combine the above factors to obtain the final weight of the object. :

[0196] ;

[0197] Personalized Quality Standards via Deep Learning (PQSDL): Utilizing users' historical rating data... Training personalized regression networks :

[0198] ;

[0199] in, For users to view images The rating, By labeling users with object weight preferences, a personalized quality assessment can be achieved.

[0200] 3.2 Cross-Domain Knowledge Transfer (CDKT):

[0201] Core objective: To transfer high-quality knowledge from the "professional photography domain" (source domain S, high quality, normal lighting, no occlusion) to low-quality target domains T such as "surveillance video domain" and "medical imaging domain" (low light, rain, fog, motion blur), thereby improving the model's generalization ability in the target domain.

[0202] Implementation method (five-step method):

[0203] Step 1: Domain Definition and Data Preprocessing

[0204] Source domain data Includes 100,000+ professional photographic images (such as Flickr 30k-HQ), labeled with target masks. and human subjective rating ; Target domain data Includes 50,000+ low-quality surveillance images (such as UDIAC-Surveillance), labeled in the same way. Domain enhancement preprocessing: for Perform "domain feature simulation" to generate "source domain stylized target domain data". :

[0205] Low-light images: Enhancing lighting with RetinexNet ;

[0206] Rain and fog images: DehazeNet was used for dehazing and deraining. ;

[0207] Motion blur: Use DeblurGANv2 to remove blur. ;

[0208] Building a hybrid dataset This lays the data foundation for subsequent migrations.

[0209] Step 2: High-quality knowledge extraction module (source domain knowledge modeling)

[0210] Design a "Quality Knowledge Encoder" Extracting high-quality features and quality mapping patterns from source domain data: Feature extraction: Input to (Based on the ViT-L / 14 pre-trained model), outputs high-quality features from the source domain. Quality mapping learning: Training a quality regression head ,study To subjective rating Mapping:

[0211] ;

[0212] The loss function uses mean squared error (MSE):

[0213] ;

[0214] Knowledge distillation: + Knowledge distillation into lightweight networks This ensures the real-time performance of the model after migration (inference speed is improved by 2 times).

[0215] Step 3: Domain adaptive migration architecture to reduce domain differences:

[0216] It adopts a Domain-Adversarial Neural Network (DANN) architecture, which includes three core modules: a shared feature extractor. :based on Initialization, input image I (source domain / target domain) output shared features Domain classifier :judge Probability of output domain, whether from source domain or target domain Quality predictor :based on and mask features Output quality score ( , ).

[0217] Training objective: Quality prediction loss (supervised source domain quality): Domain adversarial loss (confused domain classifier, reducing domain discrepancies): ,in (source domain), (Target domain); Total loss: , ( Balanced quality prediction and domain adaptation).

[0218] Step 4: Optimize under extreme conditions to achieve fine-grained adaptation to the target domain:

[0219] For extreme scenarios in the target domain (such as low light or strong occlusion), a "scene perception correction module" is designed. :

[0220] Scene classification: Training a scene classifier The target domain image is divided into four categories: "low light," "rain and fog," "occlusion," and "motion blur," and scene labels are output. Correction coefficient calculation: For each scenario type, a specific correction coefficient is calculated for the pre-training scenario. (such as low-light scenes) (Enhancing quality sensitivity in low-brightness areas).

[0221] Quality rating correction:

[0222] ;

[0223] in, For scene-specific offsets (such as occluded scenes) (This reduces the overestimation of the quality of the obscured area).

[0224] Step 5: Verification of migration results:

[0225] Verify the transfer effect using the "Cross-Domain Pearson Correlation Coefficient (CD-PCC):" which calculates the target domain prediction score. Compared with human subjective ratings The correlation coefficient requires CD-PCC ≥ 0.65; Domain Difference Reduction Rate (DDR): Calculates the reduction ratio of the KL (Kullback–Leibler Divergence) divergence between the source and target domain features before and after migration.

[0226] ;

[0227] in, Features of the target domain before migration, For the features after migration, DDR ≥ 40% is required.

[0228] The advantages are: in the surveillance video domain test, the model reduces the Mean Absolute Error (MAE) of low-light images by 28% and that of rain and fog images by 32%, and the generalization ability is significantly improved.

[0229] Step 4: Full image quality aggregation and visualization output, from "object level" to "full image level" scoring and display.

[0230] 4.1 Area-weighted Object Quality (AOQ) aggregation function:

[0231] Core idea: Normalize the weights according to the object mask area, merge the quality scores of all objects, and generate the overall quality score of the whole image.

[0232] Calculation formula:

[0233] ;

[0234] Where, n: the total number of identifiable objects in the image; The fusion quality score of the i-th object (weighted fusion of five dimensions): = 0.2×OC + 0.2×FS + 0.2×EP + 0.2×OR + 0.2×QP (weights can be adjusted according to business needs). : The area normalization weight of the i-th object. The larger the area, the higher the contribution to the overall image quality.

[0235] ;

[0236] Example: If an image contains a "face" (Area=2000 pixels, Q=0.85) and a "background" (Area=8000 pixels, Q=0.6), then: , AOQ = 0.2×0.85 + 0.8×0.6 = 0.17 + 0.48 = 0.65 (The overall image quality score is 0.65 out of 1.0).

[0237] 4.2 Visualization output capability:

[0238] Quality Heatmap Generation: Grid Division: Divide the image into a 10×10 grid (100 cells in total); Cell Quality Score: Calculate the average quality score of all objects within each cell. (k is the number of objects in the cell); Color mapping: using a gradient of "green → yellow → red" ( ≥0.8 is green, 0.5≤ <0.8 is yellow. (Areas <0.5 are marked in red), generating a heatmap (I) to visually display low-quality areas. Low-quality component attribution hints: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] For objects, analyze the lowest score in the five-dimensional indicators and generate attribution text: If OR < 0.5: "Occlusion leads to a low occlusion robustness (OR) score, it is recommended to optimize the image acquisition angle to reduce occlusion"; if EP < 0.5: "Edge blurring leads to a decrease in edge precision (EP), it is recommended to enhance image edge sharpening processing"; if FS > 0.2 (large variance, unstable features): "Poor feature stability (FS), it is recommended to improve the stability of the image acquisition device".

[0239] Interactive Reassessment: User Operation: Supports users to select areas of interest using the mouse. Recomputation logic: Only retain The object, recalculated and Output: Returns the specific quality score and heatmap of the area of ​​interest to meet the needs of "targeted quality inspection" (such as focusing only on the "license plate" area in the monitoring image).

[0240] Based on the above overall method and process, the advantages of this application compared with the prior art and the core points of this application will be specifically introduced.

[0241] I. Background and Objectives of the Plan:

[0242] Existing image quality assessment methods (such as PSNR and SSIM) mostly focus on pixel-level differences across the entire image, lacking a refined evaluation of the quality of individual targets within the image. This proposal, based on SAM2's segmentation capabilities, trajectory management logic, and cross-target interaction mechanism, designs a target-level image quality assessment scheme (IQA) to quantify the quality of each target in the image and aggregate them into a global image quality score.

[0243] This solution is the first to apply SAM2 to an image quality assessment system. The core objectives of introducing SAM2 include: evaluating the completeness of target segmentation (e.g., whether the mask completely covers the target); quantifying the stability of target features (e.g., edge sharpness, texture consistency); handling target quality degradation in occluded scenes; and supporting zero-shot scenes (no need to train for specific target categories).

[0244] The revolutionary significance of this solution lies in achieving a fundamental shift in evaluation perspective from pixel-level to semantic-level—from evaluating how blurry an image is to assessing whether a face is clear. Existing methods simply cannot realistically explain pixel intensity differences, edge blurring, noise, and texture variations. SAM2, however, can perform zero-sample, high-precision instance-level semantic segmentation of images without additional training, identifying objects, regions, and their precise boundaries. Using SAM2's segmentation results as a guide means that IQA can shift from "How blurry is this image?" to "Is the outline of this object clear? Is the texture of this region distorted?" This semantic-level evaluation is more in line with human visual perception and practical application needs.

[0245] II. Technical Principles and Core Module Design

[0246] 2.1 Correlation between SAM2 segmentation accuracy and IQA:

[0247] The relationship between the accuracy of SAM2 segmentation and IQA can be analyzed from the following perspectives: (1) SAM2 segmentation and confidence output: SAM2 is used to generate target segmentation mask (M) and logits score (reflecting the stability of target features; the higher the score, the more reliable the features). Experimental verification shows that the higher the confidence of SAM2 object segmentation, the higher the corresponding object IQA score. (2) Target status judgment of trajectory management system: The target status classification logic of "reliable - pending - suspicious - lost" is used to quantify the target quality decay process. The judgment of the suspiciousness of the target in blurry, reflective, and low-light photos also affects the quality score of the target and the whole image. (3) Occlusion processing of cross-target interaction: The occlusion relationship is identified by the mask intersection-union ratio (mIoU). The influence of occlusion on target quality is judged by the variance of logits score. The IoU value of the mask output by SAM2 and the real target is used to judge the evaluation result of the occluded object in the IQA stage.

[0248] 2.2 Five-Dimensional Target Quality Evaluation System:

[0249] Image object-level quality assessment relies on the completeness of objects segmented by SAM2 as a partial basis. The segmentation results of SAM2 are evaluated from the following five dimensions, thus providing guidance for the IQA stage, as shown in Table 2.

[0250] Table 2

[0251]

[0252] Experimental results show that SAM2's logits score is strongly positively correlated with human subjective ratings (PCC ≥ 0.7), proving that it can be used for quality modeling.

[0253] III. Implementation of Core Technologies

[0254] 3.1 Step 1: Advanced Object-Level Segmentation

[0255] The embodiments of this application use the SAM2 model as the basic segmentation engine, and combine two key enhancement strategies to improve segmentation accuracy and semantic consistency.

[0256] Key Point 1: Dynamic Mask Refinement (DMR). This technique automatically adjusts the mask refinement strategy based on the initial segmentation results. For objects with blurred boundaries or complex backgrounds, it uses an iterative optimization algorithm to gradually approximate the true contours. The algorithm inputs include: X (input image or feature map), M_initial (initial predicted mask), F_initial (initial feature representation, optional, such as features from the backbone network), RefinementModule (mask optimization module, including an attention module and an iterative optimizer), StoppingCriterion (stopping condition function, such as based on confidence, change, or number of iterations), MaxIterations (maximum number of optimization iterations), and ConfidenceThreshold (confidence threshold used to determine whether optimization is needed; the mask optimization threshold can be dynamically adjusted by combining the loss value of the object-level quality score and the image quality score). The algorithm output M_final is the optimized object mask region. The biggest advantage of dynamic mask optimization is significantly improved edge fit, mainly solving scene problems involving reflective, transparent, or partially occluded objects in the image.

[0257] Key Point 2: Semantic-Guided Segmentation. This technique leverages SAM2's robust prior knowledge to provide semantic context for quality assessment, offering additional guidance to IQA (such as segmenting the target region of an object and factors like ambiguity and noise within that region). This helps distinguish objects with similar appearances and improves the semantic consistency of segmentation results. Its main functions include: distinguishing regions that are similar in appearance but different in meaning (e.g., a face vs. a doll); guiding the IQA model to focus on key semantic regions (e.g., faces, license plates, text); and supporting interactive evaluation—users can perform targeted quality checks by clicking / selecting specific regions of interest.

[0258] Semantic segmentation-guided image quality assessment is revolutionary, significantly improving its accuracy, interpretability, and semantic relevance. The core idea behind region-aware quality assessment is that different regions of an image have varying sensitivities to quality; for example, distortion in a face is more severe than distortion in the background sky. SAM2 can segment key semantic regions (such as faces, animals, text, and foreground objects) and non-key regions (such as a uniform background). The IQA model assigns different weights to the quality distortion of different regions based on the segmented region type, penalizing distortion in key regions more and tolerating distortion in non-key regions more, thus obtaining a weighted quality score that better reflects subjective perception. It can also generate a quality heatmap, visually displaying which semantic regions are of good quality and which are of poor quality, providing strong interpretability.

[0259] Step 1 Output: A list of segmented objects {Q1, Q2, ..., Q} n} and its refined mask information.

[0260] 3.2 Step 2: Intelligent Quality Assessment & Aggregation

[0261] Key Point 1: Adaptive Weight Assignment Mechanism (AWAM). This mechanism dynamically calculates the weight of an object's quality score in the total score based on factors such as its importance and location in the image. For example, an object in the center region may have a greater impact on the overall impression than an object in the edge region. Based on the SAM2 segmentation algorithm, its interactive segmentation capabilities (point and bounding box cues) can be combined, allowing users to specify the regions of interest. In this scheme, SAM2 can act as a prior knowledge engine, segmenting objects from distorted images and then using its powerful prior knowledge (obtained through training on massive amounts of data) to judge: whether the shape of the object is reasonable (e.g., a human face should not have distorted features), whether the texture of the object is natural (e.g., the texture of grass should be delicate rather than blocky), and whether the spatial relationships between objects are reasonable. The model can learn that a region identified by SAM2 as a "dog" should have texture and boundaries that conform to the typical characteristics of a "dog," enabling NR-IQA (No-Reference Image Quality Assessment) to more intelligently infer distortion, rather than simply detecting low-level statistical anomalies. The IQA system can dynamically respond to user concerns, perform quality assessment only on the SAM2 segmentation regions specified by the user, and provide explanations for why the quality of this region is poor (such as "the face region you are concerned about has a boundary that is too far from the Hausdorff distance of the ideal contour"), which greatly improves user experience and system transparency.

[0262] Key Point 2: Personalized Quality Standards via Deep Learning (PQSDL). This involves training a personalized quality regression network using user preference data, enabling quality assessments to better align with the aesthetic habits or application scenario requirements of specific users.

[0263] Key Point 3: Cross-Domain Knowledge Transfer (CDKT). This involves transferring knowledge learned from high-quality domains (such as professional photography) to low-quality domains (such as surveillance video), improving the ability to evaluate images of different quality levels.

[0264] The method is implemented in two layers. Object-Level Quality Scoring: Each segmented object image patch Q_i is input into a pre-trained Quality Regression Network (QRN). This network can be built based on a combination of CNN (Convolutional Neural Network) and Transformer architectures for quality assessment. It is trained on large-scale image quality datasets (such as AVA, KonIQ-10k, SPAQ) and outputs a perceptual quality score q_i∈[0,10] for each object. The higher the score, the better the visual quality of the object region (such as the comprehensive performance of sharpness, noise, blur, composition, etc.). This method obtains the quality scores of various objects and avoids the differences caused by the regions where the objects are located. Image-Level Quality Aggregation: This method combines the quality scores of all objects to calculate the overall quality score Q_total for the entire image. It employs various fusion strategies, using weighted averages based on factors such as object area, saliency, boundary sharpness, and object segmentation precision. Max / min pooling can be used to reflect the worst or best region quality; alternatively, a Deep Aggregator can be employed, which uses a small neural network to learn a non-linear fusion function that considers the relationships between objects. A schematic diagram illustrating the relationship between object-level quality scores and the overall image quality score is shown below. Figure 3 As shown.

[0265] The All Objects Quality aggregation (AOQ) formula is as follows: AOQ = (∑(SOQ_i × Area_i)) / (∑Area_i), where SOQ_i (SingleObject Quality score) is the single-object quality score of the i-th object (weighted fusion of 5 dimensions, with weights calibrated through subjective evaluation experiments, and the default weight of each dimension is 0.2), and Area_i is the target mask area.

[0266] Step 2 Output: Mass fraction for each object {q1, q2, ..., q} n}; Overall image quality score Q_total; Based on the interactive points of interest (emphasis, as real-world photos may have different shooting conditions such as background blur), the overall image quality score Q̄_total is recalculated; A visual heatmap displays the quality distribution of each object area.

[0267] IV. Detailed Implementation Steps

[0268] 4.1 Input preprocessing: The input image (supporting RGB format, resolution ≥ 512×512) generates a feature map through the image encoder of SAM2; in the "auto prompt" mode of SAM2, segmentation masks (M_i) and logits scores (L_i) of all targets in the image are generated.

[0269] 4.2 Target screening and state initialization: Filter low-quality masks, retain targets with logits > τ_s (τ_s = 2.0), and eliminate noisy masks; initialize the target state, and classify targets into "reliable" (logits > τ_r = 8.0), "to be confirmed" (6.0 < logits ≤ 8.0), etc. according to the logits score.

[0270] 4.3 Target-level quality calculation: The segmentation integrity (OC) calculates the pixel ratio of the mask to the bounding rectangle of the target; the feature stability (FS) calculates the variance of the logits in the target area and normalizes it to [0,1]; the edge precision (EP) extracts the mask edge and the Canny edge and calculates the IoU; the occlusion robustness (OR) calculates the occlusion impact factor for overlapping targets with mIoU > 0.8; the quality persistence (QP) simulates the state maintenance logic of the trajectory management system and evaluates the target quality decay rate.

[0271] 4.4 Full-image quality output and visualization: Output the AOQ score (0 - 100 points, the higher the better the full-image target quality); generate a quality heat map, label the low-quality target areas (SOQ_i < 50), and prompt the low-scoring items (such as "low OR score caused by occlusion").

[0272] V. Key enhancement strategy: Multi-color space training

[0273] This solution adopts a multi-color space training strategy, converts the image into multiple spaces such as RGB, YUV, and Lab, captures visual features in different dimensions (such as brightness, hue, saturation), and enhances the robustness to interference such as illumination changes and reflections. Experiments show that indicators such as PLCC (Pearson Linear Correlation Coefficient) and SRCC (Spearman Rank Order Correlation Coefficient) of the multi-color space combination (ALL) are better than those of a single space.

[0274] Different color spaces have different focuses on the quality of photos: (1) The RGB space directly corresponds to the human eye's photoreceptor cell response with red, green and blue channels, which intuitively reflects the original distribution of colors. It can accurately capture the color gamut range, color deviation and inter-channel correlation (such as white balance deviation causing channel imbalance). It is sensitive to quality problems such as color distortion and artifacts (such as color blocks caused by compression). However, it is directly affected by light intensity. Brightness changes can easily cover up color details, and its robustness is relatively weak. (2) The YUV space separates brightness (Y) and chromaticity (U, V). The Y channel carries brightness and darkness information alone, while the U and V channels describe color differences. The Y channel is sensitive to brightness distortion such as light changes and over / underexposure. It can accurately reflect quality indicators such as contrast and dynamic range. The U and V channels focus on hue shift and abnormal color saturation (such as fading and color overflow). Since the chromaticity component is usually compressed and stored, it is more sensitive to compressed artifacts (such as chromaticity noise). It is suitable for evaluating the quality stability under light fluctuations. (3) In the Lab space, the L channel represents brightness (strongly correlated with illumination), while the a (green-red) and b (blue-yellow) channels describe the opposite dimensions of color. The L channel is sensitive to the loss of detail in shadows / highlights and brightness uniformity (such as halos and vignetting). The a / b channels can quantify hue accuracy (such as skin tone deviation) and color richness (such as insufficient saturation). Moreover, because they are device-independent, they can objectively reflect the color consistency across devices and are more robust in recognizing local color shifts caused by material reflections. By integrating the differentiated sensitivity characteristics of each space to brightness, hue, and saturation, the multi-space combination can cover various quality problems such as illumination changes, reflection interference, and compression distortion. Therefore, it performs better in quality assessment indicators and significantly improves the comprehensiveness and robustness of quality quantification.

[0275] VI. Experimental Verification Plan

[0276] 6.1 Dataset Selection

[0277] This scheme uses three datasets: (1) DanceTrack (high similarity targets, frequent occlusion); (2) UAVDT (UAV perspective, small targets); (3) BDD100K (complex scenes, multiple weather conditions). At the same time, artificial degradation processing is performed: Gaussian noise, blurring, occlusion and other degradation are added to the dataset images to generate quality gradient samples.

[0278] 6.2 Evaluation Indicators and Comparison Methods

[0279] The core metrics are the Pearson correlation coefficient and Spearman rank correlation coefficient with human subjective ratings. Comparison methods include: traditional full-image metrics such as PSNR, SSIM, and LPIPS (Learned Perceptual Image Patch Similarity) and target-level methods (quality assessment based on detection boxes, such as combining YOLO target sharpness scoring).

[0280] 6.3 Expected Results and Performance Verification

[0281] Expected performance metrics: On the DanceTrack dataset, the full-module approach achieves a SROCC ≥ 0.85, an improvement of ≥ 0.15 compared to the baseline (validating the effect of cross-target interaction); in the UAVDT small target scenario, the EPE (End-Point Error) of the EP dimension is ≤ 0.05, significantly lower than the baseline method based on detection boxes (EPE ≥ 0.12); the PCC of the AOQ score and the full-image MOS (Mean Opinion Score) of BDD100K are ≥ 0.8, demonstrating the effectiveness of area-weighted aggregation. This approach outperforms traditional methods in both PLCC and SROCC on three datasets, especially in occluded scenarios (such as DanceTrack) (expected improvement ≥ 0.15); the target-level heatmap accurately locates low-quality regions, with consistency ≥ 85% with manual annotations.

[0282] Module contribution analysis: As shown by the heatmap, the trajectory management module reduced the prediction error of the reliable → lost state in the QP dimension by ≥30%; in the occluded area where mIoU>0.8, the cross-target interaction module improved the consistency between the OR dimension score and the subjective score by ≥25%.

[0283] Zero-shot generalization validation: For the "animal" category not covered in the dataset used in this scheme (such as the COCO subset), the PCC of the ZSA index is ≥0.75, validating the transferability of SAM2's zero-shot segmentation ability in quality assessment.

[0284] Based on the SAM2 extension capabilities (zero-sample segmentation, trajectory management, and cross-target interaction) proposed in this scheme, the performance of the image quality evaluation scheme in terms of the accuracy of target-level quality quantification and the effectiveness of full-image quality aggregation is verified. The results show that: (1) the target quality dimensions (such as edge precision and feature stability) based on the SAM2 segmentation mask are consistent with human subjective perception; (2) after introducing trajectory management and cross-target interaction mechanisms, the scheme has better robustness to the evaluation of occluded and low-quality targets than existing methods.

[0285] VII. Limitations and Optimization Directions

[0286] Limitations: It relies on the segmentation accuracy of SAM2; extremely low-quality images (such as severely blurred images) may cause the mask to fail; the evaluation accuracy for transparent / reflective targets is relatively low.

[0287] Optimization directions: Improve the segmentation stability of low-quality images by combining super-resolution preprocessing; enhance edge detection of transparent targets by introducing 3D geometric constraints. This stage aims to provide an optimization direction for future researchers.

[0288] VIII. Core Value of the Solution

[0289] This solution is the first to transfer SAM2 segmentation-tracking technology to the field of image quality assessment, breaking through the limitations of traditional methods that "emphasize the whole image and neglect the target." It achieves zero-sample, target-level, and interpretable quality assessment, which can be directly applied to scenarios such as surveillance image quality inspection and generative AI (Artificial Intelligence) image verification. Its technical approach is strictly based on the SAM2 extended logic, trajectory management, and cross-target interaction mechanism proposed in the embodiments of this application. It provides target-level quality scores for different objects in the image, supports quality score output for different regions of the image, and outputs the quality score of the entire image, while ensuring the reproducibility and innovation of the method.

[0290] In summary, this solution possesses three core advantages: First, versatility. Based on SAM2's zero-shot segmentation capability, the system supports quality assessment for any object category, handling diverse objects such as animals, text, and vehicles without requiring retraining for specific target types. Second, refinement. The solution independently scores the quality of each identifiable object in the image, effectively avoiding the problem of the overall quality score masking local defects. Finally, interpretability. The system outputs a quality heatmap and provides attribution analysis for low-scoring items, clearly indicating "where the problem lies and why," providing clear directions for subsequent image optimization.

[0291] Figure 4 An image quality assessment apparatus according to an embodiment of this application is shown, such as... Figure 4 As shown, the device 40 includes:

[0292] The acquisition module 401 is used to acquire the image to be evaluated, input the image to be evaluated into the pre-trained segmentation model for initial segmentation processing, and obtain the initial segmentation result output by the segmentation model.

[0293] Execution module 402 is used to perform dynamic mask optimization processing on the initial segmentation result to obtain the optimized segmentation mask;

[0294] The optimized segmentation mask is subjected to semantic guidance processing to obtain the final segmentation mask corresponding to each object in the image to be evaluated, and the confidence information of each pixel in the final segmentation mask;

[0295] For each object, based on the final segmentation mask of the object and the confidence information of each pixel in the final segmentation mask, the scores of each dimension of the object under multiple preset evaluation dimensions are calculated, and the scores of each dimension of the object are fused to obtain the object quality score.

[0296] For each object, the object's attribute information is determined based on the final segmentation mask, the object's weight information is determined based on the attribute information, and the object quality scores of each object are weighted and aggregated based on the weight information of each object to obtain the overall image quality score of the image to be evaluated.

[0297] The quality evaluation results of the image to be evaluated are generated and output based on the overall image quality score and the object quality scores of each object.

[0298] In this embodiment, the initial segmentation result is obtained by inputting the image to be evaluated into a pre-trained segmentation model, and dynamic mask optimization is performed on it, which significantly improves the contour accuracy and robustness of the object mask in scenes with blurred edges, complex backgrounds, and partial occlusion. Then, based on semantic guidance processing, each object obtains a final segmentation mask with clear semantic information and its pixel-level confidence, thereby establishing a structured representation from pixels to semantic objects, providing a fine-grained analytical basis for quality evaluation. On this basis, for each object, scores are calculated and fused on multiple preset dimensions according to its final segmentation mask and internal confidence information, realizing a multi-angle and comprehensive quantification of the visual quality of objects, overcoming the problem of incomplete evaluation by a single indicator. Furthermore, weights are determined according to the attribute information of each object and weighted aggregation is performed, so that the overall image quality score can simultaneously reflect the overall quality level and the importance differences of different semantic regions, enhancing the consistency between the evaluation results and human subjective perception. The final output is the quality evaluation result.

[0299] This achieves a leap from whole-image statistics to semantic object-level evaluation, improving the accuracy, refinement, and semantic interpretability of image quality assessment.

[0300] This application provides a network device 50, such as... Figure 5 As shown, the network device 50 includes a processor 501, a memory 502, and a program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, it implements the steps of an image quality evaluation method as shown in the above embodiment.

[0301] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of an image quality evaluation method as shown in the above embodiments, achieving the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0302] This application also provides a computer program product, including computer instructions. When executed by a processor, the computer instructions implement the steps of the image quality evaluation method shown in the above embodiments and achieve the same technical effect. To avoid repetition, the details will not be repeated here.

[0303] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0304] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0305] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image quality assessment method, characterized in that, The method includes: The image to be evaluated is acquired, and the image to be evaluated is input into a pre-trained segmentation model for initial segmentation processing to obtain the initial segmentation result output by the segmentation model. The initial segmentation result is subjected to dynamic mask optimization processing to obtain the optimized segmentation mask; The optimized segmentation mask is subjected to semantic guidance processing to obtain the final segmentation mask corresponding to each object in the image to be evaluated, and the confidence information of each pixel in the final segmentation mask; For each object, based on the final segmentation mask of the object and the confidence information of each pixel in the final segmentation mask, the scores of the object in each of the preset multiple evaluation dimensions are calculated respectively, and the scores of the object in each dimension are fused to obtain the object quality score. For each object, the attribute information of the object is determined according to the final segmentation mask of the object, the weight information of the object is determined according to the attribute information, and the object quality scores of each object are weighted and aggregated according to the weight information of each object to obtain the overall image quality score of the image to be evaluated. The quality evaluation result of the image to be evaluated is generated and output based on the overall image quality score and the object quality scores of each object.

2. The method according to claim 1, characterized in that, The image to be evaluated is input into a pre-trained segmentation model for initial segmentation processing to obtain the initial segmentation result output by the segmentation model. The initial segmentation result is subjected to dynamic mask optimization processing to obtain an optimized segmentation mask, including: The image to be evaluated is input into the segmentation model to obtain the initial mask of each object in the image to be evaluated and the initial confidence of each pixel in the initial mask. The initial mask and the initial confidence are used as the initial segmentation result. The image quality of the image to be evaluated is pre-evaluated, the image quality level of the image to be evaluated is determined based on the result of the pre-evaluation, and a corresponding threshold is determined based on the image quality level, wherein the lower the image quality level, the lower the threshold. Perform iterative optimization processing, wherein the iterative optimization processing includes: For each pixel in the initial mask, the region containing pixels whose initial confidence is less than the threshold is defined as a low-confidence region. Extract the edge pixels of the low confidence region, generate local cue information based on the edge pixels, input the local cue information into the segmentation model, and obtain the thinning mask of the low confidence region and the thinning confidence of each pixel in the thinning mask output by the segmentation model; The fusion weights are determined based on the initial confidence level and the refined confidence level. The initial mask and the refined mask are then weighted and fused according to the fusion weights to obtain the fused mask. Calculate the intersection-union ratio (IUGR) between the fused mask obtained in the current iteration and the fused mask obtained in the previous iteration, and determine whether the IUGR reaches a preset convergence threshold. If yes, stop the iteration and use the current fused mask as the optimized segmentation mask; otherwise, use the current fused mask as the initial mask for the next iteration, use the confidence score of each pixel in the current fused mask as the initial confidence score for the next iteration, and return to execute the iterative optimization process.

3. The method according to claim 1, characterized in that, The optimized segmentation mask is subjected to semantic guidance processing to obtain the final segmentation mask corresponding to each object in the image to be evaluated, and the confidence information of each pixel in the final segmentation mask, including: Feature extraction is performed on the optimized segmentation mask to obtain mask features; Obtain a predefined semantic template library, wherein the semantic template library includes multiple object categories, feature templates corresponding to each object category, and semantic priorities corresponding to each object category; For each object category, calculate the cosine similarity between the mask feature and the feature template corresponding to the object category; compare the cosine similarities corresponding to each object category, and determine the object category with the largest cosine similarity as the semantic label of the object; The semantic priority is obtained from the semantic template library according to the semantic tag, and the attention weight of the object is determined according to the semantic priority, wherein the higher the semantic priority, the greater the attention weight; The optimized segmentation mask is associated with and stored with the semantic label and the attention weight to obtain the final segmentation mask of the object, and the confidence of each pixel in the optimized segmentation mask is used as the confidence information of each pixel in the final segmentation mask.

4. The method according to claim 1, characterized in that, For each of the objects, attribute information of the object is determined based on the final segmentation mask of the object, and weight information of the object is determined based on the attribute information, including: For each of the objects, calculate the mask area of ​​the final segmentation mask of the object, calculate the total image area of ​​the image to be evaluated, calculate the ratio of the mask area to the total image area, and use the ratio as the area factor; Calculate the center position of the final segmentation mask of the object, calculate the center position of the image to be evaluated, calculate the distance between the center position of the mask and the center position of the image, calculate the diagonal length of the image to be evaluated, and determine the position factor based on the ratio of the distance to the diagonal length, wherein the smaller the ratio of the distance to the diagonal length, the larger the position factor; The final segmentation mask of the object is input into a pre-trained saliency model to obtain the saliency factor of the object output by the saliency model. Extract the edge region of the final segmentation mask of the object, calculate the gradient value of each pixel in the edge region, calculate the mean of each gradient value, and use the mean as the boundary sharpness factor; The area factor, the position factor, the saliency factor, and the boundary clarity factor are multiplied together to obtain the fusion weight of the object; Calculate the sum of the fusion weights of each object, calculate the ratio of the fusion weight of each object to the sum of the fusion weights, and use the ratio as the weight information of the corresponding object.

5. The method according to claim 1, characterized in that, Based on the final segmentation mask of the object and the confidence information of each pixel in the final segmentation mask, the scores of the object in each of the preset evaluation dimensions are calculated, including: The image to be evaluated and the final segmentation mask of each object in the image to be evaluated are input into a pre-trained cross-domain transfer model to obtain the scores of each object in each of the preset multiple evaluation dimensions output by the cross-domain transfer model.

6. The method according to claim 5, characterized in that, The pre-trained cross-domain transfer model is trained in the following way: Acquire source domain data and target domain data, wherein the source domain data includes high-quality images and corresponding subjective ratings, and the target domain data includes low-quality scene images and corresponding subjective ratings; Scene type identification is performed on each image in the target domain data to obtain the scene type of each image, wherein the scene type includes at least one of the following: low light scene, rain and fog scene, and motion blur scene; The corresponding image enhancement method is determined according to the scene type of each image, and the image is enhanced according to the image enhancement method to obtain source domain stylized target domain data; wherein, the dark light scene corresponds to the illumination enhancement method, the rain and fog scene corresponds to the defogging and deraining method, and the motion blur scene corresponds to the deblurring method. Training a preset quality knowledge encoder and a preset quality regression head based on the source domain data includes: inputting high-quality images from the source domain data into the quality knowledge encoder to obtain high-quality features output by the quality knowledge encoder; inputting the high-quality features into the quality regression head to obtain a predicted score output by the quality regression head; calculating a quality prediction loss based on the difference between the predicted score and the subjective score; and optimizing the parameters of the quality knowledge encoder and the quality regression head based on the quality prediction loss. The knowledge from the trained quality knowledge encoder and the quality regression head is distilled into a lightweight network to obtain a lightweight quality encoder; A domain adaptive migration architecture is constructed, wherein the domain adaptive migration architecture includes a shared feature extractor, a domain classifier, and a quality predictor; the shared feature extractor is initialized based on the lightweight quality encoder; The source domain data is input into the shared feature extractor to obtain the source domain features output by the shared feature extractor; The source domain stylized target domain data is input into the shared feature extractor to obtain the target domain features output by the shared feature extractor; The source domain features and the target domain features are respectively input into the domain classifier to obtain the domain classification result output by the domain classifier; Calculate the domain adversarial loss based on the domain classification result, and optimize the parameters of the shared feature extractor and the quality predictor based on the quality prediction loss and the domain adversarial loss to reduce the difference between the source domain features and the target domain features; The optimized shared feature extractor and the quality predictor are used as the cross-domain migration model.

7. The method according to claim 5, characterized in that, After obtaining the object's mass score, the method further includes: The image to be evaluated is input into a pre-trained scene classifier to obtain the scene label of the image to be evaluated output by the scene classifier; Obtain a preset scene correction parameter table, wherein the scene correction parameter table includes the correspondence between each scene label, correction coefficient, and scene-specific offset; Based on the scene label of the image to be evaluated, the corresponding target correction coefficient and the corresponding target scene-specific offset are retrieved from the scene correction parameter table; The final object quality score is determined based on the corrected object quality score, the target correction coefficient, and the target scene-specific offset.

8. The method according to claim 1, characterized in that, The object's attribute information is determined based on its final segmentation mask, and the object's weight information is determined based on the attribute information, including: Obtain the final segmentation mask for each of the objects, and calculate the area of ​​each final segmentation mask; The sum of the areas of each final segmentation mask is calculated to obtain the total mask area. The ratio of the area of ​​each final segmentation mask to the total mask area is then calculated, and the ratio is used as the weight information of each object.

9. The method according to any one of claims 1-8, characterized in that, The quality evaluation result of the image to be evaluated is generated and output based on the overall image quality score and the object quality scores of each object, including: Obtain a preset grid size, and divide the image to be evaluated into multiple grid units according to the preset grid size; For each of the grid cells, identify the objects whose final segmentation mask overlaps with the grid cell, obtain the object quality score of each identified object, calculate the average of the object quality scores, and use the average as the cell quality score of the grid cell. Obtain a preset color mapping rule, map the unit quality score of each grid cell to the corresponding display color according to the color mapping rule, and generate a quality heatmap based on each grid cell and the corresponding display color. Obtain a preset scoring threshold. For each object, compare the object quality score with the preset scoring threshold. If the object quality score is lower than the preset scoring threshold, then the object is identified as a low-quality object. For each low-quality object, obtain the scores of each dimension of the low-quality object, compare the scores of each dimension, determine the dimension with the lowest score as the target dimension, and generate corresponding attribution prompt information based on the target dimension; Output the overall image quality score, the quality heatmap, and the attribution information.

10. The method according to claim 9, characterized in that, The method further includes: In response to the user's region selection operation, the target region is determined based on the region selection operation; For each object, calculate the intersection of the final segmentation mask of the object and the target region; determine whether the intersection is empty; if the intersection is not empty, then identify the object as the target object. For each target object, calculate the intersection area of ​​the intersection of the target objects; calculate the sum of the intersection areas of all the target objects to obtain the total intersection area; calculate the ratio of the intersection area of ​​all the target objects to the total intersection area, and use the ratio as the target region weight of each target object. The region quality score of the target region is determined based on the target region weight of each target object and the corresponding object quality score; Generate a quality heatmap of the target area based on the regional quality score, and output the quality heatmap.

11. The method according to any one of claims 1-8, characterized in that, The multiple evaluation dimensions include at least one of the following: segmentation integrity dimension, feature stability dimension, edge accuracy dimension, occlusion robustness dimension, and quality consistency dimension.

12. An image quality evaluation device, characterized in that, The device includes: The acquisition module is used to acquire the image to be evaluated, input the image to be evaluated into a pre-trained segmentation model for initial segmentation processing, and obtain the initial segmentation result output by the segmentation model. The execution module is used to perform dynamic mask optimization processing on the initial segmentation result to obtain the optimized segmentation mask; The optimized segmentation mask is subjected to semantic guidance processing to obtain the final segmentation mask corresponding to each object in the image to be evaluated, and the confidence information of each pixel in the final segmentation mask; For each object, based on the final segmentation mask of the object and the confidence information of each pixel in the final segmentation mask, the scores of the object in each of the preset multiple evaluation dimensions are calculated respectively, and the scores of the object in each dimension are fused to obtain the object quality score. For each object, the attribute information of the object is determined according to the final segmentation mask of the object, the weight information of the object is determined according to the attribute information, and the object quality scores of each object are weighted and aggregated according to the weight information of each object to obtain the overall image quality score of the image to be evaluated. The quality evaluation result of the image to be evaluated is generated and output based on the overall image quality score and the object quality scores of each object.

13. A network device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of an image quality assessment method as claimed in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of an image quality assessment method as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of an image quality assessment method as described in any one of claims 1 to 11.

Images