Image quality evaluation method and apparatus

Abstract

This application provides an image quality assessment method and apparatus. The method includes: acquiring an image to be assessed; determining an image quality assessment scenario, wherein the image quality assessment scenario includes a first scenario with a standard reference image and a second scenario without a standard reference image; if the image quality assessment scenario is the first scenario, activating a pre-trained parametric quality assessment network and inputting the image to be assessed, the standard reference image, and the region of interest mask of the standard reference image into the parametric quality assessment network to obtain a first quality score of the image to be assessed on the region of interest; if the image quality assessment scenario is the second scenario, activating a pre-trained non-parametric quality assessment network and inputting the image to be assessed into the non-parametric quality assessment network to obtain a second quality score of the image to be assessed.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and more specifically, to an image quality assessment method and apparatus. Background Technology

[0002] In the field of industrial automated production, accurate product quality assessment is a crucial step in ensuring production reliability and yield. Machine vision technology, due to its advantages of non-contact operation, high efficiency, and high precision, has been widely applied to tasks such as surface defect detection, dimensional measurement, and assembly verification of industrial products. Image quality assessment, as an important branch of machine vision, aims to quantify the degree of distortion in the image itself or the visual quality degradation of the product surface, and is of great significance in ensuring that products meet factory standards.

[0003] In recent years, with the development of deep learning technology, a series of image quality assessment algorithms based on neural networks have been proposed. These techniques typically require collecting datasets of hundreds or thousands of images with subjective quality scores and training the evaluation network based on this labeled data. For example, some full-reference evaluation methods require pairs of reference and distorted images to train the model, learning the quality differences between them; while no-reference evaluation methods directly learn a complex function mapping to quality scores from the distorted image itself. These methods have achieved good performance on specific datasets.

[0004] However, the aforementioned existing technologies face numerous challenges in practical industrial applications. First, industrial products have stringent quality standards, and the visual factors affecting quality are complex and diverse, ranging from subtle texture differences to overall color deviations, with numerous evaluation dimensions. Collecting large amounts of paired data to train a dedicated evaluation model for each quality degradation mode is not only labor-intensive and costly, but also virtually impossible in real-world industrial scenarios where obtaining a perfect reference image covering all situations is nearly impossible. Second, in many non-standardized production processes or when sampling existing products, often only the image to be evaluated exists, rendering traditional parametric evaluation methods that rely on reference images completely ineffective. Furthermore, the evaluation capabilities of existing deep learning models are fixed on the statistical characteristics of the training data, and their evaluation performance is limited by the scale and diversity of the training data, resulting in insufficient model generalization ability. Once evaluation standards or scenarios change, such as the introduction of new distortion types or switching to a completely new product line, it is necessary to re-collect data and retrain the model to adapt to the new evaluation task. This significantly increases the maintenance and update costs of the model, making it difficult to deploy stably in changing and complex industrial environments. Summary of the Invention

[0005] This application provides an image quality assessment method and apparatus for automatically screening high-quality and highly diverse training samples from massive image data of industrial production lines, replacing inefficient manual screening and solving the problems of low model training efficiency and insufficient generalization ability caused by the mixing of low-quality and redundant data.

[0006] Firstly, an image quality assessment method is provided, including: Acquire the image to be evaluated; The image quality assessment scenario is determined, which includes a first scenario where a standard reference image exists and a second scenario where a standard reference image does not exist. If the image quality assessment scenario is the first scenario, then the pre-trained parametric quality assessment network is activated, and the image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image are input into the parametric quality assessment network to obtain the first quality score of the image to be evaluated on the region of interest. If the image quality assessment scenario is the second scenario, then the pre-trained parameterless quality assessment network is activated, and the image to be assessed is input into the parameterless quality assessment network to obtain the second quality score of the image to be assessed.

[0007] Secondly, an image quality assessment device is provided, comprising: The image acquisition module is used to acquire the image to be evaluated. The scene judgment module is used to judge the image quality assessment scene, which includes a first scene with a standard reference image and a second scene without a standard reference image; The parametric evaluation module includes a pre-trained parametric quality evaluation network, which is used to input the image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image into the parametric quality evaluation network when the image quality evaluation scenario is the first scenario, so as to obtain a first quality score of the image to be evaluated on the region of interest. The parameterless evaluation module includes a pre-trained parameterless quality evaluation network, which is used to input the image to be evaluated into the parameterless quality evaluation network to obtain a second quality score of the image to be evaluated when the image quality evaluation scenario is the second scenario.

[0008] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the program, implements the steps of the image quality assessment method as described in the first aspect.

[0009] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that, when the program is executed by a processor, it implements the steps of the image quality assessment method as described in the first aspect.

[0010] Fifthly, a computer program product is provided, characterized in that it includes a computer program / instructions, which, when executed by a processor, perform the steps of the image quality assessment method described in the first aspect.

[0011] The image quality assessment method provided in this application first determines whether the image quality assessment scenario belongs to a first scenario with a standard reference image or a second scenario without a standard reference image. Then, it selectively activates a pre-trained parametric quality assessment network or a non-parametric quality assessment network. This allows the same assessment framework to flexibly cope with different industrial quality inspection needs with and without reference images, without the need to build and train a separate model for each scenario. This solves the problem of the existing technology having a single assessment scenario and poor adaptability, and significantly improves the universality and deployment flexibility of the assessment scheme. Attached Figure Description

[0012] Figure 1 This is a flowchart of an image quality assessment method provided by an exemplary embodiment of this application; Figure 2 This is a schematic diagram of the structure of a parametric quality assessment network in the image quality assessment method provided by the exemplary embodiments of this application; Figure 3 This is a schematic diagram of the structure of the parameterless quality assessment network in the image quality assessment method provided by the exemplary embodiments of this application; Figure 4 This is a schematic diagram of the training process of the parametric quality assessment network in the image quality assessment method provided in the exemplary embodiment of the application; Figure 5 This is a schematic diagram of the parameterless quality assessment network training process in the image quality assessment method provided in the exemplary embodiment of the application; Figure 6 This is a schematic diagram of the structure of an image quality assessment device provided in an exemplary embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0013] The technical solutions in this application will now be described with reference to the accompanying drawings and specific embodiments. The described embodiments are some, but not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0014] The following is a description of the terms used in this application: Region of Interest (ROI): The ROI refers to the key area in the image to be evaluated that requires quality assessment. It typically corresponds to critical inspection points on industrial products, such as textured areas, target components, or areas prone to defects. In this embodiment, the ROI is marked with a mask to guide the parametric quality assessment network to focus on this area for precise quantification, thereby avoiding interference from background areas and improving the targeting and accuracy of the assessment.

[0015] Full-Reference Image Quality Assessment (FR-IQA): FR-IQA is a quality assessment method that requires a standard reference image for comparison. This method quantifies the differences between the image to be evaluated and a high-quality standard reference image to derive a quality score for the image to be evaluated. In this embodiment, the FR-IQA network utilizes a reference image and its region of interest mask to achieve accurate quality assessment of specific regions in a query image, making it suitable for standardized detection scenarios where a standard reference image exists.

[0016] No-Reference Image Quality Assessment (NR-IQA): NR-IQA is a quality assessment method that does not require a standard reference image. This method directly analyzes the image to be evaluated, extracting features such as structure, texture, and color, and predicts the overall quality score based on these features. In this embodiment, the NR-IQA network extracts features from multiple perspectives, including local texture, multi-scale structure, and global content, through multi-branch parallel processing, achieving robust quality assessment of a single image. This is suitable for complex industrial scenarios where no reference image exists.

[0017] Region of Interest (ROI) Mask: A ROI mask is a binary image used to mark the location and shape of a region of interest in a reference image. Regions with a pixel value of 1 represent the ROI, and regions with a pixel value of 0 represent the background. In this embodiment, the mask is used to crop template features from the reference feature map, serving as the basis for subsequent matching and localization, ensuring that the parametric quality assessment network can accurately locate the corresponding ROI in the query image.

[0018] Feature Extraction Network (Feat_Net): The feature extraction module is a deep feature extractor built on a convolutional neural network, used to convert the input image into a high-dimensional feature map. This module can extract semantic information, texture features, and structural information of the image, providing basic feature representations for subsequent matching, differencing, and regression tasks. In the embodiments of this application, the parametric quality assessment network uses lightweight MobileNetV2 or EfficientNet-B0 as the feature extraction module with shared weights to balance accuracy and computational efficiency.

[0019] The Region of Interest (ROI) Matching and Localization Module (ROI_Match) is used to locate regions on the query feature map that correspond to the features of the reference template. Borrowing ideas from the field of object tracking, this module first generates a response map through deep cross-correlation to achieve coarse localization, and then predicts the precise bounding box coordinates through a bounding box regression network. In this embodiment, this module achieves rapid localization from coarse to fine, ensuring that the parametric quality assessment network can accurately acquire the target region in the query map.

[0020] The Differential Feature Extraction Network (Diff_Net) module extracts the difference between template features and query region of interest features. By calculating the feature differences between the two, this module effectively captures the degree of quality degradation between the reference and query images in terms of texture, structure, and color. In this embodiment, the differential features serve as input to the quality regression module, providing crucial discriminative information for the final quality score prediction.

[0021] The Quality Regression Network (Regress_Net) is a regression network built on fully connected layers, used to map differential features to a final quality score. This module performs a non-linear transformation on the input features using a multilayer perceptron, outputting a continuous value as the predicted image quality. In this embodiment, the Quality Regression Network supports various branches of both parametric and non-parametric evaluation networks, achieving an end-to-end mapping from features to quality scores.

[0022] Multi-Scale Transformation (MST): MST is an image preprocessing technique that alters the resolution and scale information of an image by downsampling and then upsampling the original image. In this scheme, MST is used in the second branch of the nonparametric quality assessment network. By scaling the image, the network can capture structural information under different receptive fields, thereby enhancing its robustness to scale changes.

[0023] The Fusion Regression Network (Fusion_Regress_Net) module integrates the quality assessment vectors output from multiple parallel branches in the parameterless quality assessment network and outputs the final quality score. This module uses a multilayer perceptron to adaptively weight and fuse multiple vectors, dynamically adjusting the importance of each branch according to different distortion types. In this embodiment, the Fusion Regression Network enhances the robustness and generalization ability of the parameterless quality assessment network.

[0024] Mean Opinion Score (MOS): A MOS is an image quality evaluation metric obtained through human scoring, typically presented as a continuous value or discrete levels. This score reflects the human eye's perception of image quality and is widely used as a supervisory signal for image quality assessment models. In this embodiment, the MOS is used to construct the labels for the non-parametric evaluation dataset and as the true quality score for the corresponding region of the query image in the parametric evaluation dataset.

[0025] End-to-End Training: End-to-end training refers to jointly optimizing all modules of the entire network as a whole, without the need for pre-training or phased training of individual modules. In the embodiments of this application, both the parametric quality evaluation network and the non-parametric quality evaluation network are trained in an end-to-end manner. The weights of all network modules are updated simultaneously through gradient backpropagation of the loss function, simplifying the training process and improving the overall performance of the model.

[0026] As described in the background section, image quality assessment methods in related technologies are typically only adaptable to a single assessment scenario: either a standard reference image is required for parametric assessment, or only blind assessment without a reference image is supported. This makes it difficult to flexibly address the different needs in industrial quality inspection, whether or not a reference image exists, within the same framework. When the assessment scenario changes, it is often necessary to rebuild or retrain the model, resulting in long deployment cycles and poor adaptability, thus limiting the widespread application of image quality assessment technology in diverse industrial environments.

[0027] Based on this, the embodiments of this application propose an image quality assessment method. This method first determines whether the current image quality assessment scene belongs to a first scene with a standard reference map or a second scene without a standard reference map. Then, it selectively activates a pre-trained parametric quality assessment network or a parameterless quality assessment network. This allows for compatibility with both assessment modes with and without reference maps within a single framework. It eliminates the need to build and train models separately for different scenes, significantly improving the versatility and deployment flexibility of the assessment scheme.

[0028] The specific process of the image quality assessment method provided in this application embodiment is as follows: Figure 1 As shown, it includes the following steps S110 to S140.

[0029] S110, Obtain the image to be evaluated.

[0030] In some exemplary embodiments, before acquiring the image to be evaluated, the method provided in this application further includes: Construct the first training dataset. Each sample in the first training dataset includes a standard reference map, a region of interest mask labeled on the standard reference map, a query map containing distortion, and the ground truth bounding box and ground truth quality score of the corresponding region of interest in the query map. Using the first training dataset, the initial parametric quality assessment network is trained in an end-to-end manner. During the training process, the accuracy of the parametric quality assessment network on two tasks, region of interest localization and quality regression, is optimized by minimizing the first loss function. After training, the pre-trained parametric quality assessment network is obtained. Construct a second training dataset, in which each sample includes a distorted image and its corresponding subjective quality score; Using the second training dataset, the initial nonparametric quality assessment network is trained in an end-to-end manner. During the training process, the regression accuracy of the nonparametric quality assessment network is optimized by minimizing the second loss function. After training, the pre-trained nonparametric quality assessment network is obtained.

[0031] Here, the true bounding box refers to the actual location coordinates of the region of interest in the query graph, obtained through manual annotation or measurement using professional equipment, during the training of the parametric quality assessment network. This bounding box is represented by four parameters: the x-coordinate and y-coordinate of the top-left corner, and the width and height of the bounding box. In this embodiment, the true bounding box serves as a supervisory signal, used to train the region of interest matching and localization module, enabling it to accurately predict the corresponding target region in the query graph.

[0032] The true quality score refers to the quality evaluation standard value used as a supervisory signal during network training. For parametric evaluation tasks, the true quality score is the subjective quality score corresponding to the region of interest in the query image; for non-parametric evaluation tasks, the true quality score is the overall subjective quality score of the distorted image. In the embodiments of this application, the true quality score is used to calculate the loss function and guide the network to optimize its quality regression accuracy.

[0033] The first loss function is a combined loss function used to train the parametric quality assessment network, consisting of a weighted average of the localization loss and the quality regression loss. The localization loss measures the difference between the predicted bounding box output by the region of interest matching and localization module and the ground truth bounding box, while the quality regression loss measures the difference between the first quality score output by the parametric quality assessment network and the ground truth quality score. In this embodiment, by minimizing the first loss function, the performance of the parametric quality assessment network on both localization accuracy and regression accuracy tasks can be optimized simultaneously.

[0034] The second loss function is used to train the parameterless quality assessment network, and it is composed of a weighted average of the final output loss and the auxiliary loss. The final output loss measures the difference between the second quality score output by the fusion regression module and the true quality score, while the auxiliary loss measures the difference between the quality assessment vector output by each parallel branch and the true quality score. In this embodiment, by introducing the auxiliary loss, the stability of the parameterless quality assessment network training and the model's generalization ability can be enhanced.

[0035] In one specific embodiment of this application, before performing image quality assessment, it is necessary to pre-train both the parametric quality assessment network and the non-parametric quality assessment network.

[0036] For training the parametric quality assessment network, a first training dataset is first constructed. Each sample in the first training dataset includes a standard reference map, a mask of the region of interest (ROI) labeled on the standard reference map, a query map containing distortion, and the ground truth bounding box and ground truth quality score of the corresponding ROI in the query map. The ground truth quality score uses the Mean Opinion Score (MOS) as the evaluation metric.

[0037] In one specific embodiment of this application, the training data comes from two parts: one part consists of texture-type feature images and target-type feature images from industrial scenes, which require manual annotation to construct parametric image quality assessment task datasets and non-parametric image quality assessment task datasets; the other part consists of various publicly available image quality assessment databases, such as LIVE, CSIQ, and TID2013. All datasets are divided into training sets, validation sets, and test sets, where the training set is used for network parameter optimization, the validation set is used for model selection and hyperparameter tuning, and the test set is used to evaluate model performance.

[0038] Using the aforementioned first training dataset, the initial parametric quality assessment network was trained end-to-end. During training, the accuracy of the parametric quality assessment network on both region-of-interest (ROI) localization and quality regression tasks was optimized by minimizing a first loss function. The first loss function is a weighted average of the localization loss and the quality regression loss. The localization loss uses Smooth L1 Loss to calculate the difference between the predicted bounding box and the ground truth bounding box, while the quality regression loss uses Mean Squared Error (MSE) to calculate the difference between the predicted quality score and the ground truth quality score. After training, the pre-trained parametric quality assessment network was obtained.

[0039] In one specific embodiment of this application, the training hyperparameters of the parametric quality assessment network are set as follows: the total number of training steps is set to 100,000 epochs, the batch size is set according to hardware conditions, the optimizer is AdamW, the initial learning rate is set to 0.0001, a cosine annealing learning rate update strategy is adopted, and the network input size is adjusted to 640×640. The localization loss weight λ1 and the quality regression loss weight λ2 in the total loss function are set according to empirical values. During training, after each certain number of training epochs, the model performance is evaluated on the validation set, and the model with the best performance on the validation set is saved as the pre-trained parametric quality assessment network. The training data is a merged dataset of industrial datasets and public datasets such as LIVE, CSIQ, and TID2013.

[0040] Figure 2 This is a schematic diagram illustrating the training process of a parametric quality assessment network provided in an embodiment of this application. Figure 2As shown, during the training phase, the standard reference image I_ref and the image to be evaluated I_dist are input into the shared-weight feature extraction module Feat_Net to obtain the reference feature map F_ref and the query feature map F_dist, respectively. The template feature T_ref is cropped from the reference feature map F_ref based on the region of interest mask M_ref. The template feature T_ref and the query feature map F_dist are input into the region of interest matching and localization module ROI_Match to obtain the predicted region of interest bounding boxes. The query region of interest feature F_roi_dist is cropped from the query feature map F_dist based on the predicted bounding boxes. The template feature T_ref and the query region of interest feature F_roi_dist are input into the differential feature extraction module Diff_Net to obtain the differential feature D_diff. The differential feature D_diff is input into the quality regression module Regress_Net to output the predicted quality score Q_pred. During training, the localization loss L_loc is calculated based on the predicted bounding box and the ground truth bounding box, and the quality regression loss L_qa is calculated based on the predicted quality score and the ground truth quality score. The network parameters are optimized by minimizing the total loss of the weighted combination.

[0041] For training the parameterless quality assessment network, a second training dataset is first constructed. Each sample in the second training dataset includes a distorted image and its corresponding subjective quality score, which serves as the true quality score.

[0042] Using the second training dataset mentioned above, the initial non-parametric quality assessment network is trained end-to-end. During training, the regression accuracy of the non-parametric quality assessment network is optimized by minimizing a second loss function. The second loss function is a weighted sum of the final output loss and the auxiliary loss. The final output loss uses mean squared error to measure the difference between the second quality score output by the fused regression module and the true quality score; the auxiliary loss also uses mean squared error to measure the difference between the quality assessment vectors output by at least two parallel processing branches and the true quality scores. By setting the auxiliary loss, additional supervision signals can be provided to the intermediate layers of the network, enhancing the stability of training and the generalization ability of the model. After training, the pre-trained non-parametric quality assessment network is obtained.

[0043] In one specific embodiment of this application, the training hyperparameters of the parameterless quality assessment network are set as follows: the total number of training steps is set to 100,000 epochs, the batch size is set according to hardware conditions, the optimizer is Adam, the betas parameter is set to (0.9, 0.999), the initial learning rate is set to 0.001, and a Step LR learning rate decay strategy is adopted, with the learning rate decaying to 0.1 every 300 epochs. The network input baseline size is set to 384×384. During training, after a certain number of training epochs, the model performance is evaluated on the validation set, and the model with the best performance on the validation set is saved as the pre-trained parameterless quality assessment network. The training data is a merged dataset of industrial datasets and public datasets such as LIVE, CSIQ, and TID2013.

[0044] Figure 3 This is a schematic diagram illustrating the training process of a parameterless quality assessment network provided in an embodiment of this application. Figure 3 As shown, during the training phase, the first branch randomly crops the image I_dist to obtain a local image patch I_crop. I_crop is then input into the first feature extraction module Feat_Net1 to obtain local texture features F1, which are then processed by the first regression module Regress_Net1 to output the first quality evaluation vector Q1. The auxiliary loss L_branch1 of this branch is also calculated. The second branch performs multi-scale transformation on the image I_dist to obtain a multi-scale transformed image I_scale. I_scale is then input into the second feature extraction module Feat_Net2 to obtain multi-scale structural features F2, which are then processed by the second regression module Regress_Net2 to output the second quality evaluation vector Q2. The auxiliary loss L_branch2 of this branch is also calculated. The third branch directly inputs the image I_dist to be evaluated into the third feature extraction module Feat_Net3 to obtain global features F3, which are then processed by the third regression module Regress_Net3 to output the third quality evaluation vector Q3. The auxiliary loss L_branch3 of this branch is also calculated. The three quality evaluation vectors Q1, Q2, and Q3 are concatenated to obtain the fused feature F_fused. F_fused is then input into the fusion regression module Fusion_Regress_Net, which outputs the predicted quality score Q_pred and calculates the final output loss L_final. During training, the network parameters are optimized by minimizing the weighted sum of the final output loss and the auxiliary losses of each branch.

[0045] This application embodiment constructs a first training dataset including a reference map, mask, query map, ground truth bounding boxes, and ground truth quality scores. A first loss function is then used to train the parametric quality assessment network end-to-end, enabling the network to simultaneously optimize localization and regression accuracy during training. The localization loss forces the region of interest matching and localization module to accurately predict the location of the region of interest in the query map, while the quality regression loss forces the quality regression module to accurately predict the quality score based on differential features. The joint optimization of these two loss functions allows the network to focus its attention on the target region and accurately quantify its quality. Therefore, this training method significantly improves the evaluation accuracy of the parametric quality assessment network in standardized detection scenarios and its robustness to background interference.

[0046] In some exemplary embodiments, the second loss function is a weighted sum of the final output loss and the auxiliary loss. The final output loss is used to measure the difference between the second quality score output by the parameterless quality assessment network and the true quality score, and the auxiliary loss is used to measure the difference between the quality assessment vectors output by at least two parallel processing branches and the true quality scores.

[0047] The final output loss is a loss term used during the training of the parameterless quality assessment network to measure the difference between the network's final prediction and the true quality score. This loss is calculated using mean squared error and reflects the deviation between the second quality score output by the fusion regression module and the true quality score. In this embodiment, the final output loss is a major component of the second loss function, directly driving the overall network output to approximate the true quality score.

[0048] The auxiliary loss is an additional supervisory signal introduced during the training of a parameterless quality assessment network. It measures the difference between the quality assessment vector output by each parallel branch and the true quality score. This loss is also calculated using mean squared error and can provide direct gradient feedback to the intermediate layers of the network. In the embodiments of this application, by setting the auxiliary loss, the gradient vanishing problem in deep network training can be alleviated, improving the convergence speed and generalization performance of the model.

[0049] In a specific embodiment of this application, the second loss function is configured as follows: Let the final output loss be L_final, and the auxiliary losses of each branch be L_branch1, L_branch2, and L_branch3, respectively. Then the second loss function L_total is expressed as: L_total = L_final + α × (L_branch1 + L_branch2 + L_branch3).

[0050] Here, α is a balancing coefficient used to adjust the weight of the auxiliary loss in the total loss. With the above configuration, the parameterless quality assessment network can optimize the final fusion output during training, while ensuring that each parallel branch independently learns effective quality assessment capabilities, thereby improving the robustness of the model in multi-distortion scenarios.

[0051] This application embodiment constructs a second training dataset containing distorted images and their subjective quality scores, and uses a second loss function to train the non-parametric quality assessment network end-to-end. This allows the non-parametric quality assessment network to simultaneously optimize the final fused output and the outputs of each branch during training. The final output loss drives the fusion regression module to learn how to adaptively and weightedly fuse the evaluation results of multiple branches, while the auxiliary loss provides direct supervision to each parallel branch, forcing each branch to independently learn quality features from different perspectives, such as local texture, multi-scale structure, and global content. Through the weighted joint optimization of the final output loss and the auxiliary loss, the stability of network training and the generalization ability of the model can be enhanced. This enables the trained non-parametric quality assessment network to make robust and accurate overall quality assessments of single images in complex industrial scenarios without the need for a reference image.

[0052] S120, determine the image quality assessment scenario.

[0053] Image quality assessment scenarios include a first scenario with a standard reference image and a second scenario without a standard reference image.

[0054] The image quality assessment scenario refers to the application environment conditions during image quality assessment, specifically including the availability of standard reference images. In this embodiment, the image quality assessment scenarios are divided into two categories: a first scenario and a second scenario. Different scenarios correspond to different assessment strategies and network branches, and the system dynamically selects and activates the corresponding assessment path based on the scenario type.

[0055] The first scenario refers to an image quality assessment scenario where a standard reference image exists. In this scenario, the image to be evaluated can be compared with a high-quality standard reference image, thereby achieving accurate quality assessment of a specific region of interest. In the embodiments of this application, the first scenario corresponds to activating the parametric quality assessment network, which requires inputting the image to be evaluated, the standard reference image, and its region of interest mask.

[0056] The second scenario refers to an image quality assessment scenario where there is no standard reference image. In this scenario, only the image to be evaluated exists, and quality cannot be assessed by comparing it with a reference image. In the embodiments of this application, the second scenario corresponds to activating a parameterless quality assessment network, where only the image to be evaluated needs to be input to obtain the overall quality score.

[0057] In one specific embodiment of this application, the determination of an image quality assessment scenario can be achieved as follows: the system detects whether the user input contains a standard reference image field. If the user provides both the image to be evaluated and the standard reference image, it is automatically identified as the first scenario; if the user only provides the image to be evaluated without providing the standard reference image, it is automatically identified as the second scenario. The above determination method is simple and efficient, requires no complex configuration, and is easy to deploy quickly in industrial production lines.

[0058] S130, if the image quality assessment scenario is the first scenario, then activate the pre-trained parametric quality assessment network, and input the image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image into the parametric quality assessment network to obtain the first quality score of the image to be evaluated on the region of interest.

[0059] In some exemplary embodiments, the parametric quality assessment network includes: a feature extraction module, a region of interest matching and localization module, a differential feature extraction module, and a quality regression module; The image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image are input into a parametric quality evaluation network to obtain the first quality score of the image to be evaluated on the region of interest, including: The feature extraction module extracts features from the standard reference image and the image to be evaluated, respectively, to obtain the reference feature map of the standard reference image and the query feature map of the image to be evaluated. Based on the region of interest mask, the template features are cropped from the reference feature map; The region of interest matching and localization module performs matching and localization on the query feature map based on template features to obtain the predicted region of interest bounding box. Based on the predicted region of interest bounding box, the query region of interest features are cropped from the query feature map; The differential feature extraction module extracts the differential features between the template features and the features of the query region of interest. The first quality score is obtained through the quality regression module based on differential feature regression.

[0060] The reference feature map refers to the depth feature representation output by the feature extraction module after inputting a standard reference map. This feature map retains the semantic, texture, and structural information of the reference map and is presented in the form of a two-dimensional feature map. In this embodiment, the reference feature map serves as the source of template features and is used for matching and differential comparison with the query feature map.

[0061] The query feature map refers to the depth feature representation output by the feature extraction module after the image to be evaluated is input. This feature map and the reference feature map are generated by the feature extraction module with shared weights, ensuring that they are in the same feature space, which facilitates subsequent matching and differencing operations. In the embodiments of this application, the query feature map serves as the search space for the region of interest matching and localization module.

[0062] Template features refer to local feature regions cropped from a reference feature map based on a region of interest mask. These features correspond to the regions of interest marked on the standard reference map and serve as a baseline template for subsequent matching and localization. In this embodiment, the template features are used to locate the corresponding target region on the query feature map.

[0063] The query region of interest (ROI) feature refers to a local feature region cropped from the query feature map based on the predicted ROI bounding box. This feature corresponds to the target region in the image to be evaluated that matches the ROI of the reference map. In this embodiment, the query ROI feature and the template feature are jointly input into the differential feature extraction module to calculate the difference between them.

[0064] Figure 4 This is a schematic diagram of the structure of a parametric quality assessment network provided in an embodiment of this application. Figure 4 As shown, the parametric quality assessment network includes a shared-weight feature extraction module (Feat_Net), a region of interest (ROI) matching and localization module (ROI_Match), a differential feature extraction module (Diff_Net), and a quality regression module (Regress_Net). During the inference phase, the standard reference image (I_ref) and the image to be evaluated (I_dist) are input into the shared-weight feature extraction module to obtain a reference feature image (F_ref) and a query feature image (F_dist). A template feature (T_ref) is cropped from the reference feature image (F_ref) based on the region of interest mask (M_ref). The template feature (T_ref) and the query feature image (F_dist) are input into the ROI matching and localization module (ROI_Match) to obtain the predicted region of interest bounding box, and the query region of interest feature (F_roi_dist) is cropped from the query feature image (F_dist) based on this bounding box. The template feature (T_ref) and the query region of interest feature (F_roi_dist) are input into the differential feature extraction module (Diff_Net) to obtain the differential feature (D_diff). The differential feature (D_diff) is input into the quality regression module (Regress_Net), and finally, the first quality score (Q_pred) is output.

[0065] In one specific embodiment of this application, the parametric quality assessment network includes a shared weight feature extraction module, a region of interest matching and localization module, a differential feature extraction module, and a quality regression module.

[0066] When the image quality assessment scenario is the first scenario, the system activates the pre-trained parametric quality assessment network. The image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image are input into the network, and the network performs the following operations to output a first quality score.

[0067] First, a shared-weighted feature extraction module is used to extract deep features from both the standard reference image and the image to be evaluated. This module employs a lightweight convolutional neural network design, such as MobileNetV2 or EfficientNet-B0, which efficiently extracts high-level semantic features from the images. After feature extraction, a reference feature map of the standard reference image and a query feature map of the image to be evaluated are obtained.

[0068] Then, template features are cropped from the reference feature map based on the input region of interest mask. Regions with a pixel value of 1 in the mask correspond to the region of interest, and the corresponding local feature regions are extracted from the reference feature map based on this mask as template features.

[0069] Next, the template features and the query feature map are input into the region of interest matching and localization module. This module draws inspiration from object tracking, first generating a response map through deep cross-correlation to achieve coarse localization. This involves calculating the similarity between the template features and the query feature map at various locations, with the location having the highest response value being the coarse localization result. Then, a bounding box regression network refines the coarse localization result, predicting the precise bounding box in the query feature map corresponding to the template features, which serves as the predicted region of interest bounding box. Deep cross-correlation is an operation used to calculate the similarity between two feature maps, commonly used in object tracking and feature matching tasks. This operation generates a response map by sliding convolution on the query feature map using the template features as the convolution kernel; higher response values ​​indicate a higher degree of matching with the template features. In this embodiment, deep cross-correlation is used to achieve coarse localization of the region of interest. The bounding box regression network is a sub-network used to predict the precise coordinates of the target location. Based on the coarse localization result, this network further regresses the precise coordinate parameters of the target bounding box, including the x-coordinate and y-coordinate of the top-left corner of the bounding box, as well as its width and height. In this embodiment, the bounding box regression network and the deep cross-correlation operation work together to achieve rapid localization from coarse to fine.

[0070] Then, based on the predicted region of interest bounding box, the query region of interest features are cropped from the query feature map. These features correspond to the target region in the image to be evaluated that matches the region of interest in the reference map.

[0071] Next, the template features and the query region of interest features are input into the differential feature extraction module to extract the difference features between the two. The differential features can effectively capture the degree of quality degradation between the reference image and the query image in terms of texture, structure, and color.

[0072] Finally, the differential features are input into the quality regression module, where a multilayer perceptron performs a nonlinear transformation on the differential features to obtain the first quality score. This quality score is the accurate quality assessment result of the image to be evaluated in the region of interest.

[0073] S140, if the image quality assessment scenario is the second scenario, then activate the pre-trained parameterless quality assessment network and input the image to be assessed into the parameterless quality assessment network to obtain the second quality score of the image to be assessed.

[0074] In some exemplary embodiments, the nonparametric quality assessment network includes: at least two parallel processing branches and a fusion regression module; The image to be evaluated is input into a parameterless quality assessment network to obtain a second quality score for the image, including: The image to be evaluated is input into at least two parallel processing branches to obtain at least two quality evaluation vectors. Each processing branch is used to extract image features from different perspectives and output the corresponding quality evaluation vector. The fusion regression module is used to fuse at least two quality assessment vectors to obtain a second quality score.

[0075] In this context, parallel processing branches refer to multiple independent subnetworks running simultaneously within the parameterless quality assessment network. Each branch performs different preprocessing and feature extraction operations on the input image, capturing image quality features from different perspectives. The branches operate independently and in parallel, ultimately integrating their respective output quality assessment vectors through a fusion regression module. In this embodiment, three parallel processing branches are set up, focusing on local texture details, multi-scale structural information, and global content information, respectively.

[0076] Figure 5 This is a schematic diagram of the structure of the parameterless quality assessment network provided in an embodiment of this application. Figure 5As shown, the parameter-free quality assessment network comprises three parallel processing branches and a fusion regression module, Fusion_Regress_Net. The first branch randomly crops the image I_dist to obtain a local image patch I_crop. I_crop is then input into the first feature extraction module, Feature_Net1, to obtain local texture features F1, which are then processed by the first regression module, Regress_Net1, to output the first quality assessment vector Q1. The second branch performs multi-scale transformation on the image I_dist to obtain a multi-scale transformed image I_scale. I_scale is then input into the second feature extraction module, Feature_Net2, to obtain multi-scale structural features F2, which are then processed by the second regression module, Regress_Net2, to output the second quality assessment vector Q2. The third branch directly inputs the image I_dist to be evaluated into the third feature extraction module, Feature_Net3, to obtain global features F3, which are then processed by the third regression module, Regress_Net3, to output the third quality assessment vector Q3. The three quality assessment vectors Q1, Q2, and Q3 are concatenated to obtain the fusion feature F_fused. F_fused is then input into the fusion regression module, Fusion_Regress_Net, to finally output the second quality score Q_pred.

[0077] In some exemplary embodiments, at least two parallel processing branches include: The first branch, which focuses on local texture details, is configured to: randomly crop the image to be evaluated to obtain local image patches; input the local image patches into the first feature extraction module to obtain local texture features; and input the local texture features into the first regression module to output the first quality evaluation vector. The second branch focuses on multi-scale structural information and is configured to: perform multi-scale transformation on the image to be evaluated to obtain a multi-scale transformed image; input the multi-scale transformed image into the second feature extraction module to obtain multi-scale structural features; input the multi-scale structural features into the second regression module to output the second quality evaluation vector; The third branch focuses on global content information and is configured to: input the image to be evaluated into the third feature extraction module to obtain global features; input the global features into the third regression module to output the third quality evaluation vector.

[0078] In this context, a local image patch refers to a sub-region of an image obtained from the image to be evaluated through random cropping. This sub-region retains the local texture information and detail features of the original image, and can reflect the quality degradation of the image at a microscopic scale. In this embodiment, the local image patch serves as the input to the first branch for extracting local texture features.

[0079] A multi-scale transformed image refers to an image obtained by downsampling and then upsampling the image to restore it to its original size. This transformation operation can change the scale information of the image, enabling the network to capture structural features under different receptive fields and enhancing its robustness to scale changes. In the embodiments of this application, the multi-scale transformed image is used as the input of the second branch to extract multi-scale structural features.

[0080] The quality assessment vector refers to the preliminary quality assessment result output by each parallel processing branch, typically in one-dimensional vector form. This vector reflects the current branch's judgment of image quality from a specific perspective and serves as the input feature for the fusion regression module. In this embodiment, the three parallel processing branches output a first quality assessment vector, a second quality assessment vector, and a third quality assessment vector, respectively.

[0081] The first feature extraction module is a feature extraction network for processing local image patches. It is based on a lightweight convolutional neural network design and focuses on efficiently extracting local texture features from image patches. In this embodiment, the first feature extraction module employs lightweight network structures such as MobileNetV2 to ensure computational efficiency while effectively capturing texture details.

[0082] The second feature extraction module is a feature extraction network for processing multi-scale transformed images. Borrowing from the idea of ​​a multi-scale feature pyramid, it captures structural information under different receptive fields through dilated convolution. In this embodiment, the second feature extraction module is specifically used to evaluate the integrity of the image structure.

[0083] The third feature extraction module is a feature extraction network used to process the original full image. It employs a deeper network structure to obtain global semantic and color consistency information. In this embodiment, the third feature extraction module uses a deeper network architecture such as EfficientNet-B0, which can effectively extract global content features of the image.

[0084] In one specific embodiment of this application, the first feature extraction module of the first branch is designed based on the lightweight convolutional neural network MobileNetV2, which focuses on efficiently extracting local texture features from image patches; the first regression module is responsible for evaluating the severity of distortion artifacts and mapping the local texture features into a first quality evaluation vector that reflects the quality degradation of the local region.

[0085] In one specific embodiment of this application, the second feature extraction module of the second branch draws on the idea of ​​multi-scale feature pyramids and captures structural information under different receptive fields through dilated convolution, which can expand the receptive field without increasing the number of parameters; the second regression module is specifically used to evaluate the integrity of the image structure and maps the multi-scale structural features into a second quality evaluation vector that reflects the degree of structural degradation.

[0086] In one specific embodiment of this application, the third feature extraction module of the third branch adopts a deep convolutional neural network EfficientNet-B0, which can extract global semantic information and color consistency features of the image; the third regression module makes a preliminary judgment on the overall naturalness of the image and maps the global features into a third quality evaluation vector that reflects the overall visual quality.

[0087] The first, second, and third regression modules are quality regression networks corresponding to three parallel branches, used to map the features extracted by each branch into preliminary quality assessment vectors. Each regression module independently learns the mapping relationship from features to quality scores, providing diverse evaluation perspectives for the final fusion. In this embodiment, the first regression module assesses the severity of distortion artifacts, the second regression module assesses the integrity of the image structure, and the third regression module makes a preliminary judgment on the overall naturalness of the image.

[0088] In one specific embodiment of this application, the parameterless quality assessment network includes three parallel processing branches and a fusion regression module. The three processing branches focus on local texture details, multi-scale structural information, and global content information, respectively. Each branch runs in parallel without interfering with the others, and finally, the output quality assessment vector is fed into the fusion regression module for integration.

[0089] When the image quality assessment scenario is the second scenario, the system activates the pre-trained parameterless quality assessment network, inputs only the image to be evaluated into the network, and the network performs the following operations to output the second quality score.

[0090] The first branch focuses on local texture details. First, a random cropping operation is performed on the image to be evaluated, cropping local image patches from the original image. For example, if the image size is 384×384, a local image patch of size 224×224 is randomly cropped. Then, the local image patch is input into the first feature extraction module, which uses a lightweight convolutional neural network, MobileNetV2, to focus on extracting local texture features from the image patch. Finally, the local texture features are input into the first regression module, which maps the local texture features into a first quality evaluation vector through a fully connected layer. This vector reflects the severity of distortion artifacts in the local region.

[0091] This embodiment of the application sets up a first branch that randomly crops the image to be evaluated and extracts local texture features, enabling the network to focus on the microscopic details of the image. The random cropping operation allows the network to access different local areas of the image during training, enhancing its ability to perceive texture distortion artifacts; the lightweight first feature extraction module ensures computational efficiency, enabling the network to efficiently capture local texture changes. Through these techniques, the first branch can accurately assess the degree of quality degradation of the image at the texture detail level, such as scratches, blemishes, and texture blurring.

[0092] The second branch focuses on multi-scale structural information. First, the image to be evaluated undergoes a multi-scale transformation, for example, downsampling the original 384×384 image to 192×192 and then upsampling it back to 384×384, resulting in a multi-scale transformed image. This operation alters the image's scale information, enabling the network to capture structural features within different receptive fields. Then, the multi-scale transformed image is input into a second feature extraction module. This module, drawing inspiration from multi-scale feature pyramids, uses dilated convolution to capture structural information within different receptive fields, obtaining multi-scale structural features. Finally, these multi-scale structural features are input into a second regression module, which maps them to a second quality evaluation vector, reflecting the integrity of the image's structure.

[0093] This application embodiment sets up a second branch to perform multi-scale transformations on the image to be evaluated and extract multi-scale structural features, enabling the network to capture structural information at different receptive fields. The multi-scale transformation operation simulates the observation effect of the image at different distances, enhancing the network's robustness to scale changes; the feature extraction module using dilated convolution can expand the receptive field without increasing the number of parameters, effectively capturing the structural integrity information of the image. Through the above technical means, the second branch can accurately assess the degree of quality degradation of the image at the structural level, such as defects like deformation, misalignment, and structural fracture.

[0094] The third branch focuses on global content information. First, the image to be evaluated is directly input without any cropping or scaling. Then, the image is fed into the third feature extraction module, which uses a deep convolutional neural network, EfficientNet-B0, to extract global semantic information and color consistency features, resulting in global features. Finally, these global features are input into the third regression module, which maps them to a third quality evaluation vector, reflecting the overall naturalness of the image.

[0095] This embodiment of the application sets up a third branch to directly process the original full image and extract global features, enabling the network to grasp the semantic content and color consistency of the image from a holistic perspective. The deeper third feature extraction module can extract high-level semantic features, effectively perceiving problems such as global color deviation, uneven illumination, and overall blur. Through the above technical means, the third branch can accurately assess the degree of quality degradation of the image in terms of overall naturalness and color consistency.

[0096] The first, second, and third quality assessment vectors output from the three parallel branches are concatenated along the feature dimension to obtain the fused feature. This fused feature is then input into a fusion regression module, which uses a multilayer perceptron to adaptively weight and fuse the three quality assessment vectors, learning the importance of each branch under different distortion types, and finally outputting a second quality score. This score represents the overall quality assessment result of the image to be evaluated.

[0097] This application embodiment achieves intelligent integration of evaluation results from different perspectives by setting up a fusion regression module to adaptively weight and fuse the quality evaluation vectors output by three parallel branches. The fusion regression module adopts a multilayer perceptron structure, which can learn the importance weights of each branch under different distortion types during training and dynamically adjust the fusion strategy for different quality degradation modes. Compared with simple averaging or fixed-weight fusion methods, adaptive weighted fusion can give full play to the advantages of each branch and make up for the shortcomings of a single branch, thereby making more robust and accurate overall quality decisions.

[0098] This application's embodiments construct a parameterless quality assessment network comprising three parallel processing branches and a fusion regression module, and employ an end-to-end approach for training and inference. This enables the network to comprehensively evaluate image quality from multiple perspectives, including local texture, multi-scale structure, and global content. The three branches model different characteristics of the image, independently extracting features and outputting preliminary evaluation results. Finally, the fusion regression module performs adaptive weighted fusion. This multi-view fusion design allows the network to more comprehensively perceive various image distortions, avoiding the limitations of single-view evaluation, thereby significantly improving the robustness and accuracy of parameterless quality assessment.

[0099] The image quality assessment method provided in this application first determines whether the image quality assessment scenario belongs to a first scenario with a standard reference image or a second scenario without a standard reference image. Then, it selectively activates a pre-trained parametric quality assessment network or a non-parametric quality assessment network. This allows the same assessment framework to flexibly cope with different industrial quality inspection needs with and without reference images, without the need to build and train a separate model for each scenario. This solves the problem of the existing technology having a single assessment scenario and poor adaptability, and significantly improves the universality and deployment flexibility of the assessment scheme.

[0100] Figure 6 This is a schematic diagram of the structure of an image quality assessment device 600 provided in an exemplary embodiment of this application. Figure 6 As shown, the image quality assessment device 600 includes: an image acquisition module 610, a scene judgment module 620, a parametric assessment module 630, and a parameterless assessment module 640, wherein: Image acquisition module 610 is used to acquire the image to be evaluated; The scene judgment module 620 is used to judge the image quality assessment scene, the scene including a first scene with a standard reference image and a second scene without a standard reference image; The parametric evaluation module 630 includes a pre-trained parametric quality evaluation network, which is used to input the image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image into the parametric quality evaluation network when the image quality evaluation scenario is the first scenario, so as to obtain a first quality score of the image to be evaluated on the region of interest. The parameterless evaluation module 640 includes a pre-trained parameterless quality evaluation network, used to input the image to be evaluated into the parameterless quality evaluation network when the image quality evaluation scenario is the second scenario, and obtain a second quality score of the image to be evaluated.

[0101] The image quality assessment device provided in this application first determines whether the image quality assessment scenario belongs to a first scenario with a standard reference image or a second scenario without a standard reference image, and then activates the pre-trained parametric quality assessment network or the non-parametric quality assessment network accordingly. This allows the same assessment framework to flexibly cope with different industrial quality inspection needs with and without a reference image, without the need to build and train a model separately for each scenario. This solves the problem of the existing technology having a single assessment scenario and poor adaptability, and significantly improves the universality and deployment flexibility of the assessment scheme.

[0102] Optionally, the parametric quality assessment network includes: a feature extraction module, a region of interest matching and localization module, a differential feature extraction module, and a quality regression module; When the parametric evaluation module 630 inputs the image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image into the parametric quality evaluation network to obtain the first quality score of the image to be evaluated on the region of interest, it is specifically used for: The feature extraction module extracts features from the standard reference image and the image to be evaluated, respectively, to obtain a reference feature map of the standard reference image and a query feature map of the image to be evaluated. Based on the region of interest mask, the template features are cropped from the reference feature map; The region of interest matching and localization module matches and locates the template features on the query feature map to obtain the predicted region of interest bounding box. Based on the predicted region of interest bounding box, the query region of interest features are cropped from the query feature map; The differential feature extraction module extracts the differential features between the template features and the query region of interest features. The first quality score is obtained through the quality regression module based on the differential feature regression.

[0103] Optionally, the parameterless quality assessment network includes: at least two parallel processing branches and a fusion regression module; When the parameterless evaluation module 640 inputs the image to be evaluated into the parameterless quality evaluation network to obtain the second quality score of the image to be evaluated, it is specifically used for: The image to be evaluated is input into the at least two parallel processing branches to obtain at least two quality evaluation vectors. Each processing branch is used to extract image features from different perspectives and output the corresponding quality evaluation vector. The fusion regression module is used to fuse the at least two quality assessment vectors to obtain the second quality score.

[0104] Optionally, the at least two parallel processing branches include: The first branch is used to focus on local texture details and is configured to: randomly crop the image to be evaluated to obtain local image patches; input the local image patches into the first feature extraction module to obtain local texture features; input the local texture features into the first regression module to output the first quality evaluation vector; The second branch is used to focus on multi-scale structural information and is configured to: perform multi-scale transformation on the image to be evaluated to obtain a multi-scale transformed image; input the multi-scale transformed image into a second feature extraction module to obtain multi-scale structural features; input the multi-scale structural features into a second regression module to output a second quality evaluation vector; The third branch is used to focus on global content information and is configured to: input the image to be evaluated into the third feature extraction module to obtain global features; input the global features into the third regression module to output the third quality evaluation vector.

[0105] Optionally, before acquiring the image to be evaluated, the device further includes a training module for: Construct a first training dataset. Each sample in the first training dataset includes a standard reference map, a region of interest mask labeled on the standard reference map, a query map containing distortion, and the ground truth bounding box and ground truth quality score of the corresponding region of interest in the query map. Using the first training dataset, the initial parametric quality assessment network is trained in an end-to-end manner. During the training process, the accuracy of the parametric quality assessment network on the two tasks of region of interest localization and quality regression is optimized by minimizing the first loss function. After training, the pre-trained parametric quality assessment network is obtained. Construct a second training dataset, in which each sample includes a distorted image and its corresponding subjective quality score; Using the second training dataset, the initial nonparametric quality assessment network is trained in an end-to-end manner. During the training process, the regression accuracy of the nonparametric quality assessment network is optimized by minimizing the second loss function. After training, the pre-trained nonparametric quality assessment network is obtained.

[0106] Optionally, the second loss function is a weighted sum of the final output loss and the auxiliary loss, wherein the final output loss is used to measure the difference between the second quality score output by the parameterless quality assessment network and the true quality score, and the auxiliary loss is used to measure the difference between the quality assessment vector output by each of the at least two parallel processing branches and the true quality score.

[0107] The image quality assessment device 600 can achieve Figures 1-5 For details of the method implementation examples, please refer to [link / reference]. Figures 1-5 The image quality assessment method of the illustrated embodiment will not be described in detail again.

[0108] Figure 7 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. For example... Figure 7 As shown, the device includes a memory 71 and a processor 72.

[0109] Memory 71 is used to store computer programs and can be configured to store various other data to support operation on the computing device. Examples of this data include instructions for any application or method used to operate on the computing device, contact data, phone book data, messages, images, videos, etc.

[0110] The processor 72, coupled to the memory 71, is used to execute a computer program in the memory 71 for: acquiring an image to be evaluated; determining an image quality evaluation scenario, the image quality evaluation scenario including a first scenario with a standard reference image and a second scenario without a standard reference image; if the image quality evaluation scenario is the first scenario, then activating a pre-trained parametric quality evaluation network, and inputting the image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image into the parametric quality evaluation network to obtain a first quality score of the image to be evaluated on the region of interest; if the image quality evaluation scenario is the second scenario, then activating a pre-trained non-parametric quality evaluation network, and inputting the image to be evaluated into the non-parametric quality evaluation network to obtain a second quality score of the image to be evaluated.

[0111] The electronic device provided in this application first determines whether the image quality assessment scenario belongs to a first scenario with a standard reference image or a second scenario without a standard reference image, and then activates a pre-trained parametric quality assessment network or a parameterless quality assessment network accordingly. This allows the same assessment framework to flexibly cope with different industrial quality inspection needs with or without a reference image, without the need to build and train a model separately for each scenario. This solves the problem of the existing technology having a single assessment scenario and poor adaptability, and significantly improves the universality and deployment flexibility of the assessment scheme.

[0112] Furthermore, such as Figure 7 As shown, the electronic device also includes other components such as a communication component 73, a display 74, a power supply component 75, and an audio component 76. Figure 7 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 7 The components shown. Additionally, depending on the implementation of the traffic playback device, Figure 7 The components within the dashed box are optional, not mandatory. For example, when an electronic device is implemented as a terminal device such as a smartphone, tablet, or desktop computer, it may include... Figure 7 The components within the dashed box; when the electronic device is implemented as a server-side device such as a conventional server, cloud server, data center, or server array, it may be excluded. Figure 7 The component within the dashed box.

[0113] The above Figure 7The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component may further include a Near Field Communication (NFC) module, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, etc.

[0114] The above Figure 7 The memory in the memory can be implemented by any class of volatile or non-volatile storage devices or combinations thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0115] The above Figure 7 The display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0116] The above Figure 7 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0117] The above Figure 7 The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0118] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments.

[0119] Accordingly, this application also provides a computer program product, which stores instructions that, when executed by a computer, cause the computer to perform the steps in the method embodiments provided in this application.

[0120] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0125] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] It should be understood that the training and prediction processes of the AI ​​models involved in the various embodiments of this specification all adhere to multiple legal and compliant principles, including legal data sources, compliant data content, compliant data governance, compliant training objectives and schemes, compliant training processes, compliant training environments and tools, and compliant ethical verification of training results, and comply with the requirements of Article 5 of the Patent Law. Among them: Data source legitimacy: All datasets used for AI model training were obtained through legal means, covering three categories: publicly authorized data, data authorized by partners, and self-collected compliant data. Publicly authorized data comes from compliant data sources following open-source licenses such as Apache 2.0, with complete copyright attribution and authorization scope clearly marked, and no unauthorized open-source code or data reuse. Data authorized by partners has been subject to formal data usage agreements, clearly defining the scope, duration, and confidentiality obligations, and possessing a complete authorization chain. For self-collected data involving personal information, strict informed consent procedures have been followed, and anonymization processes (including but not limited to field masking, feature anonymization, and differential privacy technology applications) have been implemented to remove personally identifiable information, fully complying with the requirements of relevant laws and regulations such as the "Interim Measures for the Administration of Generative Artificial Intelligence Services" and the "Personal Information Protection Law."

[0127] Data content compliance: The AI ​​model's dataset undergoes multiple screenings and cleaning processes to remove all content that may violate social morality or harm public interests, and also excludes any instances of illegal acquisition or use of genetic resources. For data in sensitive fields (such as healthcare and finance), an additional privacy-preserving computation module (including federated learning and secure multi-party computation technologies) ensures that the data is "usable but not visible," avoiding compliance risks during the original data transmission process and ensuring that the data application scenarios and uses comply with public order and good morals and industry regulatory requirements.

[0128] Data governance norms: A complete data traceability system is established during the AI ​​model training process to automatically record the source, collection time, annotation process, cleaning rules, and permission allocation of training data, generating traceable compliance reports to ensure that the data is verifiable throughout its entire lifecycle. The dataset annotation process for AI models is completed by a professional human R&D team, clearly defining the proportion of human creative contributions and avoiding reliance on AI-generated data that has not undergone substantial human modification, thus meeting the examination requirements for "human main contributions" in AI patent applications.

[0129] Training objectives and plans are compliant: The AI ​​model training objectives focus on defect detection, image quality assessment, etc. The training scheme and the final output results do not violate any mandatory provisions of laws and administrative regulations, do not harm the public interest or the legitimate rights and interests of others, and do not pose any potential risks of being used for illegal activities, privacy infringement, or public safety disruption. It strictly adheres to the ethical principle of "intelligent for good".

[0130] Training process compliance: A closed-loop training framework is adopted to ensure compliance and controllability of the training process. The specific process is as follows: First, training samples are obtained through compliant data sources. After the aforementioned data cleaning and desensitization, they are input into the neural network model to generate preliminary training results. Second, an expert system is introduced to verify the preliminary results. Based on preset rules and human expert experience, the feasibility of the results is evaluated, and outputs that may pose ethical risks or compliance hazards are corrected (such as removing decision-making logic that violates public order and good morals, and adjusting model parameters that do not comply with safety regulations). Finally, the loss function weights are dynamically optimized based on expert system feedback to strengthen the model's learning of compliant results, avoid overfitting errors or non-compliant labels, and form a closed-loop control of "data input - model training - expert verification - parameter optimization - result feedback" to ensure that the entire training process complies with A5 ethical review requirements.

[0131] Training environment and tool compliance: AI model training is implemented using nationally licensed chips and a compliant training platform. All open-source frameworks and components used in the training process have obtained their corresponding licenses, and copyright statements and patent citation information are fully retained, with no instances of infringement or reuse. The training environment is built using virtual devices (containers / virtual machines) with fixed random seeds and initial parameter configurations to ensure the reproducibility of the training process. Furthermore, through access control and operation log recording, risks such as data leakage and parameter tampering during training are prevented, ensuring the security and compliance of the training process.

[0132] Training results ethical verification compliance: After the model is trained, it undergoes additional third-party ethical compliance assessment and algorithm filing review to verify that the model output does not violate social morality or harm public interests. For potentially sensitive scenarios (such as public services and intelligent decision-making), a special result verification mechanism is established to ensure that the model always complies with Article 5 of the Patent Law and relevant laws and regulations in practical applications.

[0133] In summary, the data and training process used in the AI ​​model of this specification strictly comply with the relevant provisions of Article 5 of the Patent Law and the Patent Examination Guidelines (2023 Edition), and there are no violations of laws, social ethics, public interests, or illegal use of genetic resources. It fully meets the compliance requirements for patent authorization.

[0134] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image quality assessment method, characterized in that, include: Acquire the image to be evaluated; The image quality assessment scenario is determined, which includes a first scenario where a standard reference image exists and a second scenario where a standard reference image does not exist. If the image quality assessment scenario is the first scenario, then the pre-trained parametric quality assessment network is activated, and the image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image are input into the parametric quality assessment network to obtain the first quality score of the image to be evaluated on the region of interest. If the image quality assessment scenario is the second scenario, then the pre-trained parameterless quality assessment network is activated, and the image to be assessed is input into the parameterless quality assessment network to obtain the second quality score of the image to be assessed.

2. The method according to claim 1, characterized in that, The parametric quality assessment network includes: a feature extraction module, a region of interest matching and localization module, a differential feature extraction module, and a quality regression module; The step of inputting the image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image into the parametric quality assessment network to obtain a first quality score of the image to be evaluated on the region of interest includes: The feature extraction module extracts features from the standard reference image and the image to be evaluated, respectively, to obtain a reference feature map of the standard reference image and a query feature map of the image to be evaluated. Based on the region of interest mask, the template features are cropped from the reference feature map; The region of interest matching and localization module matches and locates the template features on the query feature map to obtain the predicted region of interest bounding box. Based on the predicted region of interest bounding box, the query region of interest features are cropped from the query feature map; The differential feature extraction module extracts the differential features between the template features and the query region of interest features. The first quality score is obtained through the quality regression module based on the differential feature regression.

3. The method according to claim 1, characterized in that, The parameterless quality assessment network includes at least two parallel processing branches and a fusion regression module; The step of inputting the image to be evaluated into the parameterless quality assessment network to obtain a second quality score for the image to be evaluated includes: The image to be evaluated is input into the at least two parallel processing branches to obtain at least two quality evaluation vectors. Each processing branch is used to extract image features from different perspectives and output the corresponding quality evaluation vector. The fusion regression module is used to fuse the at least two quality assessment vectors to obtain the second quality score.

4. The method according to claim 3, characterized in that, The at least two parallel processing branches include: The first branch is used to focus on local texture details and is configured to: randomly crop the image to be evaluated to obtain local image patches; input the local image patches into the first feature extraction module to obtain local texture features; input the local texture features into the first regression module to output the first quality evaluation vector; The second branch is used to focus on multi-scale structural information and is configured to: perform multi-scale transformation on the image to be evaluated to obtain a multi-scale transformed image; input the multi-scale transformed image into a second feature extraction module to obtain multi-scale structural features; input the multi-scale structural features into a second regression module to output a second quality evaluation vector; The third branch is used to focus on global content information and is configured to: input the image to be evaluated into the third feature extraction module to obtain global features; input the global features into the third regression module to output the third quality evaluation vector.

5. The method according to claim 1, characterized in that, Before acquiring the image to be evaluated, the method further includes: Construct a first training dataset. Each sample in the first training dataset includes a standard reference map, a region of interest mask labeled on the standard reference map, a query map containing distortion, and the ground truth bounding box and ground truth quality score of the corresponding region of interest in the query map. Using the first training dataset, the initial parametric quality assessment network is trained in an end-to-end manner. During the training process, the accuracy of the parametric quality assessment network on the two tasks of region of interest localization and quality regression is optimized by minimizing the first loss function. After training, the pre-trained parametric quality assessment network is obtained. Construct a second training dataset, in which each sample includes a distorted image and its corresponding subjective quality score; Using the second training dataset, the initial nonparametric quality assessment network is trained in an end-to-end manner. During the training process, the regression accuracy of the nonparametric quality assessment network is optimized by minimizing the second loss function. After training, the pre-trained nonparametric quality assessment network is obtained.

6. The method according to claim 5, characterized in that, The second loss function is a weighted sum of the final output loss and the auxiliary loss. The final output loss is used to measure the difference between the second quality score output by the parameterless quality assessment network and the true quality score. The auxiliary loss is used to measure the difference between the quality assessment vector output by each of the at least two parallel processing branches and the true quality score. The first loss function is a weighted sum of localization loss and quality regression loss. The localization loss is used to measure the difference between the bounding box of the region of interest predicted by the parametric quality assessment network and the true bounding box. The quality regression loss is used to measure the difference between the first quality score output by the parametric quality assessment network and the true quality score.

7. An image quality assessment device, characterized in that, include: The image acquisition module is used to acquire the image to be evaluated. The scene judgment module is used to judge the image quality assessment scene, which includes a first scene with a standard reference image and a second scene without a standard reference image; The parametric evaluation module includes a pre-trained parametric quality evaluation network, which is used to input the image to be evaluated, the standard reference image, and the region of interest mask of the standard reference image into the parametric quality evaluation network when the image quality evaluation scenario is the first scenario, so as to obtain a first quality score of the image to be evaluated on the region of interest. The parameterless evaluation module includes a pre-trained parameterless quality evaluation network, which is used to input the image to be evaluated into the parameterless quality evaluation network to obtain a second quality score of the image to be evaluated when the image quality evaluation scenario is the second scenario.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

Images