Image quality evaluation method and device based on semi-reference edge detection, and medium

Abstract

The invention discloses an image quality evaluation method and device based on semi-reference edge detection and a medium, and the method comprises the steps: carrying out the format unification preprocessing of a to-be-evaluated image, so as to obtain an RGB feature image; performing image edge feature extraction on the RGB feature image to obtain to-be-detected edge feature data; inputting the to-be-detected edge feature data into a cascade double-layer attention HED architecture based on an evolutionary algorithm to obtain a multi-stage edge prediction map; performing cross entropy loss function analysis on the multi-stage edge prediction map to obtain an edge information map; performing edge distortion detection on the edge information graph to obtain edge detail loss evaluation data; determining an edge feature vector of the distorted image through distorted image edge feature extraction; and obtaining an image quality evaluation result through image quality evaluation according to the edge feature vector of the distorted image. According to the method, the technical problems of poor adaptability, poor accuracy and single evaluation method of image quality evaluation are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image quality assessment method, device and medium based on semi-reference edge detection. Background Art

[0002] With the rapid development of image processing technology, image quality assessment (IQA) plays an important role in image processing, computer vision and other fields. Image quality assessment methods are divided into two categories: reference and non-reference. The reference quality assessment method calculates the quality score by comparing the original image and the image to be evaluated, while the non-reference image quality assessment method only relies on the characteristics of the image to be evaluated for quality assessment.

[0003] There are still problems with existing no-reference quality assessment image quality assessment methods. First, poor adaptability to distortion types: Many no-reference quality assessment methods cannot effectively deal with different types of distortion in images (such as compression, noise, blur), and the evaluation results are not accurate enough. Second, the evaluation method is fixed: the existing evaluation methods cannot dynamically select the most appropriate evaluation strategy based on the image characteristics, resulting in unstable evaluation results on different types of images. Third, lack of detail preservation evaluation: the existing technology tends to ignore the details of the image, especially the loss of edge information, which affects the accuracy of image quality assessment. Fourth, reliance on manual assistance: the existing technology still relies on manual assistance to a greater or lesser extent, which makes the objective evaluation method far from fully intelligent objective evaluation. Summary of the invention

[0004] The embodiments of the present application provide an image quality assessment method, device and medium based on semi-reference edge detection, which solve the technical problems of poor adaptability, poor accuracy and single assessment method of image quality assessment.

[0005] In a first aspect, an embodiment of the present application provides an image quality assessment method based on semi-reference edge detection, characterized in that the method includes: obtaining an image to be evaluated, and performing format consistency preprocessing on the image to be evaluated to obtain an RGB feature image; performing image edge feature extraction on the RGB feature image to obtain edge feature data to be detected; constructing a cascaded two-layer attention HED architecture based on an evolutionary algorithm, and inputting the edge feature data to be detected into the cascaded two-layer attention HED architecture based on an evolutionary algorithm to obtain a multi-stage edge prediction map; performing cross entropy loss function analysis on the multi-stage edge prediction map to obtain an edge information map; performing edge distortion detection on the edge information map to obtain edge detail loss assessment data; based on the edge detail loss assessment data, determining the edge feature vector of the distorted image by extracting the edge features of the distorted image; and obtaining an image quality assessment result by performing image quality assessment based on the edge feature vector of the distorted image.

[0006] In one implementation of the present application, the image to be evaluated is preprocessed for format consistency to obtain an RGB feature image, specifically including: performing noise removal processing on the image to be evaluated to obtain denoised image data; based on the denoised image data, determining a visually enhanced image through denoised image enhancement; wherein the denoised image enhancement includes: histogram equalization and contrast enhancement; converting the visually enhanced image into RGB format to obtain an RGB feature image.

[0007] In one implementation of the present application, image edge feature extraction is performed on an RGB feature image to obtain edge feature data to be detected, specifically including: color feature extraction is performed on the RGB feature image to obtain image color feature data; wherein the color feature extraction includes: HSV color space conversion, color histogram calculation; texture feature extraction is performed on the RGB feature image through a grayscale co-occurrence matrix and a Gabor filter to obtain image texture feature data; and edge feature data to be detected is obtained based on the image color feature data and the image texture feature data.

[0008] In one implementation of the present application, the edge feature data to be detected is input into a cascaded two-layer attention HED architecture based on an evolutionary algorithm to obtain a multi-stage edge prediction map, specifically including: inputting the edge feature data to be detected into a cascaded two-layer attention HED architecture based on an evolutionary algorithm to obtain a stage edge prediction map; upsampling and fusing the stage edge prediction map to determine a fusion prediction map that conforms to the image to be evaluated; and obtaining a multi-stage edge prediction map based on the stage edge prediction map and the fusion prediction map.

[0009] In one implementation of the present application, a cross entropy loss function analysis is performed on a multi-stage edge prediction graph to obtain an edge information graph, specifically including: constructing a cross entropy loss function based on the multi-stage edge prediction graph; wherein the calculation formula of the cross entropy loss function is:

[0010]

[0011] in, and W are the height and width of the image to be evaluated, Y is the true edge map, For the Edge detection graph of each stage, is the fusion prediction graph; and W are the height and width of the image to be evaluated, Y is the true edge map, For the Edge detection graph of each stage, is the fusion prediction graph, represents the pixel value of the true edge map at position (h,w), and Represents the pixel value of the prediction map and the fusion prediction map at the position (h, w) of the i-th stage; according to the cross entropy loss function, the weighted sum of the losses corresponding to the multi-stage edge prediction map is performed to determine the total loss function of the multi-stage edge prediction map; wherein, the calculation formula of the total loss function of the multi-stage edge prediction map is:

[0012] in, For the The loss weight coefficient of the stage prediction graph of each stage, For the The loss weight coefficient of the fusion prediction map of each stage is calculated; the multi-stage edge prediction map is input into the total loss function of the multi-stage edge prediction map to obtain the edge information map.

[0013] In one implementation of the present application, edge distortion detection is performed on the edge information map to obtain edge detail loss assessment data, specifically including: converting the edge information map into edge information data, and performing edge structure similarity analysis on the edge information data to obtain edge structure similarity; wherein the calculation formula for edge structure similarity analysis is:

[0014] in, To predict the intersection of the edge area and the real edge area, represents the union of the predicted edge area and the real edge area; edge strength difference analysis is performed on the edge information graph to obtain edge strength difference data; wherein the edge strength difference analysis includes: image gradient amplitude analysis and error analysis, and the calculation formula of the image gradient amplitude analysis is:

[0015]

[0016] in, is the gradient in the x direction, is the gradient in the y direction, is the image to be evaluated; the edge position offset analysis is performed on the edge information graph to obtain edge position offset data; wherein the calculation formula for the edge position offset analysis is:

[0017] in, is the total number of pixels in the edge information map, For the image pixels in the real edge map, For the image The position of each pixel on the predicted edge map; according to the edge structure similarity, edge intensity difference data and edge position offset data, the edge detail loss evaluation data is obtained through comprehensive threshold score analysis.

[0018] In one implementation of the present application, an image quality assessment result is obtained through image quality assessment based on the edge feature vector of the distorted image, specifically including: performing high-level feature extraction on the edge feature vector based on a ResNet convolutional neural network with a preset self-attention mechanism to obtain edge high-level features; using the high-level edge features as evaluation learning background data for the ResNet convolutional neural network, and training the model through a mean square error loss function until convergence to obtain an image quality assessment model; inputting the image to be evaluated into the image quality assessment model to obtain an image quality assessment result.

[0019] In one implementation of the present application, after obtaining an image quality assessment result through image quality assessment based on the edge feature vector of the distorted image, the method also includes: converting the quality assessment result into an image quality score and image distortion; and outputting the image quality score and image distortion to a preset visualization system.

[0020] In the second aspect, an embodiment of the present application also provides an image quality assessment device based on semi-reference edge detection, characterized in that the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: obtain an image to be evaluated, and perform format consistency preprocessing on the image to be evaluated to obtain an RGB feature image; perform image edge feature extraction on the RGB feature image to obtain edge feature data to be detected; construct a cascaded two-layer attention HED architecture based on an evolutionary algorithm, and input the edge feature data to be detected into the cascaded two-layer attention HED architecture based on the evolutionary algorithm to obtain a multi-stage edge prediction map; perform cross entropy loss function analysis on the multi-stage edge prediction map to obtain an edge information map; perform edge distortion detection on the edge information map to obtain edge detail loss assessment data; based on the edge detail loss assessment data, determine the edge feature vector of the distorted image by extracting the edge features of the distorted image; and obtain an image quality assessment result by image quality assessment based on the edge feature vector of the distorted image.

[0021] In a third aspect, an embodiment of the present application also provides a non-volatile computer storage medium for image quality assessment based on semi-reference edge detection, storing computer executable instructions, characterized in that the computer executable instructions are set to: obtain an image to be evaluated, and perform format consistency preprocessing on the image to be evaluated to obtain an RGB feature image; perform image edge feature extraction on the RGB feature image to obtain edge feature data to be detected; construct a cascaded two-layer attention HED architecture based on an evolutionary algorithm, and input the edge feature data to be detected into the cascaded two-layer attention HED architecture based on an evolutionary algorithm to obtain a multi-stage edge prediction map; perform cross entropy loss function analysis on the multi-stage edge prediction map to obtain an edge information map; perform edge distortion detection on the edge information map to obtain edge detail loss assessment data; based on the edge detail loss assessment data, determine the edge feature vector of the distorted image by extracting edge features of the distorted image; and obtain an image quality assessment result by image quality assessment based on the edge feature vector of the distorted image.

[0022] The embodiments of the present application provide an image quality assessment method, device and medium based on semi-reference edge detection. By extracting image features, deep edge detection, edge distortion detection and distorted image edge feature extraction, combined with deep learning and attention mechanism technology, the technical problems of poor adaptability, poor accuracy and single assessment method of image quality assessment are solved, and flexible image quality assessment is achieved in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flow chart of an image quality assessment method based on semi-reference edge detection provided in an embodiment of the present application; Figure 2 A schematic diagram of the internal structure of an image quality assessment device based on semi-reference edge detection provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0025] The embodiments of the present application provide an image quality assessment method, device and medium based on semi-reference edge detection. By extracting image features, deep edge detection, edge distortion detection and distorted image edge feature extraction, combined with deep learning and attention mechanism technology, the technical problems of poor adaptability, poor accuracy and single assessment method of image quality assessment are solved, and flexible image quality assessment is achieved in different scenarios.

[0026] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0027] Figure 1 A flow chart of an image quality assessment method based on semi-reference edge detection provided in an embodiment of the present application. Figure 1 As shown, an image quality assessment method based on semi-reference edge detection provided in an embodiment of the present application specifically includes the following steps: Step 101: Obtain an image to be evaluated, and perform format consistency preprocessing on the image to be evaluated to obtain an RGB feature map.

[0028] Exemplarily, preprocessing operations such as noise removal and image enhancement are performed on the input image to ensure the quality of the image in subsequent evaluation. The present application converts the image into an RGB image to facilitate subsequent feature extraction processing.

[0029] Specifically, the image to be evaluated is preprocessed for format consistency to obtain an RGB feature image, including: performing noise removal processing on the image to be evaluated to obtain denoised image data; based on the denoised image data, determining a visually enhanced image through denoised image enhancement; wherein the denoised image enhancement includes: histogram equalization and contrast enhancement; and converting the visually enhanced image into RGB format to obtain an RGB feature image.

[0030] In one embodiment, the image to be evaluated is subjected to noise removal processing, and Gaussian filtering is used to remove noise in the image, especially salt and pepper noise and Gaussian noise in the image. After the image to be evaluated is input, a suitable filter (such as median filtering or Gaussian filtering) is selected, and filtering processing is performed according to the characteristics of the image noise to remove noise and maintain the quality of the image. By convolving the image with a Gaussian kernel, the image is smoothed and high-frequency noise is reduced. Most image details can be effectively retained.

[0031] Image enhancement uses histogram equalization and contrast enhancement methods to improve the overall visual quality of an image. First, the contrast of the image is adjusted by histogram equalization to make the brightness distribution of the image more uniform. By modifying the grayscale level of the image, the brightness contrast of the image is more obvious and the image details are enhanced. Secondly, contrast enhancement is performed to highlight the key information of the image, especially the details such as edges and textures, by adjusting the brightness and contrast of the image. By performing histogram equalization or contrast enhancement on the image, the visual effect of the image is improved, making the subsequent edge detection and quality assessment more accurate.

[0032] Finally, format conversion is performed to convert the image into an RGB image to facilitate subsequent feature extraction processing so that subsequent processing modules can perform efficient processing.

[0033] Step 102: extract image edge features from the RGB feature image to obtain edge feature data to be detected.

[0034] Exemplarily, performing image edge feature extraction on an RGB feature image can extract the basic features of color, texture, and edge of the image, provide information for subsequent edge detection and quality assessment, and achieve targeted extraction of image edge features.

[0035] Specifically, image edge feature extraction is performed on the RGB feature image to obtain edge feature data to be detected, including: color feature extraction is performed on the RGB feature image to obtain image color feature data; wherein the color feature extraction includes: HSV color space conversion, color histogram calculation; texture feature extraction is performed on the RGB feature image through a grayscale co-occurrence matrix and a Gabor filter to obtain image texture feature data; based on the image color feature data and the image texture feature data, the edge feature data to be detected is obtained.

[0036] In one embodiment, first, color feature extraction is performed by converting the image into a color space and calculating a color histogram or other color features to provide support for subsequent image analysis. Furthermore, the frequency of occurrence of each color in the image is calculated through color histogram calculation to generate a color distribution map. The histogram can capture the distribution information of the colors in the image and provide data support for subsequent image analysis.

[0037] Then, the RGB image is converted to the HSV color space through color space conversion so as to extract color features from different color models. The extracted color features are used as features.

[0038] The texture features of the image are extracted using the Gray Level Co-occurrence Matrix (GLCM) and Gabor filter. The Gray Level Co-occurrence Matrix specifically refers to: extracting texture features such as contrast, homogeneity, entropy, etc. by calculating the spatial correlation of the gray values ​​of pixels in the image.

[0039] The Gabor filter is used to extract texture information from the image through convolution operations, especially for edge textures and details. The extracted features are then combined into texture features for subsequent input into the depth assessment model.

[0040] Step 103: construct a cascaded two-layer attention HED architecture based on an evolutionary algorithm, and input the edge feature data to be detected into the cascaded two-layer attention HED architecture based on an evolutionary algorithm to obtain a multi-stage edge prediction map.

[0041] Exemplarily, a cascade HED network based on an evolutionary algorithm is used to extract edge information from an image and enhance robustness to factors such as noise and blur. However, due to the large feature dimension of the image and the complexity of the calculation, the existing HED network generally has the problem of overfitting, resulting in low accuracy. In order to improve the accuracy and robustness of edge detection, this module combines the spatial attention mechanism and the channel attention mechanism, so that the network can focus more on important areas and feature channels in the image, thereby improving the ability to detect image edges. In order to improve the convergence speed of HED edge detection and reduce the risk of overfitting, this module combines the evolutionary algorithm to optimize the network hyperparameters, so that the hyperparameters can achieve high-speed parameter optimization in a probabilistic and selective manner. In order to further improve the accuracy of edge detection and detect more complex or subtle structures, this module introduces a top-down and bottom-up bidirectional information flow mechanism based on the multi-scale deep supervision of HED, and a HED network architecture with a cascaded double-layer attention mechanism.

[0042] Specifically, the edge feature data to be detected is input into the cascaded double-layer attention HED architecture based on the evolutionary algorithm to obtain a multi-stage edge prediction map, including: inputting the edge feature data to be detected into the cascaded double-layer attention HED architecture based on the evolutionary algorithm to obtain a stage edge prediction map; upsampling and fusing the stage edge prediction map to determine a fusion prediction map that conforms to the image to be evaluated; based on the stage edge prediction map and the fusion prediction map, a multi-stage edge prediction map is obtained. During this period, the evolutionary algorithm is used to optimize the network architecture parameters, reduce the risk of overfitting, and greatly improve the convergence speed of the network. The top-down and bottom-up bidirectional information flow mechanism is introduced, and the HED network architecture of the cascaded double-layer attention mechanism is used to improve the edge detection accuracy and enhance the edge detection effect.

[0043] In one embodiment, a single-layer dual-layer attention HED architecture is explained by the following Table 1.

[0044]

[0045] Table 1 The output layer first builds 6 1x1 convolutional layers (conv1_dsn, conv2_dsn, conv3_dsn, conv4_dsn, conv5_dsn, fuse), each with 1 convolution kernel, to generate edge prediction maps at different stages. Then, upsample and fuse the prediction results from conv1_dsn to conv5_dsn to the original image size. Finally, weighted sum the upsampled results from conv1_dsn to conv5_dsn to get the final edge prediction map.

[0046] The second cascaded layer of the two-layer HED architecture is symmetrical to the structure shown in Table 1, with the same number of specific stages, and the output layer is an additional output layer that is the same as the fifth stage in Table 1. The symmetrical cascade form realizes "the upper layer guides the lower layer", helping the underlying features to better focus on meaningful areas during training; "the lower layer details feed back to the upper layer", helping the upper layer to identify finer edges.

[0047] Step 104: Perform a cross entropy loss function analysis on the multi-stage edge prediction graph to obtain an edge information graph.

[0048] Exemplarily, the present application uses a pixel-level cross entropy loss function to calculate the losses of 6 edge prediction maps (prediction maps of 5 stages + fusion prediction maps) respectively, and then performs weighted summation to obtain the total loss, thereby further improving the edge detection performance of the HED network when processing images with large noise or blurred edges.

[0049] Specifically, a cross entropy loss function is analyzed on the multi-stage edge prediction graph to obtain an edge information graph, including: constructing a cross entropy loss function based on the multi-stage edge prediction graph; wherein the calculation formula of the cross entropy loss function is:

[0050]

[0051] in, and W are the height and width of the image to be evaluated, Y is the true edge map, For the Edge detection graph of each stage, is the fusion prediction graph; and W are the height and width of the image to be evaluated, Y is the true edge map, For the Edge detection graph of each stage, is the fusion prediction graph, represents the pixel value of the true edge map at position (h,w), and Represents the pixel value of the prediction map and the fusion prediction map at the position (h, w) of the i-th stage; according to the cross entropy loss function, the weighted sum of the losses corresponding to the multi-stage edge prediction map is performed to determine the total loss function of the multi-stage edge prediction map; wherein, the calculation formula of the total loss function of the multi-stage edge prediction map is: (3) in, For the The loss weight coefficient of the stage prediction graph of each stage, For the The loss weight coefficient of the fusion prediction map of each stage is calculated; the multi-stage edge prediction map is input into the total loss function of the multi-stage edge prediction map to obtain the edge information map.

[0052] In one embodiment, let the input image size be X, the size be H×W, and the true edge map be Y.

[0053] A cross entropy loss function is constructed on the multi-stage edge prediction graph; wherein the cross entropy loss function is explained by the following formula: (1) (2) in, For the Edge detection map of each stage; is the fusion prediction graph; Represents the pixel value (0 or 1) of the true edge map at position (h, w); and Represents the pixel value (probability value between 0 and 1) of the prediction map and the fused prediction map at the position (h, w) of the i-th stage.

[0054] The weighted sum of the prediction map loss of the five stages and the fusion prediction map loss is used to determine the total loss function of the multi-stage edge prediction map; wherein the total loss function of the multi-stage edge prediction map is explained by the following formula: (3) in, For the The loss weight coefficient of the stage prediction graph of each stage, For the The loss weight coefficient of the fusion prediction map of each stage is calculated; the multi-stage edge prediction map is input into the total loss function of the multi-stage edge prediction map to obtain the edge information map.

[0055] The improved HED network architecture introduces a spatial attention module (SAM) and a channel attention module (CAM) at each stage to enhance the network's attention to important areas and feature channels in the image. SAM highlights the salient areas in the image by generating a spatial attention map; CAM emphasizes feature channels with large amounts of information by generating channel attention weights. The combination of these two attention mechanisms can effectively improve the edge detection performance of the HED network, especially when processing images with large noise or blurred edges.

[0056] Furthermore, the present application adopts a spatial attention mechanism, which helps the model to selectively focus on important areas in the image spatially. It can reduce overfitting by assigning more "attention" weights to key areas in the image, so that the model can focus on these areas instead of uniformly processing the entire image. The specific implementation method is to generate a two-dimensional attention map and calculate the importance of each spatial position in the image.

[0057] Furthermore, the channel attention mechanism dynamically adjusts the importance of each channel by assigning a weight to each channel. In the HED network, different channels represent different edge features, and the channel attention mechanism allows the network to pay more attention to those feature channels that are critical to edge detection. The features of each channel are calculated by global average pooling or maximum pooling, and then the weight of each channel is generated through a small fully connected network. Finally, the channel attention weight is multiplied by each channel of the input feature map to enhance the features of important channels.

[0058] It should be noted that after the double-layer attention mechanism layer, the deep important correlation vector of the feature vector is extracted, which improves the learning efficiency of the HED network and further reduces its training complexity. Since the output of the corresponding module of this part is the edge map of the image, it is an abstract representation of the main edge information in the input image, usually an edge map represented by binary or grayscale values. The size of the output image is the same as the input image. The output edge map shows the edge information of the object contour and important structure in the image.

[0059] Step 105: Perform edge distortion detection on the edge information graph to obtain edge detail loss assessment data.

[0060] Exemplarily, distortion detection is performed on edge information in an image to evaluate whether important edge details are lost during compression, transmission, etc., thereby enabling monitoring of the distortion state of edge information in the image and improving the detection accuracy of edge details.

[0061] Specifically, edge distortion detection is performed on the edge information map to obtain edge detail loss assessment data, including: converting the edge information map into edge information data, and performing edge structure similarity analysis on the edge information data to obtain edge structure similarity; wherein the calculation formula for edge structure similarity analysis is:

[0062] in, To predict the intersection of the edge area and the real edge area, represents the union of the predicted edge area and the real edge area; edge strength difference analysis is performed on the edge information graph to obtain edge strength difference data; wherein the edge strength difference analysis includes: image gradient amplitude analysis and error analysis, and the calculation formula of the image gradient amplitude analysis is:

[0063]

[0064] in, is the gradient in the x direction, is the gradient in the y direction, is an image to be evaluated; an edge position offset analysis is performed on the edge information graph to obtain edge position offset data; wherein the edge position offset analysis is explained by the following formula:

[0065] in, is the total number of pixels in the edge information map, For the image pixels in the real edge map, For the image The position of each pixel on the predicted edge map; according to the edge structure similarity, edge intensity difference data and edge position offset data, the edge detail loss evaluation data is obtained through comprehensive threshold score analysis.

[0066] In one embodiment, first, the fidelity of the edge is evaluated by calculating the structural similarity (ESSIM) between two edge maps.

[0067] ESSIM takes into account the continuity, directionality, and strength of edges. It is usually measured using edge overlap (such as IoU (Intersection over Union) or Dice coefficient). This application uses IoU for measurement because the IoU measurement is intuitive and simple to calculate, making it more suitable for actual image analysis applications. The similarity of edge structures can be clearly evaluated by the degree of overlap with the real edge. The edge structure similarity analysis is explained by the following formula: (4) in, To predict the intersection of the edge area and the real edge area, Represents the union of the predicted edge region and the true edge region.

[0068] Furthermore, the IoU value is between 0 and 1. When it is close to 1, it means that the predicted edge and the real edge completely overlap, indicating that the edge structure is very similar (indicating that the image has significant edge distortion and may require further optimization or quality assessment).

[0069] When it is close to 0, it means that the predicted edge has almost no overlap with the real edge, indicating that the edge structure is quite different (indicating that the edge structure of the image is relatively intact and the distortion is small).

[0070] This application sets the IoU threshold to 0.7. When IoU>0.7, it is considered that the edge structure has no significant distortion; when IoU≤0.7, it is considered that the image has significant edge distortion, and subsequent quality evaluation and optimization may be required.

[0071] Then, the edge intensity difference (EID) between the corresponding positions of the two edge maps is calculated to evaluate the degree of loss of edge details. EID can reflect the change in the blurriness and sharpness of the edge. The Canny edge detector is used to calculate the gradient amplitude, and then the mean square error (MSE) or absolute error of the two images is calculated.

[0072] Edge intensity difference analysis includes: image gradient amplitude analysis, error analysis, and image gradient amplitude analysis, which is explained by the following formula: (5) (6) in, is the gradient in the x direction, is the gradient in the y direction, is the image to be evaluated.

[0073] Furthermore, the gradient magnitude G of the Canny edge detector (representing the edge strength of each pixel in the image) is calculated and explained by the following formula: (7) The edge position offset analysis is performed on the edge information graph to obtain edge position offset data; wherein the edge position offset analysis is explained by the following formula: (8) in, is the total number of pixels in the edge information map, For the image pixels in the real edge map, For the image The position of each pixel on the predicted edge map (the marked position of the edge); according to the edge structure similarity, edge intensity difference data and edge position offset data, the edge detail loss evaluation data is obtained through comprehensive threshold score analysis.

[0074] It should be noted that the smaller the MSE value is, the smaller the displacement between the predicted edge and the real edge is, and the more accurate the edge detection result is. Finally, according to the edge distortion measurement results, it is determined whether the image to be evaluated has significant edge distortion. This method uses a comprehensive threshold score and sets different thresholds for the three indicators. When one of the three edge distortion metrics exceeds the threshold, it is considered that the image to be evaluated has edge distortion and requires subsequent quality evaluation and optimization.

[0075] Therefore, the setting of threshold is very critical. Different threshold settings will lead to different distortion detection results, which in turn affects the performance of the entire image quality assessment system. In order to avoid the subjectivity of manually setting thresholds, this application uses a grid search to find the corresponding optimal threshold for three metrics.

[0076] For each set of threshold combinations, the deep edge detection model HED uses these thresholds to calculate edge distortion metrics and evaluate the performance of the model on the training set. For example, the HED model calculates edge structure similarity, edge intensity difference, and edge position offset, and combines the above metrics to judge the performance of the model.

[0077] Furthermore, the threshold setting is specifically set in the following manner: First, define the range of the threshold: According to the general setting method, the threshold of edge overlap is defined as the interval of 0.5 to 0.9, the edge clarity is set to the interval of 0.4 to 0.8, and the edge position offset is set to the interval of 0.1 to 0.9.

[0078] Then, the search is performed by traversing the combination of these thresholds. For example, the threshold of edge overlap can be set to 0.7, the threshold of edge sharpness can be set to 0.6, and the threshold of edge error can be set to 0.1 to combine multiple thresholds.

[0079] Each time a set of thresholds is set, the performance of the HED edge detection model built in S3 on the training set is evaluated, and the optimal threshold combination is selected. Since HED is an edge detection model, the better its edge detection effect is, the better the model extracts edge information. This is because the aforementioned HED model essentially generates an edge map, which is a continuous probability map that represents the strength or confidence of each pixel belonging to the edge, and the value is usually between 0 and 1. For example, when a pixel value is close to 1, it means that it has a high probability of belonging to the edge; close to 0 means that it does not belong to the edge. The threshold is used to convert this continuous value map into a binary map. By setting a threshold, it is determined which pixels are considered "edges" and which are not. Each indicator (such as edge structure similarity, edge intensity difference, edge position offset) will have its own independent threshold. All possible threshold combinations are traversed through grid search and applied to the edge map output by the HED model to evaluate the effect of these threshold combinations. Different thresholds will lead to different edge maps (i.e., binary edge maps), and then the specific values ​​of the indicators are calculated based on each edge map.

[0080] The comprehensive score is calculated for each threshold combination. The threshold combination with a higher comprehensive score is the optimal combination. The comprehensive score corresponding to the evaluation criteria is explained by the following formula: (9) in, (Intersection over Union) is the edge overlap; (Edge Intensity Difference) is the edge intensity difference; (Edge Location Shift) is the edge position shift.

[0081] The higher the value, the better the threshold evaluation score is, the closer the edge detection map of the model is to the reference map, and the more accurate the setting of the characterization threshold is.

[0082] Step 106: Based on the edge detail loss evaluation data, edge feature vectors of the distorted image are determined by extracting edge features of the distorted image.

[0083] In one embodiment, edge detection is directly performed on the distorted image based on the trained HED model. The trained HED model has strong robustness by learning the edge features in the image and can effectively process various types of distorted images, such as compression distortion, noise, blur, etc. The HED model can automatically extract edge information without explicit threshold setting, and is particularly suitable for detecting edges in distorted images.

[0084] It should be noted that the input of the dual attention mechanism HED model is the distorted image. After the image data to be evaluated undergoes preprocessing, the input model is the trained HED edge detection model based on the dual attention mechanism, and the output is the edge feature vector of the distorted image.

[0085] Step 107: Obtain an image quality assessment result by performing image quality assessment according to the edge feature vector of the distorted image.

[0086] According to the edge feature vector of the distorted image, an image quality assessment result is obtained through image quality assessment, which specifically includes: based on the ResNet convolutional neural network with a preset self-attention mechanism, high-level feature extraction is performed on the edge feature vector to obtain edge high-level features; the high-level edge features are used as the evaluation learning background data of the ResNet convolutional neural network, and the model is trained through the mean square error loss function until the model converges to obtain an image quality assessment model; the image to be evaluated is input into the image quality assessment model to obtain the image quality assessment result.

[0087] After obtaining an image quality assessment result through image quality assessment based on the edge feature vector of the distorted image, the method also includes: converting the quality assessment result into an image quality score and an image distortion condition; and outputting the image quality score and the image distortion condition to a preset visualization system.

[0088] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, the embodiment of this application also provides an image quality assessment device based on semi-reference edge detection, whose structure is as follows: Figure 2 shown.

[0089] Figure 2 The internal structure diagram of an image quality assessment device based on semi-reference edge detection provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the device includes: at least one processor 201; and, a memory 202 communicatively connected to the at least one processor; The memory 202 stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor 201 to enable at least one processor 201 to: The image to be evaluated is obtained, and the image to be evaluated is preprocessed for format consistency to obtain an RGB feature image; the image edge feature is extracted from the RGB feature image to obtain the edge feature data to be detected; a double-layer attention HED architecture is constructed, and the edge feature data to be detected is input into the double-layer attention HED architecture to obtain a multi-stage edge prediction map; the multi-stage edge prediction map is analyzed by a cross entropy loss function to obtain an edge information map; edge distortion detection is performed on the edge information map to obtain edge detail loss evaluation data; based on the edge detail loss evaluation data, the edge feature vector of the distorted image is determined by extracting the edge feature of the distorted image; according to the edge feature vector of the distorted image, an image quality evaluation result is obtained by image quality evaluation.

[0090] Some embodiments of the present application provide corresponding Figure 1 A non-volatile computer storage medium for image quality assessment based on semi-reference edge detection stores computer executable instructions, wherein the computer executable instructions are set to: The image to be evaluated is obtained, and the image to be evaluated is preprocessed for format consistency to obtain an RGB feature image; the image edge feature is extracted from the RGB feature image to obtain the edge feature data to be detected; a double-layer attention HED architecture is constructed, and the edge feature data to be detected is input into the double-layer attention HED architecture based on the evolutionary algorithm to obtain a multi-stage edge prediction map; the multi-stage edge prediction map is analyzed by cross entropy loss function to obtain an edge information map; edge distortion detection is performed on the edge information map to obtain edge detail loss evaluation data; based on the edge detail loss evaluation data, the edge feature vector of the distorted image is determined by edge feature extraction of the distorted image; according to the edge feature vector of the distorted image, an image quality evaluation result is obtained by image quality evaluation.

[0091] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the IoT device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0092] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0093] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0094] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0095] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0097] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0098] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0099] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0100] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0101] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An image quality assessment method based on semi-reference edge detection, characterized in that: The method comprises: Acquire an image to be evaluated, and perform format consistency preprocessing on the image to be evaluated to obtain an RGB feature image; Extracting edge features of the RGB feature image to obtain edge feature data to be detected; Constructing a cascaded double-layer attention HED architecture based on an evolutionary algorithm, and inputting the edge feature data to be detected into the cascaded double-layer attention HED architecture based on an evolutionary algorithm to obtain a multi-stage edge prediction graph; Performing a cross entropy loss function analysis on the multi-stage edge prediction graph to obtain an edge information graph; Performing edge distortion detection on the edge information graph to obtain edge detail loss assessment data; Based on the edge detail loss evaluation data, determining an edge feature vector of the distorted image by extracting edge features of the distorted image; An image quality assessment result is obtained by performing image quality assessment according to the edge feature vector of the distorted image.

2. The image quality assessment method based on semi-reference edge detection according to claim 1, characterized in that: The image to be evaluated is preprocessed to obtain an RGB feature image, specifically including: Performing noise removal processing on the image to be evaluated to obtain denoised image data; Based on the denoised image data, a visually enhanced image is determined by denoising image enhancement; wherein the denoising image enhancement includes: histogram equalization and contrast enhancement; The visual enhancement image is converted into RGB format to obtain the RGB feature image.

3. The image quality assessment method based on semi-reference edge detection according to claim 1, characterized in that: Performing image edge feature extraction on the RGB feature image to obtain edge feature data to be detected specifically includes: Color feature extraction is performed on the RGB feature image to obtain image color feature data; wherein the color feature extraction includes: HSV color space conversion and color histogram calculation; Extracting texture features from the RGB feature image using a gray-level co-occurrence matrix and a Gabor filter to obtain image texture feature data; The edge feature data to be detected is obtained based on the image color feature data and the image texture feature data.

4. The image quality assessment method based on semi-reference edge detection according to claim 1, characterized in that: The edge feature data to be detected is input into the dual-layer attention HED architecture to obtain a multi-stage edge prediction map, specifically including: Inputting the edge feature data to be detected into the double-layer attention HED architecture to obtain a stage edge prediction map; Upsampling and fusing the edge prediction graphs of the stages to determine a fused prediction graph that conforms to the image to be evaluated; Based on the stage edge prediction map and the fusion prediction map, the multi-stage edge prediction map is obtained.

5. The image quality assessment method based on semi-reference edge detection according to claim 1, characterized in that: Performing a cross entropy loss function analysis on the multi-stage edge prediction graph to obtain an edge information graph specifically includes: Based on the multi-stage edge prediction graph, a cross entropy loss function is constructed; wherein the calculation formula of the cross entropy loss function is: in, and W are the height and width of the image to be evaluated, Y is the true edge map, For the Edge detection graph of each stage, is the fusion prediction graph, represents the pixel value of the true edge map at position (h,w), and Represents the pixel value of the prediction map and the fused prediction map at the position (h, w) of the i-th stage; According to the cross entropy loss function, the losses corresponding to the multi-stage edge prediction graph are weighted and summed to determine the total loss function of the multi-stage edge prediction graph; wherein the calculation formula of the total loss function of the multi-stage edge prediction graph is: in, For the The loss weight coefficient of the stage prediction graph of the stage, For the The loss weight coefficient of the fusion prediction graph at each stage; The multi-stage edge prediction map is input into the multi-stage edge prediction map total loss function to obtain the edge information map.

6. The image quality assessment method based on semi-reference edge detection according to claim 1, characterized in that: Performing edge distortion detection on the edge information graph to obtain edge detail loss assessment data specifically includes: The edge information graph is converted into edge information data, and edge structure similarity analysis is performed on the edge information data to obtain edge structure similarity; wherein the calculation formula of the edge structure similarity analysis is: in, To predict the intersection of the edge area and the real edge area, represents the union of the predicted edge region and the true edge region; Perform edge strength difference analysis on the edge information graph to obtain edge strength difference data; wherein the edge strength difference analysis includes: image gradient amplitude analysis and error analysis, and the calculation formula of the image gradient amplitude analysis is: in, is the gradient in the x direction, is the gradient in the y direction, is the image to be evaluated; An edge position offset analysis is performed on the edge information graph to obtain edge position offset data; wherein the calculation formula for the edge position offset analysis is: in, is the total number of pixels in the edge information map, For the image pixels in the real edge map, For the image The position of pixels on the predicted edge map; The edge detail loss evaluation data is obtained through comprehensive threshold score analysis based on the edge structure similarity, the edge strength difference data and the edge position offset data.

7. The image quality assessment method based on semi-reference edge detection according to claim 1, characterized in that: According to the edge feature vector of the distorted image, an image quality assessment result is obtained through image quality assessment, which specifically includes: Based on a ResNet convolutional neural network with a preset self-attention mechanism, high-level feature extraction is performed on the edge feature vector to obtain edge high-level features; The high-level edge features are used as evaluation learning background data of the ResNet convolutional neural network, and trained by a mean square error loss function until the model converges to obtain an image quality evaluation model; The image to be evaluated is input into an image quality evaluation model to obtain the image quality evaluation result.

8. The image quality assessment method based on semi-reference edge detection according to claim 1, characterized in that: After obtaining an image quality assessment result by performing image quality assessment according to the edge feature vector of the distorted image, the method further includes: Converting the quality assessment result into an image quality score and an image distortion condition; The image quality score and the image distortion condition are output to a preset visualization system.

9. An image quality assessment device based on semi-reference edge detection, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Acquire an image to be evaluated, and perform format consistency preprocessing on the image to be evaluated to obtain an RGB feature image; Extracting edge features of the RGB feature image to obtain edge feature data to be detected; Constructing a cascaded double-layer attention HED architecture based on an evolutionary algorithm, and inputting the edge feature data to be detected into the cascaded double-layer attention HED architecture based on an evolutionary algorithm to obtain a multi-stage edge prediction graph; Performing a cross entropy loss function analysis on the multi-stage edge prediction graph to obtain an edge information graph; Performing edge distortion detection on the edge information graph to obtain edge detail loss assessment data; Based on the edge detail loss evaluation data, determining an edge feature vector of the distorted image by extracting edge features of the distorted image; An image quality assessment result is obtained by performing image quality assessment according to the edge feature vector of the distorted image.

10. A non-volatile computer storage medium for image quality assessment based on semi-reference edge detection, storing computer executable instructions, characterized in that: The computer executable instructions are configured to: Acquire an image to be evaluated, and perform format consistency preprocessing on the image to be evaluated to obtain an RGB feature image; Extracting edge features of the RGB feature image to obtain edge feature data to be detected; Constructing a cascaded double-layer attention HED architecture based on an evolutionary algorithm, and inputting the edge feature data to be detected into the cascaded double-layer attention HED architecture based on an evolutionary algorithm to obtain a multi-stage edge prediction graph; Performing a cross entropy loss function analysis on the multi-stage edge prediction graph to obtain an edge information graph; Performing edge distortion detection on the edge information graph to obtain edge detail loss assessment data; Based on the edge detail loss evaluation data, determining an edge feature vector of the distorted image by extracting edge features of the distorted image; An image quality assessment result is obtained by performing image quality assessment according to the edge feature vector of the distorted image.

Images