Image redirection quality evaluation method based on global contour matching

Abstract

The present invention discloses a method for evaluating the quality of image redirection based on global contour matching. Unlike existing evaluation indicators, the global contour area of ​​the redirected image is extracted using a method based on Color‑SIFT feature point matching. The features of the image to be evaluated are aligned with the source image through chamfer matching, and the displacement of the object contour before and after redirection is estimated using the transformation distance, so as to reasonably calculate the distortion of the global contour area. In addition, an adaptive redirected image classification evaluation framework is proposed. First, the foreground image is extracted based on the deep learning network, and the image is adaptively classified by judging the threshold set based on human visual perception. Evaluation indicators corresponding to different types of images are given, and the performance of the evaluation results of the method of the present invention is further improved by combining information loss and local significant area distortion.

Description

Technical Field

[0001] The present invention belongs to the technical field of image quality evaluation, and more specifically, relates to an image redirection quality evaluation method based on global contour matching. Background Art

[0002] The human eye can perceive image content through different features such as color, texture and contour. The information perceived by the human eye can be extracted from the image by detecting and identifying features such as edges and contours. [1] The initial edge map is generated by the structured label of the local image block, thereby obtaining the edge feature map of the image. However, this detection algorithm is difficult to produce clear edge segmentation, resulting in incomplete extracted image contours, especially when the contrast between the foreground and background is insufficient, it is difficult to reflect the true distortion of the salient object. Therefore, Zhang et al. [2] Using Algorithms [1] The extracted edge feature map is difficult to accurately evaluate the distortion of the redirected image.

[0003] Figure 1 The given image is a redirected image with the width of the original image reduced to 50%. Figure 1 (a) is the original image, Figure 1 (b) is the image after CR processing, Figure 1 (c) is the image after SC processing, Figure 1 (d) is the image after WARP processing. Clearly, the red curves in the redirected images indicate significant geometric deformation and edge distortion of the object outlines. The first two rows of redirected images show significant geometric distortion (squeezing) of the salient objects, while the last two rows of redirected images show edge distortion of the background regions. These distortions manifest as distorted outlines of salient objects or background regions. To better measure outline distortion in redirected images, this article refers to this as global outline distortion.

[0004] These distortions are highly perceptible to the human eye and manifest as deformation and distortion of the object's outlines and edges in the retargeted image. To address these issues, it is necessary to evaluate these distortions and convert them into distortion of the image's global outline. This can be used to assess the quality of the retargeted image and improve the accuracy of the evaluation algorithm.

[0005] On the other hand, there are some existing evaluation algorithms [3-5] The focus is on measuring the distortion of salient areas, and the effective evaluation of salient area distortion is highly dependent on the accuracy of the detection algorithm. If there are errors in the detection of salient areas or if salient areas cannot be detected, the objective evaluation results will be difficult to match the subjective perception. Figure 2 The method is given [6]Extracting salient regions and methods of original images [7] The extracted global contour features. Figure 2 (a) is the original image, Figure 2 (b) is the significant regional result, Figure 2 (c) is the global contour feature. Figure 2 As shown in (b), the image BedRoom in the first row cannot pass the existing saliency detection algorithm. [8-10] Obtain salient regions; the salient regions detected by the image group Getty in the second row and the salient regions obtained by the image group Jon in the third row are incomplete and have errors. On the other hand, the global contour map can describe the contour features of all objects (salient objects and background areas) in the image, such as Figure 2 Therefore, for the above types of images, if the distortion of the global contour in the redirected image is effectively measured, it will be beneficial to improve the performance of the evaluation algorithm. Summary of the Invention

[0006] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides an image redirection quality evaluation method based on global contour matching, the purpose of which is to evaluate the quality of the redirected image through global contour matching between the original image and the redirected image, thereby solving the technical problem of improving the consistency between the objective evaluation results of image redirection and human eye perception.

[0007] To achieve the above object, the present invention provides an image redirection quality evaluation method based on global contour matching, comprising the following steps:

[0008] Step 1: Input the original image and its redirected image, detect the global contour boundary map of the original image, use the deep learning model EGNet to extract the edge contour features of the original image, and then use the OTSU algorithm to adaptively binarize the image and eliminate the erroneous and messy boundary noise through dilation and erosion to obtain the global contour boundary map;

[0009] Step 2: Extract the global contour of the redirected image, use the improved SIFT-flow to complete the dense pixel matching between the redirected image and the original image, and combine the matching of the global contour area of ​​the original image to obtain the global contour boundary map of the redirected image;

[0010] Step 3: Adaptively classify different feature images, use the PicNet model to detect the salient information of the original image to obtain the foreground object map, and set the threshold based on experience to classify the original image into the non-salient area class and the salient area class;

[0011] Step 4: Calculate the distortion metrics, which are divided into global metrics and local metrics. Global distortion includes Q based on global contours. EGS , image block ARSblock and forward information loss f FIL Evaluation, local distortion includes SARS based on salient regions, FARS based on face blocks, and backward information loss f SBIL Evaluate;

[0012] Step 5: Quality evaluation of the redirected image. Based on the characteristics of the two types of images, the distortion of the redirected image is described from both global and local perspectives. An effective evaluation mechanism for different types of images is developed, and the quality evaluation score of the image redirection is output.

[0013] In one embodiment of the present invention, in step 2, extracting the global contour of the redirected image and using the improved SIFT-flow to complete dense pixel matching between the redirected image and the original image specifically includes:

[0014] Step 2.1: Implement dense pixel matching between the redirected image and the original image. Use Color-flow to complete dense pixel matching between the redirected image and the original image, and obtain the positions of all pixels in the redirected image in the original image. The solution formula is as follows:

[0015]

[0016] Among them, f org and f ret Denote the original image and the redirected image, and p is a pixel in the redirected image. Here, we need to optimize the flow field ω, where u(p) and v(p) are the two components of the flow vector ω(p). Let ε be the global contour space neighborhood extracted from the redirected image. The parameter α is used to control the continuity of the flow field and remove the minimum displacement term in the SIFT energy function. The constants t and d are thresholds set to minimize the constraint flow vector.

[0017] Step 2.2: Mark the global contour of the original image, and mark the pixel positions of the global contour area of ​​the original image. The marking formula is (2), corresponding to formula (3), and the set of pixel positions of the global contour area of ​​the original image is (p sx ,p sy ), recorded as set L; similarly in formula (4), the obtained reverse dense matching position set (p wx ,p wy ) is marked to obtain a set R;

[0018] F=(a-1)*W+b (2)

[0019] L=(p sx -1)*W+p sy (3)

[0020] R=(p ωx-1)*W+p ωy (4)

[0021] Among them, p sx , p ωx ={1,2,3,...,n};p sy , p ωy ={1,2,3,...,m}; n≤W, m≤H, W, H are the width and height of the original image respectively;

[0022] Step 2.3: Get the global contour area of ​​the redirected image. Assume that the set of global contour area points of the original image is L, and the set of global contour pixel positions of the redirected image is R. Then find the intersection C of the set L and the set R, that is, the set of global area points of the redirected image corresponding to the original image, assign a value to the position p(x, y) of the set C of the redirected image, and record the other positions as the set And no value is assigned, so we have:

[0023] C=L∩R (5).

[0024] In one embodiment of the present invention, in step 2, obtaining a global contour boundary map of the redirected image by combining matching of the global contour area of ​​the original image specifically includes:

[0025] Step 2.4: Define the distance transformation template. In order to obtain the distance transformation result of the redirected image, we first need to define a distance transformation template mask, as shown in formula (6). The center of the mask is placed on each pixel in the binary image. When the center position of the mask is superimposed on the target feature point in the binary image, its background feature point will be transformed with the position distance of the mask to obtain the distance value at each position. For each pixel of the binary image to be matched, it is iteratively calculated according to formula (7);

[0026]

[0027]

[0028] in, is the transformed distance when the point (i, j) is iterated m times, and the minimum distance value obtained when it is iterated to m times is taken; (k, l) is the position in the mask, the center is (0, 0), and c(k, l) is the local distance to the mask;

[0029] Step 2.5: Obtain the distance transform of the original image and the redirected image respectively; according to the definition of distance transform, use the global contour binary images of the original image and the redirected image to calculate their distance transforms to finally obtain the matching template and the template to be matched;

[0030] Step 2.6: Perform global contour matching on the original image and the redirected image. After obtaining the distance transformation between the original image and the redirected image, match the matching template with the template to be matched. The specific operation is to superimpose the feature points of the matching template on the template to be matched.

[0031] In one embodiment of the present invention, the step 4 calculates the Q based on the global profile. EGS , specifically including:

[0032] Step 4.1: Calculate the chamfer distance between the original image and the redirected image; let the pixel point set of the DT distance transformation map of the original image be G = {p i}, the pixel set of the DT distance transformation map of the redirected image is G′={p i '}; Calculate the chamfer distance d between (G, G') (G,G′) , d (G,G′) As the optimal chamfer distance between the global contour boundary G′ of the redirected image and the global contour boundary G of the original image:

[0033]

[0034] Where N represents the total number of white pixels 1 in the redirected global contour boundary, and |p i ′-p i | represents the matched redirected image pixel p i ′ and original image pixel p i the distance between them;

[0035] Step 4.2: Weighting and normalization processing; the change between the salient area of ​​the redirected image and the salient area of ​​the original image is used as the control parameter to obtain the retained contour boundary, using an exponential It is weighted and normalized, where γ represents the sum of the ratios between the maximum width and the maximum height of the salient regions of the original image and the redirected image,

[0036]

[0037] Step 4.3: Calculate the global contour distortion score of the redirected image; after weighting and normalization, the global contour distortion evaluation index Q EGS The calculation method of is as follows:

[0038]

[0039] In one embodiment of the present invention, in step 4, the distortion measurement ARS of the image block block Specifically include:

[0040] Based on the image block aspect ratio similarity algorithm, the visual importance map is weighted by utilizing local block changes to evaluate the visual quality of the global content of the redirected image. The calculation formula is as follows:

[0041]

[0042] The grid of the image block is divided into 16×16, and the number of the grid is i×j. ij represents the width-to-height ratio of the (i, j)th grid block. The specific calculation formula is shown in (12). V ij represents the weighted average saliency value of the (i, j)th grid block. The detection method uses the saliency map detection method. The grid block width-to-height ratio between the redirected image and the original image is calculated as follows:

[0043]

[0044] Among them, R w =W r / W o and R h =H r / H o is the ratio of the height to width of the redirected image grid block to the original image grid block, R m =(R w +R h ) / 2 is the average value of the aspect ratio; K is a constant value.

[0045] In one embodiment of the present invention, in step 4, the forward information loss f FIL The assessment specifically includes:

[0046] To measure the information loss from the original image to the redirected image, we introduce the saliency loss rate proposed by Hsu. We only consider the information loss from the original image to the redirected image through the forward process. The calculation formula of the information loss is as follows:

[0047]

[0048] f FIL Indicates the degree of preservation of global salient information after the image is redirected. When f FIL When the value of is 0, it means that the redirected image has no saved information, and f FIL When the value is 1, it means that the image information is preserved completely.

[0049] In one embodiment of the present invention, in step 4, the backward information loss f SBIL Assessment, including:

[0050] The backward information loss evaluation is performed on the image with significant foreground to obtain the saliency map of the original image. SIFT-flow is used to establish feature point matching between the redirected image and the source image to obtain the saliency map of the redirected image. The backward information loss evaluation is completed by the saliency information between the saliency map and the saliency map of the original image. The specific calculation formula is as follows:

[0051]

[0052] In order to verify the accuracy of the reconstructed image, ψ is set as the consistency weight value, and the similarity between the original image and the reconstructed image is calculated using the triangular similarity formula; the ψ calculation formula is as follows:

[0053]

[0054] Where K is a real number 1e-6; I O (p) is the original image, I Rec (p) is the reconstructed image; f BIL The maximum value is 1. When it is equal to 1, it proves that the more pixels the redirected structure can restore, the better its quality.

[0055] In one embodiment of the present invention, in step 3, the original image is classified into a non-salient region class and a salient region class, specifically including:

[0056] The saliency model PiCANet is introduced to detect the saliency map of the image and classify the input image based on the saliency map. The calculation formula is as follows:

[0057]

[0058] Among them, F O is the saliency map extracted by PiCANet, and the saliency value of the pre-existing pixel position at position p is represented as F O (p), N is the total number of pixels with non-zero saliency values, and a threshold T1 is set. If δ is greater than T1, it is determined to be an image group with salient areas, otherwise it is an image group without salient areas.

[0059] In one embodiment of the present invention, in step 5, the NSR redirection image quality evaluation is specifically as follows:

[0060] NSR images do not have obvious salient areas, and the local contents are relatively similar. It is impossible to compensate for the deficiency of global distortion evaluation by measuring the distortion of salient areas. Only global distortion is measured for such images. The specific calculation formula is as follows:

[0061] Q NSR =(1-ALL ARS)+0.2×γ×f FIL +(1-Q EGS ) (17)

[0062] Among them, ALL ARS is the geometric distortion metric; γ is as shown in formula (9), f FIL is a measure of forward information loss, and Q EGS is a global contour distortion measure.

[0063] In one embodiment of the present invention, in step 5, the SR redirected image quality evaluation is specifically as follows:

[0064] In SR images, (1) global ARS can well measure the overall geometric deformation of the image; (2) global contour boundary Q EGS It can more accurately measure the change in the contour position of the object; (3) The change in the ARS of the salient area depends on its proportion in the image and needs to be limited by multiplying the change in the area of ​​the salient area between the redirected image and the source image; (4) f SBIL The backward information loss of the salient area is achieved by using the color histogram change H between the redirected image and the source image. dist as the weight; the information loss of the redirected image is related to the scaling ratio, and the scaling ratio γ of the redirected image and the salient area of ​​the original image is used for constraint; (5) considering the distortion change of the face block, the specific calculation formula is as follows:

[0065]

[0066] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0067] First, the present invention proposes an image redirection quality evaluation index based on global contour matching. Existing methods lack distortion measurement indicators that can accurately describe contours. The present invention introduces a chamfer matching method to measure the distortion of paired global contour features. By calculating the edge contour features of the redirected image and the feature pixels corresponding to the original image, the difference between the original image and the redirected image is calculated using distance transformation, and then the nearest feature pixel distance between them is obtained. In this way, the distortion scores of all object contours after image redirection are accurately calculated, and the objective evaluation results are improved to a level consistent with the subjective evaluation results;

[0068] Second, this paper proposes a method for extracting global contours from redirected and original images. Because the redirected and original images differ in size, it is often necessary to match the image content before performing feature comparison. This method effectively overcomes the difficulty of accurately locating object contours due to changes in image content and structure during the redirection process, completely extracting the corresponding object contours in the redirected and original images.

[0069] Third, experimental results show that, using an existing public standard image redirection database for performance verification, the proposed method has higher and more stable evaluation accuracy compared with existing algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is a schematic diagram of the distortion of different types of redirected images; Figure 1 (a) is the original image, Figure 1 (b) is CR, Figure 1 (c) is SC, Figure 1 (d) is WARP;

[0071] Figure 2 It is a schematic diagram of the method for extracting the salient areas of the original image and the extracted global contour features, where Figure 2 (a) is the original image, Figure 2 (b) is the significant area result, Figure 2 (c) is the global contour feature;

[0072] Figure 3 2 is a schematic diagram of an IRQA algorithm framework based on global contour matching in an embodiment of the present invention;

[0073] Figure 4 : is a graph showing the salient object contour feature detection result and the binarization denoising result of the EGNet method in an embodiment of the present invention; Figure 4 (a) is kodim03 and its redirected image, Figure 4 (b) is kodim23 and its redirected image;

[0074] Figure 5 2. This is a schematic diagram of extracting contour areas of redirected image objects according to an embodiment of the present invention;

[0075] Figure 6 is a schematic diagram of chamfer matching of different shapes in an embodiment of the present invention, wherein Figure 6 (a) is the original image, Figure 6 (b) is the outline of the original image, Figure 6 (c) is the contour matching distance 148.5, Figure 6 (d) is the contour matching distance 759.4, Figure 6 (e) is the contour matching distance 84.3, Figure 6 (f) is the contour matching distance 217.0;

[0076] Figure 7 is a mask schematic diagram describing distance transformation in an embodiment of the present invention;

[0077] Figure 8 Schematic diagram of a binary image and its distance transform DT according to an embodiment of the present invention, wherein Figure 8 (a) is a binary image, Figure 8 (b) is the distance transformation result;

[0078] Figure 9 is a schematic diagram of a global contour matching process in an embodiment of the present invention;

[0079] Figure 10 2. This is a schematic diagram of information retention of a redirected image according to an embodiment of the present invention;

[0080] Figure 11 is a schematic diagram of image classification based on δ value in an embodiment of the present invention;

[0081] Figure 12 2. Schematic diagram of global contour area detection results of a butterfly image and its redirected image according to an embodiment of the present invention;

[0082] Figure 13 : is a schematic diagram of the global contour area detection result of the redirected image according to an embodiment of the present invention; Figure 13 (a) is the image kodim23, Figure 13 (b) is image kodim07;

[0083] Figure 14 Schematic diagram of the degree of fit of 37 groups of redirected images in the RetargetMe database in an embodiment of the present invention. DETAILED DESCRIPTION

[0084] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0085] To solve the above problems, the present invention proposes a method for evaluating the quality of image redirection based on global contour distortion, as follows:

[0086] (1) A quality evaluation index for image redirection based on global contour matching is proposed. Existing methods lack distortion measurement indicators that can accurately describe contours. The method of the present invention introduces a chamfer matching method to measure the distortion of paired global contour features. By calculating the edge contour features of the redirected image and the feature pixels corresponding to the original image, the difference between the original image and the redirected image is calculated using distance transformation, and then the nearest feature pixel distance between them is obtained. In this way, the distortion scores of all object contours after image redirection are accurately calculated, and the objective evaluation results are improved to a level consistent with the subjective evaluation results. This paper conducts extensive verification experiments and ablation studies to prove the effectiveness of the proposed global contour index.

[0087] (2) A global contour extraction method for the redirected image and the original image is proposed. Since the size of the redirected image is different from that of the original image, it is usually necessary to match the image content and then perform feature comparison. In order to obtain an accurate global contour map of the redirected image, this paper uses the global contour of the original image as a reference. First, a deep learning network is used for preprocessing to obtain the global contour feature map of the original image, which is then binarized using the OTSU algorithm. Then, the dilation and erosion operations are used to eliminate erroneous and messy boundary noise, thereby obtaining the global contour map of the original image. Then, the corresponding global contour information of the redirected image is extracted by a marking method through inverse matching. This effectively overcomes the difficulty of accurately locating the object contour information after changes in the image content and structure during the redirection process, and completely extracts the object contours corresponding to the redirected image and the original image.

[0088] (3) An evaluation method for adaptive classification of input images based on saliency maps is proposed. This paper adopts an adaptive classification method for input images based on image saliency maps, dividing the images to be evaluated into two categories: those with salient regions and those without salient regions. Different distortion calculation methods are used to evaluate the quality of different types of redirected images. Global contour matching is used to increase the overall distortion metric of images with salient regions, and the distortion of background regions can be measured. This effectively compensates for the consistency between the distortion of salient and background regions and the distortion perceived by the human eye.

[0089] Methodological Framework

[0090] This method aims to evaluate the quality of the redirected image by global contour matching between the original image and the redirected image. Experimental results show that this method can effectively evaluate the distortion of both the salient area and the background area of ​​the redirected image, improving the consistency between the objective evaluation results and human perception. The overall framework of the method of the present invention is as follows: Figure 3 As shown, the following steps can be used to describe it.

[0091] Objective: Get a quality assessment score for the redirected image.

[0092] Input: original image and its redirected image.

[0093] Step 1: Detect the global contour boundary map of the original image. First, use the deep learning model EGNet [7] Extract edge contour features of the original image using the OTSU algorithm

[11] After adaptive binarization processing, the erroneous and messy boundary noise points are eliminated through dilation and erosion to obtain a clear global contour boundary map.

[0094] Step 2: Extract the global contour of the redirected image using the improved SIFT-flow

[13] Complete the dense pixel matching of the redirected image to the original image, and combine the matching of the global contour area of ​​the original image to obtain the global contour boundary map of the redirected image.

[0095] Step 3: Adaptively classify images with different features. Using PicNet [6] The model detects salient information in the original image and obtains a foreground object map. Based on the HVS characteristic, the human eye generally only notices salient objects whose saliency values ​​exceed a certain range. Based on empirically set thresholds, the original image is classified into two categories: No Salient Region (NSR) and Salient Region (SR).

[0096] Step 4: Calculate the distortion metric. This method is divided into global metric and local metric. The global distortion is calculated by Q based on the global profile. EGS , image block ARS block and forward information loss f FIL Evaluation. Local distortion is composed of SARS based on salient regions, FARS based on face patches, and backward information loss f SBIL Evaluate.

[0097] Step 5: Quality evaluation of the redirected image. Based on the characteristics of the two types of images, the method describes the distortion of the redirected image from both global and local perspectives, and formulates an effective evaluation mechanism for different types of images. For example, for NSR images with no significant areas, the global distortion evaluation index should be Q based on the global contour. EGS , ARS of image blocks block and forward information loss f FIL .

[0098] Output: Quality assessment score of the image redirection.

[0099] The technical solution of the present invention is specifically introduced below:

[0100] Global contour extraction and matching

[0101] Since the image undergoes pixel gain or loss during the redirection process, the reconstructed image undergoes changes in composition, color, and lines. However, existing image contour feature detection methods cannot determine whether the extracted redirected area is consistent with the original image.

[0102] 1. Global contour extraction of original image

[0103] In order to overcome the above problems, this module uses the deep learning network EGNet [7] Detect object edge feature maps from images. In addition, the OTSU algorithm is also used

[74] Processing is performed to obtain a clear global contour boundary map. Figure 4 The EGNet method is given [7] The results of the salient object contour feature detection and binary denoising results are shown in Figure 2. Figure 4 (a) is kodim03 and its redirected image, Figure 4 (b) is kodim23 and its redirected image. In more detail, the first row is the original images kodim03 and kodim23 and their redirected images reduced to 75% and 50%, the second row is the corresponding result image in the first row extracted by the EGNet network, and the third row is the binary denoising result image.

[0104] 2 Redirect image global contour extraction

[0105] Due to the different sizes of the original and retargeted images, accurately assessing the distortion of the retargeted image typically requires establishing a matching relationship between the retargeted and original images. However, the retargeted image suffers from structural distortion during scaling, which introduces additional computational errors into the subsequent quality assessment. Furthermore, traditional edge detection methods often produce sparse edges due to insufficient contrast when extracting features, making it difficult for the extracted features to represent the global outline of the image.

[0106] To solve this problem, the proposed method uses an improved SIFT-flow

[12] to obtain the feature points between the original image and the redirected image to obtain the global contour area of ​​the redirected image. In addition, an improved color-based Color-flow method

[14] is used to align the best description method for feature point matching. Figure 5 The following is a diagram of object contour area extraction in the redirected image. The specific steps are as follows:

[0107] Step 2.1: Achieve dense pixel matching between the redirected image and the original image. Use Color-flow to achieve dense pixel matching between the redirected image and the original image, and obtain the positions of all pixels in the redirected image in the original image. The solution formula is as follows:

[0108]

[0109] Among them, f org and f ret Denote the original image and the redirected image, and p is the pixel of the redirected image. Here, the flow field ω needs to be optimized, where u(p) and v(p) are the two components of the flow vector ω(p). Let ε be the global contour space neighborhood extracted from the redirected image, and the parameter α is used to control the continuity of the flow field. Here, α=20 is taken. Since most of the redirected image consists of discontinuous image blocks of the original image, some flow vectors may have considerable amplitudes. To solve this problem, the method of literature

[15] is used in this step.

[13] The method removes the minimum displacement term in the SIFT energy function, where constants t and d are thresholds set to minimize the constraint flow vector. Here, t and d are set to 0.6.

[0110] Step 2.2: Mark the global contour of the original image. Mark the pixel positions of the global contour area of ​​the original image. The marking formula is (2). Corresponding to formula (3), the set of pixel positions of the global contour area of ​​the original image is (p sx ,p sy ), recorded as set L; similarly, in formula (4), the reverse dense matching position set (p wx ,p wy ) can be marked to obtain the set R.

[0111] F=(a-1)*W+b (2)

[0112] L=(p sx -1)*W+p sy (3)

[0113] R=(p ωx -1)*W+p ωy (4)

[0114] Among them, p sx , p ωx ={1,2,3,...,n}. p sy , p ωy = {1, 2, 3, ..., m}. n ≤ W, m ≤ H. W and H are the width and height of the original image, respectively.

[0115] Step 2.3: Obtain the global contour region of the redirected image. Assume that the set of global contour region points of the original image is L, and the set of global contour pixel positions of the redirected image is R. Next, find the intersection C of set L and set R, that is, the set of global region points of the redirected image corresponding to the original image. Assign a value to the position p(x, y) in set C of the redirected image, and record the other positions as set C without assigning any values. This will accurately extract the global contour region of the redirected image calculated by the method of the present invention. Thus, we have:

[0116] C=L∩R (5)

[0117] 3 Global contour matching between original image and redirected image

[0118] Broken and distorted lines / edges are two common sources of distortion in retargeted images. Distortion of straight lines and smooth curves is often the determining factor in overall quality, and contour deformation is the most visible manifestation of line / edge distortion, leading to significantly worsening artifacts.

[0119] like Figure 6 The example shown is the chamfer matching process of different shapes. This process is implemented by using chamfer matching

[15] To perform contour alignment, (a) is the original image, (b) is the contour detection result of (a), (c) to (f) are chamfer matching with (b) after different aspect ratio transformations, and the obtained chamfer distances are: 148.5, 759.4, 84.3 and 217.0 respectively. This method is based on chamfer matching.

[15] The distortion of the global contour is measured by the following method. The specific steps are as follows:

[0120] Step 3.1: Define the distance transformation template. The matching process of the method of the present invention refers to the chamfer matching in the literature

[15] . In order to obtain the distance transformation result of the redirected image, it is necessary to first define a distance transformation template mask, as shown in formula (6). The center of the mask is placed on each pixel in the binary image. Whether the sequential or parallel method is used, the final mask distance transformation is consistent. This paper uses the sequential method for iterative calculation. When the center position of the mask is superimposed on the target feature point (pixel value 1) in the binary image, its background feature point (pixel value 0) will be transformed with the position distance of the mask to obtain the distance value at each position. As shown in Figure 7 As shown in FIG, the mask of the distance transformation is described, where 0 is the masked target feature point, other pixel positions are masked background feature points, and +c1 to +c5 represent distance values ​​1 to 5. At this time, each pixel of the binary image to be matched is iteratively calculated according to formula (7).

[0121]

[0122]

[0123] in, It represents the transformed distance when the point (i, j) is iterated m times. Since the distance value of the same position is different each time the mask is applied, this step takes the minimum distance value obtained when it is iterated to m times; (k, l) is the position in the mask, with the center at (0, 0), and c(k, l) is the local distance to the mask.

[0124] Step 3.2: Obtain the distance transform of the original image and the redirected image respectively. According to the definition of the distance transform in step 1, the distance transform of the original image and the redirected image is calculated using the global contour binary images. Figure 8 Figure 1 shows a binary image and its distance transform (DT). (a) shows a binary image with features * and non-features -. The feature point in the binary image is "F." After the distance transform, all pixels have a value corresponding to the distance from "F." (b) shows the distance transform result from each pixel to the nearest feature point. This process ultimately yields a matching template (the distance transform of the original image) and a template to be matched (the distance transform of the retargeted image).

[0125] Step 3.2: Perform global contour matching on the original image and the redirected image. After obtaining the distance transformation of the original image and the redirected image according to step 2, match the matching template with the template to be matched. The specific operation is to superimpose the feature points of the matching template on the template to be matched. The matching process is as follows: Figure 9 shown.

[0126] 4 Retargeted Image Quality Evaluation Index Based on Global Contour Matching

[0127] In order to evaluate the distortion of the global contour of the redirected image, this paper proposes a redirected image quality evaluation index based on global contour matching after matching the global contours of the original image and the redirected image. The implementation process is as follows:

[0128] Step 4.1: Calculate the chamfer distance between the original image and the redirected image. Let the pixel set of the DT distance transformation map of the original image be G = {p i}, the pixel set of the DT distance transformation map of the redirected image is G′={p i ′}. Calculate the chamfer distance d between (G, G′) (G,G′) , d (G,G′) As the optimal chamfer distance between the global contour boundary G′ of the redirected image and the global contour boundary G of the original image:

[0129]

[0130] Where N represents the total number of white pixels 1 in the redirected global contour boundary, and |p i ′-p i | represents the matched redirected image pixel p i ′ and original image pixel p i The distance between them.

[0131] Step 4.2: Weighting and normalization. In this step, the difference between the salient area of ​​the redirected image and the salient area of ​​the original image is used as the control parameter for retaining the contour boundary. Therefore, the method of the present invention adopts an index It is weighted and normalized, where γ represents the sum of the ratios between the maximum width and the maximum height of the salient regions of the original image and the redirected image,

[0132]

[0133] Step 4.3: Calculate the global contour distortion score of the redirected image. After weighting and normalization in step 2, the global contour distortion evaluation index Q EGS The calculation method of is as follows:

[0134]

[0135] Distortion metrics

[0136] The proposed method mainly considers two aspects of distortion measurement: (1) Global distortion index: This method calculates the distortion of all object contours and estimates the distortion of image blocks in a traversal manner. (2) Local distortion index. The proposed method describes the overall distortion of the image by the distortion degree of the global contour, and adds distortion evaluation of salient area information to compensate for the local distortion degree, simulating the human eye's perception of the redirected image distortion process.

[0137] Global Measurement

[0138] For some foreground objects that are not obvious and occupy a large area of ​​the original image, as well as image groups with dense information or global patterns, it is necessary to take a global measurement approach to measure their distortion. In this paper, a new Q-based global contour distortion method is proposed. EGS Indicators, introduction of literature

[12] The ARS indicator proposed in the literature

[40] The forward information loss index FIL is used to adapt to the evaluation framework of the method of the present invention.

[0139] (1) Distortion measurement of object outline

[0140] This distortion type uses the measurement index Q proposed in this paper EGS Make the measurement, i.e. formula (10).

[0141] (2) Distortion measurement of image blocks

[0142] This paper uses the image block aspect ratio similarity algorithm, utilizes local block changes, assigns weights to the visual importance map, and evaluates the visual quality of the global content of the redirected image. The calculation formula is as follows:

[0143]

[0144] The grid of the image block is divided into 16×16, and the number of the grid is i×j. ij Represents the width-to-height ratio of the (i, j)th grid block. The specific calculation formula is shown in (12). ij It represents the weighted average significance value of the (i, j)th grid block. The detection method uses the literature [6] The proposed saliency map detection method. The grid block aspect ratio between the redirected image and the original image is calculated as follows:

[0145]

[0146] Among them, R w =W r / W o and R h =H r / H o is the ratio of the height to width of the redirected image grid block to the original image grid block, R m =(R w +R h ) / 2 is the average aspect ratio. K is a constant value of 1e-6, and α is a smoothing coefficient of 0.3.

[0147] (3) Forward information loss measurement

[0148] In order to measure the information loss from the original image to the redirected image, we introduce Hsu et al.

[40] The proposed saliency loss rate. In this method, only the information loss from the original image to the redirected image is considered through the forward process. Figure 10 The calculation process of information loss is shown in Figure 1. The calculation formula is as follows:

[0149]

[0150] f FIL Indicates the degree of preservation of global salient information after the image is redirected. FIL When the value of is 0, it means that the redirected image has no saved information, and f FIL When the value is 1, it means that the image information is preserved completely.

[0151] 4.2 Local Measurement

[0152] The method of the present invention evaluates the local distortion of the image by considering the distortion of the salient area to compensate for the deficiency of the global evaluation, thereby better optimizing the evaluation framework.

[14] The proposed aspect ratio similarity is used to measure the geometric similarity of salient areas and faces before and after redirection. The second method is based on the reverse matching information of the uniform saliency map to measure the ability of the redirected image to preserve important information.

[0153] (1) Geometric distortion index of significant areas

[0154] This paper draws on the literature [5] The proposed geometric distortion estimation method based on salient region aspect ratio similarity estimates the aspect ratio of the detected salient regions. For images with faces, the aspect ratio of the face block is used for estimation to assess the degree of distortion of the salient object that the human eye perceives most acutely, thereby compensating for the shortcomings of the global distortion metric. The calculation process is consistent with Equations (11) and (12).

[0155] (2) Significant area information loss index

[0156] In the process of image retargeting, pixel deletion is inevitable. When a large amount of information is lost in an image group with a significant region class, an obvious missing block effect will occur.

[16] The proposed bidirectional saliency information loss assessment method evaluates backward information loss for images with significant foreground features. First, the saliency map of the original image is obtained. SIFT-flow is then used to match feature points between the redirected image and the source image, resulting in a saliency map of the redirected image. Therefore, backward information loss can be assessed by measuring the amount of salient information between the saliency map and the original image's saliency map. The specific calculation formula is as follows:

[0157]

[0158] In order to verify the accuracy of the reconstructed image, ψ is set as the consistency weight value, and the similarity between the original image and the reconstructed image is calculated using the triangular similarity formula. The ψ calculation formula is as follows:

[0159]

[0160] Where K is a real number 1e-6. O (p) is the original image, I Rec (p) is the reconstructed image. f BIL The maximum value is 1. When it is equal to 1, it proves that the more pixels the redirected structure can restore, the better its quality.

[0161] 5 Image Redirection Quality Classification Evaluation Mechanism

[0162] Since different images have different salient information contents, the degree of distortion encountered during the redirection process is also different. In order to adapt to the quality evaluation of different types of images, this paper introduces the saliency model PiCANet [6] To detect the saliency map of the image, the input image is classified based on the saliency map. The calculation formula is as follows:

[0163]

[0164] Among them, F O is the saliency map extracted by PiCANet, and the saliency value of the pre-existing pixel position at position p is represented as F O (p), N is the total number of pixels with non-zero saliency values. Set a threshold T1, here let T1 = 60, if δ is greater than T1, then it is determined to be an image group with salient regions, otherwise it is an image group without salient regions. An example of image classification detection based on δ value is as follows Figure 11 As shown, the first row is the original image, and the second row is the foreground target detection result corresponding to the original image in the first row.

[0165] 5.1 NSR Redirection Image Quality Evaluation

[0166] NSR images do not have obvious salient areas, and the local content is relatively similar. It is impossible to compensate for the deficiency of global distortion evaluation by measuring the distortion of salient areas. Therefore, only global distortion is measured for these images. The specific calculation formula is as follows:

[0167] Q NSR =(1-ALL ARS )+0.2×γ×f FIL +(1-Q EGS ) (17)

[0168] Among them, ALL ARS It comes from the literature

[14] The geometric distortion metric of γ is shown in formula (9), f FIL References

[17] The forward information loss metric proposed here is multiplied by a weight of 0.2 to measure the global information loss as a weakly significant pixel; and Q EGS This is the global contour distortion measurement index proposed by the method of the present invention.

[0169] 5.2 SR-class redirected image quality evaluation

[0170] In SR images, (1) global ARS can well measure the overall geometric deformation of the image; (2) global contour boundary Q EGS It can measure the change of the object's contour position more accurately; (3) The change of the ARS of the salient area depends on its proportion in the image, and needs to be limited by multiplying the change of the salient area area between the redirected image and the source image; (4) f SBIL The backward information loss of the salient area is achieved by using the color histogram change H between the redirected image and the source image. dist As the weight. The information loss of the redirected image is related to the scaling ratio, so the scaling ratio γ of the redirected image and the salient area of ​​the original image is used as a constraint; (5) Consider the distortion change of the face block. The specific calculation formula is as follows:

[0171]

[0172] 6 Experimental results and analysis

[0173] In this section, the method of the present invention uses two publicly available databases RetargetMe

[18] and CUHK

[19] Experiments are conducted to demonstrate the effectiveness of the objective evaluation method proposed in this paper. In addition, the method of the present invention is compared with the most advanced IRQA method to further demonstrate the advantages of the method.

[0174] 3.6.1 Test Database

[0175] (1) RetargetMe database

[0176] The RetargetMe database contains 37 source images. The consistency calculation formula used in the database comes from Kendall rank correlation coefficient (KRCC)

[20] , which is used here to measure the correlation between subjective and objective rankings, is calculated as follows:

[0177]

[0178] Among them, n c represents the consistency between two ranking pairs, n d represents an outlier pair, and n is the total number of pairs. KRCC ranges from -1 to 1. The closer KRCC is to 1, the more similar the sequences between the objective and subjective quality rankings are. Conversely, a negative correlation indicates a greater difference between the sequence pairs.

[0179] (2)CUHK Database

[0180] CUHK Database [2]The dataset consists of 57 original images. In addition to the eight classic retargeting methods, the retargeting algorithm also adds the SCSC and ENER retargeting methods. Five different ratings (poor, slightly poor, fair, good, and excellent) are used for evaluation in this dataset to generate a mean subjective score (MOS).

[0181] Four metrics were used for comparison of the CUHK database: Pearson's linear correlation coefficient (PLCC), Spearman's rank correlation coefficient (SRCC), root mean square error (RMSE), and outlier ratio (OR). The outlier ratio represents the percentage of retargeted images whose mapping scores fall outside the interval [MOS - 2σ, MOS + 2σ], where MOS and σ are the mean and deviation of the target scores, respectively. An effective objective quality assessment method generally has high PLCC and SRCC values, but relatively low RMSE and OR values.

[0182] 3.6.2 Performance of Objective Evaluation Methods for Image Retargeting Quality

[0183] The experimental results of the proposed scheme are as follows: 1) the performance improvement of the proposed image redirection quality objective evaluation method; 2) the effectiveness evaluation of the proposed image redirection quality objective evaluation method for different types of images; 3) the proposed Q EGS Validity of indicators.

[0184] (1) Performance of global contour region extraction

[0185] In order to verify the performance of this method in extracting global contour features, the image groups butterfly, kodim23 and kodim07 are selected from the redirected image benchmark database RetargetMe and CUHK for verification. Figure 12 and Figure 13 As shown in this paper, the proposed global contour feature extraction method and method

[21] The extracted edge features are compared. Correspondingly, the first line in the figure is the source image and its redirected image, and the second line is the method

[21] The third row of the generated edge feature map shows the global contour result map extracted by the method in this paper. As can be seen from the results shown, the contour features extracted by the method of the present invention are more obvious.

[0186] (2) RetargetMe database evaluation performance

[0187] All performance tests of our method were conducted on the RetargetMe database, calculating the average KRCC of 37 image groups to verify the similarity between our method and subjective human evaluation. Average KRCC values ​​were calculated for each image attribute to visually demonstrate the detailed performance of our method. Algorithm stability is measured using variance; smaller variance indicates more stable algorithms. Table 1 shows a comprehensive comparison of our method with 15 classic objective evaluation methods. All comparison data are derived from experimental data presented in the corresponding papers.

[0188] As can be seen from Table 1, the average KRCC of the method of the present invention reaches 0.606. Figure 14 The KRCC correlation value histogram of the objective scores of eight redirection methods and their corresponding subjective scores for 37 images in the database is given.

[22] and RPCS based on machine learning

[23] Compared with the other methods, the average KRCC value of the proposed method is much higher than that of MLF and RPCS, and the overall KRCC value is much higher than that of other algorithms.

[0189] Table 1 Comparison of attribute evaluation on the RetargetMe database

[0190]

[0191]

[0192] Based on the classification characteristics of this method, the overall performance improvement for the NSR (8 groups) and SR (29 groups) image sets in RetargeteMe can be detailed, as shown in Table 2. Compared with three existing algorithms with good performance based on object contour distortion assessment, MLF, FGM, and RPCS, in the evaluation of the NSR image set, this method achieved an average correlation of 0.563, an improvement of 0.167 over MLF, a 42.2% increase. It also improved by 0.069 over FGM, a 14.0% increase. It also improved by 0.117 over RPCS, a 26.2% increase. For the SR image set, the average correlation of this method improved by 13.6% over MLF, 7.1% over FGM, and 1.1% over RPCS. Furthermore, the algorithm demonstrated optimal stability, with a lower variance than existing evaluation algorithms.

[0193] Table 2 Comparison of evaluation performance of NSR and SR redirected images

[0194]

[0195] It can be seen that in the evaluation of these two types of image sets, regardless of which type of image set, the overall evaluation performance of the method of the present invention is excellent. In addition, from a statistical point of view, the distribution of the correlation between the objective scores and the subjective scores of the 37 groups is observed. As shown in Table 3, the average KRCC values ​​in the first 5 to the first 20 different intervals of the 37 evaluation results are statistically analyzed. Compared with MLF and RPCS, the advantage of this paper is that the average KRCC values ​​in the last two intervals maintain an average correlation of more than 0.75, while the average KRCC values ​​in the last two intervals of other algorithms have dropped to 0.75 or below 0.70. This shows that the correlation values ​​of the method of the present invention are evenly distributed, and most of the image groups have high consistency between objective evaluation and subjective evaluation.

[0196] Table 3 Ranking statistics of RetargetMe database

[0197]

[0198] (3) CUHK database evaluation performance

[0199] On the CUHK database, this method fits the obtained objective data with the subjective data, and uses this fitting method to determine the degree of similarity between the subjective and objective evaluations. Furthermore, this method uses five-fold cross-validation (LOOCV) to train the database, iteratively leveraging the scarce available data to generate objective scores. LOOCV fully utilizes data, and based on this characteristic, during training, 80% of the data is iteratively randomly selected as the training set, with the remaining 20% ​​as the validation set.

[0200] This method classifies 57 groups of images in the CUHK database into NSR (10 groups) and SR (47 groups). Since this method uses inconsistent evaluation schemes for different types of image sets, different fitting parameters 2 are used. c and 2 r It is necessary. Experiments show that when evaluating the NSR image set, the optimal parameters of MLF are c = 0.1, r = 0.1, while the optimal parameters of this paper are c = 0.1, r = 1. When evaluating the SR image set, the optimal parameters of MLF are c = 0.5, r = 1, while the optimal parameters of this paper are c = 0.1, r = 0.1.

[0201] For comparison, the performance comparison of this experiment introduces the results of 10 IRQA algorithms. The experimental results are listed in Table 4. Compared with the evaluation algorithm compared in this paper, the average PLCC obtained is as high as 0.7713, which is higher than the MLF which also measures the edge features of the image.

[22] The algorithm improved by 3.2%.

[0202] Table 4 Evaluation performance comparison on CUHK database

[0203]

[0204] Table 5 Comparison of classification evaluation performance on CUHK database

[0205]

[0206] In addition, in order to verify the performance of classification evaluation, the MLF which also measures the edge features is introduced

[22] and FGM [4] The evaluation method was compared and verified. Where δ = 0 represents the performance results for the NSR class, and δ > 0 represents the performance results for the SR class. Table 5 shows that the metrics designed by this method achieve optimal performance to a large extent. Specifically, the evaluation results after classification using the method of the present invention all reach optimal values, effectively evaluating the quality of images of different categories.

[0207] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0208] References:

[0209] [1]P Dollár, C Zitnick. Fast Edge Detection Using Structured Forests[J]. IEEE Transactionson Pattern Analysis and Machine Intelligence, 2015, 37(8): 1558–1570.

[0210] [2]Y Zhang, W Lin, Q Li, et al. Multiple-level feature-based measure for retargeted image quality [J]. IEEE Transactions on Image Processing, 2018, 27(1): 451–463.

[0211] [3]M Karimi,S Samavi,N Karimi,et al.Quality assessment of retargetedimages bysalient region deformity analysis[J].Signal Processing:ImageCommunication,2017,43(2017):108-118.

[0212] [4]Y Li,L Guo,L Jin.A Content-Aware Image Retargeting QualityAssessment MethodUsing Foreground and Global Measurement[J].IEEE Access.2019,(07):91912-91923. [5]Z Tang,J Yang.Image retargeting quality assessment based onsaliency-drivenclassification[J].Signal Processing:Image Communication,2022,(105),116688:1-13.

[0213] [6]M Cheng,G Zhang,N Mitra,et al.Global contrast based salient regiondetection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,37(3):409–416. [7]Y Li,L Guo,L Jin.AContent-Aware Image Retargeting QualityAssessment MethodUsing Foreground and Global Measurement[J].IEEE Access.2019,(07):91912-91923. [8]M Eitz,J Hays,M Alexa.How do humans sketch objects?[J].ACMTransactions onGraphics,2012,31(4):44.

[0214] [9]C Zitnick,P Dollár.Edge boxes:Locating object proposals from edges[C] / / in ComputerVision—ECCV(Lecture Notes in Computer Science),2014:391–405.

[0215]

[10] Y Liu,C Y u,M Y u,et al.Manifold SLIC:Afast method to computecontent-sensitivesuperpixels[C] / / in:Proceedings of

[0216]

[11] Q Hou,M Cheng,X Hu,et al.Deeply supervised salient objectdetection with shortconnections[J].IEEE Transactions on Pattern Analysis andMachine Intelligence,2019,41(4):815-828.

[0217]

[12] O Pele,M Werman.Fast and robust earth mover’s distances[C] / / in:Proceedings of theIEEE International Conference on Computer Vision,2009:460–467.

[0218]

[13] Z Luo,A Mishra,AAchkar,et al.Non-local deep features for salientobject detection[C] / / in:Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition,2017:6593–6601.

[0219]

[14] F Shao, W Lin, W Lin, et al. QoE-guided warping for stereoscopicimage retargeting[J]. IEEE Trans.Image Process, 2017, 26(10): 4790–4805.

[0220]

[15] Z Tang, J Yang. Image retargeting quality assessment based onsaliency-driven classification [J]. Signal Processing: Image Communication, 2022, (105), 116688: 1-13.

[0221]

[16] Chen Zhibo, Lin Jianxin. Image retargeting quality evaluation method based on statistical similarity and bidirectional saliency fidelity[P]. CN106447654B, 2019.

[0222]

[17] Z Karni, D Freedman, C Gotsman. Energy-based image deformation[J]. ComputerGraphics Forum, 2009, 28(5): 1257-1268.

[0223]

[18] M Rubinstein, D Gutierrez, O Sorkine, et al. A comparative study of image retargeting[J]. ACM Transactions on Graphics, 2010, 29(6): Article No.160.

[0224]

[19] D Cho, J Park, T Oh, et al. Weakly-and Self-Supervised Learning for Content-AwareDeep Image Retargeting[C] / / in: Proceedings of the IEEEInternational Conference on Computer Vision, 2017: 4568-4577.

[0225]

[20] N Liu,J Han,M Yang.PiCANet:Pixel-Wise Contextual AttentionLearning forAccurate Saliency Detection[J] / / IEEE Transactions on ImageProcessing,2020,29(2020):6438–6451.

[0226]

[21] M Karimi,S Samavi,N Karimi,et al.Quality assessment of retargetedimages bysalient region deformity analysis[J].Signal Processing:ImageCommunication,2017,43(2017):108-118.

[0227]

[22] Y Fang,K Zeng,Z Wang,et al.Objective Quality Assessment for ImageRetargetingBased on Structural Similarity[J].IEEE Journal on Emerging andSelected Topics inCircuits and Systems,2014,4(1):95-105.

[0228]

[23] J Zhang,C Kuo.An Objective Quality of Experience(QoE)AssessmentIndex forRetargeted Images[C] / / in:Proceedings of the ACM internationalconference onMultimedia,2014:257–266.

Claims

1. A method for evaluating image redirection quality based on global contour matching, characterized in that: The steps include: Step 1: Input the original image and its redirected image, detect the global contour boundary map of the original image, use the deep learning model EGNet to extract the edge contour features of the original image, and then use the OTSU algorithm to adaptively binarize the image and eliminate the erroneous and messy boundary noise through dilation and erosion to obtain the global contour boundary map; Step 2: Extract the global contour of the redirected image, use the improved SIFT-flow to complete the dense pixel matching between the redirected image and the original image, and combine the matching of the global contour area of ​​the original image to obtain the global contour boundary map of the redirected image; Step 3: Adaptively classify different feature images, use the PicNet model to detect the salient information of the original image to obtain the foreground object map, and set the threshold based on experience to classify the original image into the non-salient area class and the salient area class; Step 4: Calculate the distortion metrics, which are divided into global metrics and local metrics. Global distortion includes Q based on global contours. EGS , image block ARS block and forward information loss f FIL Evaluation, local distortion includes SARS based on salient regions, FARS based on face blocks, and backward information loss f SBIL Evaluation; the step 4 calculates Q based on the global profile EGS , specifically including: Step 4.1: Calculate the chamfer distance between the original image and the redirected image; let the pixel point set of the DT distance transformation map of the original image be G = {p i }, the pixel set of the DT distance transformation map of the redirected image is G′={p′ i }; Calculate the chamfer distance d between (G, G′) (G,G′) , d (G,G′) As the optimal chamfer distance between the global contour boundary G′ of the redirected image and the global contour boundary G of the original image: Where N represents the total number of white pixels 1 in the redirected global contour boundary, and |p i ′-p i | represents the matched redirected image pixel p i ′ and original image pixel p i the distance between them; Step 4.2: Weighting and normalization processing; the change between the salient area of ​​the redirected image and the salient area of ​​the original image is used as the control parameter to obtain the retained contour boundary, using an exponential It is weighted and normalized, where γ represents the sum of the ratios between the maximum width and the maximum height of the salient regions of the original image and the redirected image, Step 4.3: Calculate the global contour distortion score of the redirected image; after weighting and normalization, the global contour distortion evaluation index Q EGS The calculation method of is as follows: Step 5: Quality evaluation of the redirected image. Based on the characteristics of the two types of images, the distortion of the redirected image is described from both global and local perspectives. An effective evaluation mechanism for different types of images is developed, and the quality evaluation score of the image redirection is output.

2. The image redirection quality evaluation method based on global contour matching according to claim 1, characterized in that: In step 2, the global contour of the redirected image is extracted, and the pixel-wise dense matching between the redirected image and the original image is completed using the improved SIFT-flow, which specifically includes: Step 2.1: Implement dense pixel matching between the redirected image and the original image. Use Color-flow to complete dense pixel matching between the redirected image and the original image, and obtain the positions of all pixels in the redirected image in the original image. The solution formula is as follows: Among them, f org and f ret Denote the original image and the redirected image, and p is a pixel in the redirected image. Here, we need to optimize the flow field ω, where u(p) and v(p) are the two components of the flow vector ω(p). Let ε be the global contour space neighborhood extracted from the redirected image. The parameter α is used to control the continuity of the flow field and remove the minimum displacement term in the SIFT energy function. The constants t and d are thresholds set to minimize the constraint flow vector. Step 2.2: Mark the global contour of the original image, and mark the pixel positions of the global contour area of ​​the original image. The marking formula is (2), corresponding to formula (3), and the set of pixel positions of the global contour area of ​​the original image is (p sx ,p sy ), recorded as set L; similarly in formula (4), the obtained reverse dense matching position set (p wx ,p wy ) is marked to obtain a set R; F=(a-1)*W+b (2) L=(p sx -1)*W+p sy (3) R=(p ωx -1)*W+p ωy (4) Among them, p sx , p ωx ={1,2,3,...,n};p sy , p ωy ={1,2,3,...,m}; n≤W, m≤H, W, H are the width and height of the original image respectively; Step 2.3: Get the global contour area of ​​the redirected image. Assume that the set of global contour area points of the original image is L, and the set of global contour pixel positions of the redirected image is R. Then find the intersection C of the set L and the set R, that is, the set of global area points of the redirected image corresponding to the original image, assign a value to the position p(x, y) of the set C of the redirected image, and record the other positions as the set And no value is assigned, so we have: C=L∩R (5).

3. The image redirection quality assessment method based on global contour matching according to claim 1 or 2, characterized in that: In step 2, the global contour boundary map of the redirected image is obtained by combining the matching of the global contour area of ​​the original image, which specifically includes: Step 2.4: Define the distance transformation template. In order to obtain the distance transformation result of the redirected image, we first need to define a distance transformation template mask, as shown in formula (6). The center of the mask is placed on each pixel in the binary image. When the center position of the mask is superimposed on the target feature point in the binary image, its background feature point will be transformed with the position distance of the mask to obtain the distance value at each position. For each pixel of the binary image to be matched, it is iteratively calculated according to formula (7); in, is the transformed distance when the point (i, j) is iterated m times, and the minimum distance value obtained when it is iterated to m times is taken; (k, l) is the position in the mask, the center is (0, 0), and c(k, l) is the local distance to the mask; Step 2.5: Obtain the distance transform of the original image and the redirected image respectively; according to the definition of distance transform, use the global contour binary images of the original image and the redirected image to calculate their distance transforms to finally obtain the matching template and the template to be matched; Step 2.6: Perform global contour matching on the original image and the redirected image. After obtaining the distance transformation between the original image and the redirected image, match the matching template with the template to be matched. The specific operation is to superimpose the feature points of the matching template on the template to be matched.

4. The image redirection quality assessment method based on global contour matching according to claim 1 or 2, characterized in that: In step 4, the distortion measurement ARS of the image block block Specifically include: Based on the image block aspect ratio similarity algorithm, the visual importance map is weighted by utilizing local block changes to evaluate the visual quality of the global content of the redirected image. The calculation formula is as follows: The grid of the image block is divided into 16×16, and the number of the grid is i×j. ij represents the width-to-height ratio of the (i, j)th grid block. The specific calculation formula is shown in (12). V ij represents the weighted average saliency value of the (i, j)th grid block. The detection method uses the saliency map detection method. The grid block width-to-height ratio between the redirected image and the original image is calculated as follows: Among them, R w =W r / W o and R h =H r / H o is the ratio of the height to width of the redirected image grid block to the original image grid block, R m =(R w +R h ) / 2 is the average value of the aspect ratio; K is a constant value.

5. The image redirection quality assessment method based on global contour matching according to claim 1 or 2, characterized in that: In step 4, the forward information loss f FIL The assessment specifically includes: To measure the information loss from the original image to the redirected image, we introduce the saliency loss rate proposed by Hsu. We only consider the information loss from the original image to the redirected image through the forward process. The calculation formula of the information loss is as follows: f FIL Indicates the degree of preservation of global salient information after the image is redirected. When f FIL When the value of is 0, it means that the redirected image has no saved information, and f FIL When the value is 1, it means that the image information is preserved completely.

6. The image redirection quality assessment method based on global contour matching according to claim 1 or 2, characterized in that: In step 4, the backward information loss f SBIL Assessment, including: The backward information loss evaluation is performed on the image with significant foreground to obtain the saliency map of the original image. SIFT-flow is used to establish feature point matching between the redirected image and the source image to obtain the saliency map of the redirected image. The backward information loss evaluation is completed by the saliency information between the saliency map and the saliency map of the original image. The specific calculation formula is as follows: In order to verify the accuracy of the reconstructed image, ψ is set as the consistency weight value, and the similarity between the original image and the reconstructed image is calculated using the triangular similarity formula; the ψ calculation formula is as follows: Where K is a real number 1e-6; I O (p) is the original image, I Rec (p) is the reconstructed image; f BIL The maximum value is 1. When it is equal to 1, it proves that the more pixels the redirected structure can restore, the better its quality.

7. The image redirection quality assessment method based on global contour matching according to claim 1 or 2, characterized in that: In step 3, the original image is classified into a non-salient region class and a salient region class, specifically including: The saliency model PiCANet is introduced to detect the saliency map of the image and classify the input image based on the saliency map. The calculation formula is as follows: Among them, F O is the saliency map extracted by PiCANet, and the saliency value of the pre-existing pixel position at position p is represented as F O (p), N is the total number of pixels with non-zero saliency values, and a threshold T1 is set. If δ is greater than T1, it is determined to be an image group with salient areas, otherwise it is an image group without salient areas.

8. The image redirection quality assessment method based on global contour matching according to claim 1 or 2, characterized in that: In step 5, the NSR redirection image quality evaluation is specifically as follows: NSR images do not have obvious salient areas, and the local contents are relatively similar. It is impossible to compensate for the deficiency of global distortion evaluation by measuring the distortion of salient areas. Only global distortion is measured for such images. The specific calculation formula is as follows: Q NSR =(1-ALL ARS )+0.2×γ×f FIL +(1-Q EGS ) (17) Among them, ALL ARS is the geometric distortion metric; γ is as shown in formula (9), f FIL is a measure of forward information loss, and Q EGS is a global contour distortion measure.

9. The image redirection quality assessment method based on global contour matching according to claim 1 or 2, characterized in that: In step 5, the SR redirected image quality evaluation is specifically as follows: In SR images, (1) global ARS can well measure the overall geometric deformation of the image; (2) global contour boundary Q EGS It can more accurately measure the change in the contour position of the object; (3) The change in the ARS of the salient area depends on its proportion in the image and needs to be limited by multiplying the change in the area of ​​the salient area between the redirected image and the source image; (4) f SBIL The backward information loss of the salient area is achieved by using the color histogram change H between the redirected image and the source image. dist as the weight; the information loss of the redirected image is related to the scaling ratio, and the scaling ratio γ of the redirected image and the salient area of ​​the original image is used for constraint; (5) considering the distortion change of the face block, the specific calculation formula is as follows:

Images