Screen image quality assessment method and related device

Abstract

The present application relates to the field of image processing technology, and proposes a screen image quality assessment method and related devices, which are used to solve the problem of how to simply and efficiently evaluate the screen image quality in the prior art. The present application simulates the display characteristics of the display device through a distorted channel, models the target image and the distorted image through the human eye visual perception channel model, obtains a first visual image simulating the target image seen by the human eye and a second visual image of the distorted image, and determines the image quality of the target image displayed on the display device according to the difference between the first visual image and the second visual image. Thus, by calculating the difference between the first visual image and the second visual image, it simulates the situation where the human eye observes the reference image relative to the target image to quantify the image quality through subjective perception, thereby improving the efficiency of evaluating image quality. By determining the image quality of the target image after being displayed by the display device, the problem that the objective quality assessment method is not suitable for the quality assessment of screen displayed images is overcome.

Description

Background Art

[0002] Screen content images are composite images that contain computer text, computer graphics, and natural images. With the rapid development of the Internet and cloud computing industries, distance education, video conferencing, online virtual screen sharing, and various cloud applications have become widely popular, and most of the images in these applications are screen content images, so it is crucial to accurately measure the quality of screen content images.

[0003] In the related art, there are two main methods for image quality assessment, namely, subjective quality assessment method and objective quality assessment method. The subjective assessment method refers to directly observing the image with the human eye and quantifying the image quality based on the subjective perception of the image information by humans. This assessment method has high accuracy, but it is time-consuming and labor-intensive, and the evaluation process is cumbersome, making it difficult to apply in practice. The objective quality assessment method quantifies the image quality by establishing an evaluation model to simulate the visual perception of the human eye. This method is simple to operate, low in cost, and easy to apply in practice. However, this method is mainly designed for natural images, not for screen content images. Therefore, the problem of how to simply and efficiently evaluate the quality of screen images needs to be solved urgently. Summary of the invention

[0004] The purpose of this application is to provide a screen image quality assessment method to solve the problem of how to simply and efficiently evaluate the screen image quality in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a method for evaluating screen image quality, the method comprising:

[0006] Get the target image;

[0007] Acquire a distorted image of the target image according to a distortion channel; the distortion channel is obtained by modeling a display device;

[0008] Obtaining a first visual image of the target image and a second visual image of the distorted image based on a human eye visual perception channel model respectively from the target image and the distorted image;

[0009] The image quality of the target image after being displayed by the display device is determined based on the difference between the first visual image and the second visual image.

[0010] Optionally, determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image includes:

[0011] determining first image mutual information between the target image and the first visual image, and second image mutual information between the target image and the second visual image;

[0012] Determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information;

[0013] Determining respective gradient information of the target image and the distorted image, and determining the gradient similarity between the target image and the distorted image according to the gradient information;

[0014] A pooling operation is performed on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image.

[0015] Optionally, acquiring the distorted image of the target image according to the distorted channel includes:

[0016] Divide the target image into n*n image blocks;

[0017] Constructing vector expressions of each image block based on the pixel value of each image block;

[0018] The vector representation of each image block is processed based on the distorted channel to obtain the distorted image.

[0019] Optionally, constructing a vector expression of each image block based on the pixel value of each image block includes:

[0020] For any of the image blocks, obtaining a pixel value of the image block and constructing a pixel value vector;

[0021] The pixel value vector is processed by using a Gaussian mixture model to obtain a vector expression of the image block, wherein the Gaussian mixture model is used to convert the pixel value vector of each image block into a normally distributed vector with a covariance of a specified covariance.

[0022] Optionally, the processing the vector expression of each image block based on the distorted channel to obtain the distorted image includes:

[0023] The vector expression of each image block is processed based on the following distorted image model to obtain the distorted image:

[0024] d = g i ×c i ′+V i

[0025] Among them, d represents the vector expression of the distorted image of the image block, c i ′ represents the vector expression of the i-th image block in the frequency domain, g i Represents the scalar noise gain parameter of the i-th image block, V i Represents the independent additive zero-mean Gaussian white noise of the i-th image patch.

[0026] Optionally, the human eye visual perception channel model is:

[0027]

[0028] Among them, Q i represents the i-th image block of the input, and P represents the corresponding image block Q i The corresponding image block in the visual image; if Q i represents the i-th image block in the target image, then P represents the i-th image block in the first visual image; if Q i represents the i-th image block in the distorted image, and P represents the i-th image block in the second visual image.

[0029] Optionally, the first image mutual information and the second image mutual information include mutual information of each image block, and determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information includes:

[0030] The fidelity of the visual information is determined based on the following formula:

[0031]

[0032] Wherein, PSM represents the fidelity of the visual information, M represents the number of image blocks divided from the target image, and f i represents the mutual information of the i-th image block in the distorted image, e i Represents the mutual information of the i-th image block in the target image.

[0033] Optionally, the determining the gradient information of each of the target image and the distorted image, and determining the gradient similarity between the target image and the distorted image according to the gradient information, includes:

[0034] The gradient similarity between the target image and the distorted image is determined based on the following formula:

[0035]

[0036] Among them, G r =r*o h *o v , =d*o h *o v , r represents the target image, d represents the distorted image, * represents the convolution operation, o h represents the horizontal gradient operator, o v represents the vertical gradient operator, G r represents the gradient of the target image, G drepresents the gradient of the distorted image, GSM represents the gradient similarity between the target image and the distorted image, and ε represents a preset parameter.

[0037] Optionally, performing a pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image includes:

[0038] The quality score of the distorted image is calculated based on the following formula:

[0039]

[0040] Among them, GSM i Represents the gradient similarity of the image block, PSM i represents the visual information fidelity of the image block, and K represents the total number of pixels;

[0041] The quality score has a positive correlation with the quality of the target image.

[0042] In a second aspect, the present application proposes a screen image quality assessment device, the device comprising:

[0043] An image acquisition module is configured to acquire a target image;

[0044] A distortion module, configured to obtain a distorted image of the target image according to a distortion channel; the distortion channel is obtained by modeling a display device;

[0045] A human eye visual perception channel module is configured to execute the operation of obtaining a first visual image of the target image and a second visual image of the distorted image based on a human eye visual perception channel model.

[0046] The quality assessment module is configured to determine the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image.

[0047] Optionally, the determining of the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image is performed, and the quality assessment module is configured to perform:

[0048] determining first image mutual information between the target image and the first visual image, and second image mutual information between the target image and the second visual image;

[0049] Determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information;

[0050] Determining respective gradient information of the target image and the distorted image, and determining the gradient similarity between the target image and the distorted image according to the gradient information;

[0051] A pooling operation is performed on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image.

[0052] Optionally, the step of acquiring the distorted image of the target image according to the distorted channel is performed, and the distortion module is configured to perform:

[0053] Divide the target image into n*n image blocks;

[0054] Constructing vector expressions of each image block based on the pixel value of each image block;

[0055] The vector representation of each image block is processed based on the distorted channel to obtain the distorted image.

[0056] Optionally, the step of constructing the vector representation of each image block based on the pixel value of each image block is performed, and the distortion module is configured to perform:

[0057] For any of the image blocks, obtaining a pixel value of the image block and constructing a pixel value vector;

[0058] The pixel value vector is processed by using a Gaussian mixture model to obtain a vector expression of the image block, wherein the Gaussian mixture model is used to convert the pixel value vector of each image block into a normally distributed vector with a covariance of a specified covariance.

[0059] Optionally, the vector expression of each image block is processed based on the distorted channel to obtain the distorted image, and the distortion module is configured to perform:

[0060] The vector expression of each image block is processed based on the following distorted image model to obtain the distorted image:

[0061] d = g i ×c i ′+V i

[0062] Among them, the vector expression of the distorted image of the d-generation image block, c i ′ represents the vector expression of the i-th image block in the frequency domain, g i Represents the scalar noise gain parameter of the i-th image block, V i Represents the independent additive zero-mean Gaussian white noise of the i-th image patch.

[0063] Optionally, the human eye visual perception channel model is:

[0064]

[0065] Among them, Q i represents the i-th image block of the input, and P represents the corresponding image block Q i The corresponding image block in the visual image; if Q i represents the i-th image block in the target image, then P represents the i-th image block in the first visual image; if Q i represents the i-th image block in the distorted image, and P represents the i-th image block in the second visual image.

[0066] Optionally, the first image mutual information and the second image mutual information include mutual information of each image block, and the visual information fidelity module is configured to perform:

[0067] The fidelity of the visual information is determined based on the following formula:

[0068]

[0069] Wherein, PSM represents the fidelity of the visual information, M represents the number of image blocks divided from the target image, and f i represents the mutual information of the i-th image block in the distorted image, e i Represents the mutual information of the i-th image block in the target image.

[0070] Optionally, the step of determining the gradient information of each of the target image and the distorted image, and determining the gradient similarity between the target image and the distorted image according to the gradient information, the gradient similarity module is configured to execute:

[0071] The gradient similarity between the target image and the distorted image is determined based on the following formula:

[0072]

[0073] Among them, G r =r*o h *o v , G d =d*o h *o v , r represents the target image, d represents the distorted image, * represents the convolution operation, o h represents the horizontal gradient operator, o v represents the vertical gradient operator, G r represents the gradient of the target image, G drepresents the gradient of the distorted image, GSM represents the gradient similarity between the target image and the distorted image, and ε represents a preset parameter.

[0074] Optionally, the pooling operation is performed on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image, and the quality assessment module 404 is configured to perform:

[0075] The quality score of the distorted image is calculated based on the following formula:

[0076]

[0077] Among them, GSM i Represents the gradient similarity of the image block, PSM i represents the visual information fidelity of the image block, and K represents the total number of pixels;

[0078] The quality score has a positive correlation with the quality of the target image.

[0079] In a third aspect, the present application further provides an electronic device, including:

[0080] processor;

[0081] a memory for storing instructions executable by the processor;

[0082] The processor is configured to execute the instructions to implement the steps of any method described in the first aspect.

[0083] In a fourth aspect, the present application further provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the steps of any method described in the first aspect.

[0084] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any method described in the first aspect are implemented.

[0085] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0086] The present application simulates the display characteristics of a display device through a distortion channel, and models the target image and the distorted image through a human visual perception channel model, thereby obtaining a first visual image of the target image and a second visual image of the distorted image that simulate what the human eye sees, and calculating the difference between the first visual image and the second visual image to determine the image quality of the target image after being displayed by the display device. Thus, by calculating the difference between the first visual image and the second visual image, the situation of the subjective perception of the human eye observing the reference image relative to the target image to quantify the image quality is simulated, thereby improving the efficiency of evaluating image quality. By determining the image quality of the target image after being displayed by the display device, the problem that the objective quality evaluation method is not suitable for the quality evaluation of the display screen image is overcome.

[0087] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0089] Figure 1 A schematic diagram of a device provided in an embodiment of the present application;

[0090] Figure 2 One of the schematic diagrams of the screen image quality assessment process provided in the embodiment of the present application;

[0091] Figure 3 The second schematic diagram of the screen image quality assessment process provided in the embodiment of the present application;

[0092] Figure 4 A screen image quality assessment device provided in an embodiment of the present application;

[0093] Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0094] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0095] In the description of the embodiments of the present application, unless otherwise specified, the term "multiple" refers to two or more, and other quantifiers are similar and should be understood. The preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments of the present application and the features therein may be combined with each other if there is no conflict.

[0096] Below, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0097] Gaussian Mixture Model (GMM): Use Gaussian probability density function (normal distribution curve) to accurately quantify things. It is a model that decomposes things into several Gaussian probability density function (normal distribution curve).

[0098] Human Visual System (HVS): It is a system that humans use to form vision by perceiving light, and is one of the most critical systems for humans to capture external information.

[0099] Visual Information Fidelity (VIF): is a method of judging image quality based on natural scene statistical models, image distortion, and human visual perception channel models.

[0100] Screen content images are composite images that contain computer text, computer graphics, and natural images. With the rapid development of the Internet and cloud computing industries, distance education, video conferencing, online virtual screen sharing, and various cloud applications have become widely popular, and most of the images in these applications are screen content images, so it is crucial to accurately measure the quality of screen content images.

[0101] In the related art, there are two main methods for image quality assessment, namely, subjective quality assessment method and objective quality assessment method. The subjective assessment method refers to directly observing the image with the human eye and quantifying the image quality based on the subjective perception of the image information by humans. This assessment method has high accuracy, but it is time-consuming and labor-intensive, and the evaluation process is cumbersome, making it difficult to apply in practice. The objective quality assessment method quantifies the image quality by establishing an evaluation model to simulate the visual perception of the human eye. This method is simple to operate, low in cost, and easy to apply in practice. However, this method is mainly designed for natural images, not for screen content images. Therefore, the problem of how to simply and efficiently evaluate the quality of screen images needs to be solved urgently.

[0102] In an embodiment of the present application, the distortion channel simulates the display characteristics of the display device, and models the target image and the distorted image through the human eye visual perception channel model to obtain a first visual image of the target image and a second visual image of the distorted image that simulate the human eye sees, and calculates the difference between the first visual image and the second visual image to determine the image quality of the target image after being displayed by the display device. Thus, by calculating the difference between the first visual image and the second visual image, the situation of quantifying the image quality by subjectively perceiving the reference image relative to the target image by the human eye observation is replaced, thereby improving the efficiency of evaluating image quality. By determining the image quality of the target image after being displayed by the display device, the problem that the objective quality evaluation method is not suitable for the quality evaluation of the display screen image is overcome.

[0103] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application. The method can be executed in the order of the method shown in the embodiment or drawings or in parallel during the actual processing process or when the control device is executed.

[0104] like Figure 1 , which is a schematic diagram of a related device provided in an embodiment of the present application.

[0105] After acquiring the target image r, the target image r′ obtained by modeling with the GMM model is modeled through the distortion channel to obtain the distorted image d. The target image r′ and the distorted image d are passed through the HVS model to obtain the first visual image of the target image r′ and the second visual image of d. The target image r′, the distorted image d, the first visual image and the second visual image are then passed through the mutual information extraction model in the VIF model to calculate the first image mutual information of the target image and the first visual image, and the second image mutual information of the distorted image and the second visual image. The VIF model calculates the visual information fidelity of the distorted image relative to the target image based on the first image mutual information and the second image mutual information.

[0106] At the same time, the target image r and the distorted image d are input into the gradient information extraction model to obtain the gradient of the target image and the gradient of the distorted image respectively. Then, the gradient of the target image and the gradient of the distorted image are calculated by the gradient similarity model to obtain the gradient similarity between the distorted image and the target image. Finally, the gradient similarity and the visual information fidelity are input into the quality assessment model to obtain the quality score of the distorted image relative to the target image.

[0107] For ease of understanding, Figure 2The steps shown further describe a screen image quality assessment method provided in an embodiment of the present application.

[0108] Step 201, obtaining a target image.

[0109] The target image in the embodiment of the present application may be the original digital image content input to the screen for display, the target image may be web page content, video conference content, video document content, etc.

[0110] In the embodiment of the present application, in order to facilitate the evaluation of image quality and improve the accuracy of the evaluation, the obtained target image is divided into n*n image blocks. The pixel values ​​of all pixels in the image block are used as the vector expression c of the image block. If the image is a grayscale image, the grayscale value of the pixel is directly used as the pixel value of the pixel; if the image is a color image, the color image is converted into a grayscale image, and the grayscale value of the grayscale image is used as the pixel value of the pixel.

[0111] Step 202: acquiring a distorted image of the target image according to a distortion channel, wherein the distortion channel is obtained by modeling a display device.

[0112] In the embodiment of the present application, the image data is time domain data. Before obtaining the distorted image of the image block through the distorted channel, the vector expression c of the image block can be converted from the time domain to the frequency domain c′ according to GMM using formula (1).

[0113] c′=u………(1)

[0114] In formula (1), u is the covariance C u A normally distributed vector, “=” represents distributing the vector expression c according to the distribution of the normally distributed vector.

[0115] In the embodiment of the present application, since the image will inevitably be lost during the transmission and acquisition process, in order to make the image as close as possible to the image after the loss, after the target image r is transmitted to the relevant device, it is displayed on the screen of the relevant device. The image displayed on the screen will be distorted due to the loss during the transmission process. Therefore, the distortion channel can be modeled in formula (2) according to the target image and the image displayed on the screen, and the parameter g of the distortion channel can be obtained by modeling. i and V i .

[0116] After obtaining the vector expression c′ of the image block in the frequency domain, c′ is input into the distortion channel according to formula (2) to obtain the vector expression d of the distorted image of the image block.

[0117] d = g i ×c i ′+V i………(2)

[0118] In formula (2), d represents the vector expression of the distorted image of the image block, c i ′ represents the vector expression of the i-th image block in the frequency domain, g i Represents the scalar noise gain parameter of the i-th image block, V i Represents the independent additive zero-mean Gaussian white noise of the i-th image patch.

[0119] Step 203 : obtaining a first visual image of the target image and a second visual image of the distorted image based on a human eye visual perception channel model.

[0120] In the embodiment of the present application, formula (3) is the human eye visual perception channel model provided in the embodiment of the present application.

[0121]

[0122] In formula (2), Q i represents the i-th image block of the input, and P represents the corresponding image block Q i The corresponding image block in the visual image; if Q i represents the i-th image block in the target image, then P is taken as the i-th image block in the first visual image; if Q i represents the i-th image block in the distorted image, and P is the i-th image block in the second visual image.

[0123] Step 204 : determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image.

[0124] The operation of determining the image quality of the screen display can be as follows Figure 3 As shown, the following steps are included:

[0125] Step 301 : determining first image mutual information between a target image and a first visual image, and second image mutual information between the target image and a second visual image.

[0126] In the embodiment of the present application, the method for calculating the mutual information of images is shown in formula (4).

[0127]

[0128] In formula (4), s i represents the i-th image block in the target image, f i represents the i-th image block in the second visual image, e i represents the i-th image block in the first visual image, I represents the identity matrix, σ n and σ vrepresents the fitting parameter, c′ i A vector representation of an image patch converted to the frequency domain.

[0129] Step 302: Determine the visual information fidelity of the distorted image relative to the target image based on the first image mutual information and the second image mutual information.

[0130] In the embodiment of the present application, the first image mutual information and the second image mutual information include mutual information of each image block. After obtaining the first image mutual information and the second image mutual information, the visual information fidelity of the distorted image relative to the target image is calculated in the VIF module according to formula (5).

[0131]

[0132] In formula (5), PSM represents the fidelity of visual information, M represents the number of image blocks divided from the target image, and f i represents the mutual information of the i-th image block in the distorted image, e i Represents the mutual information of the i-th image block in the target image.

[0133] The value of PSM in formula (5) is between [0, 1]. The closer the value of PSM is to 1, the closer the distorted image is to the target image.

[0134] Step 303: Determine the gradient information of the target image and the distorted image respectively, and determine the gradient similarity between the target image and the distorted image according to the gradient information.

[0135] In the embodiment of the present application, the gradient information of the target image and the distorted image is first calculated according to formula (6).

[0136] G r =r*o h *o v , G d =d*o h *o v ………(6)

[0137] In formula (6), r represents the target image, d represents the distorted image, * represents the convolution operation, and o h represents the horizontal gradient operator, o v represents the vertical gradient operator, G r represents the gradient of the target image, G d represents the gradient of the distorted image.

[0138] In the embodiment of the present application, o v and h It can be set based on experience. For example, in this application, If you need to obtain gradient information different from that in this application, you can also change o v and h The value in the matrix or change o v and h The matrix size is not limited in this application.

[0139] After the gradient information of the target image and the distorted image is calculated, the gradient similarity between the target image and the distorted image is determined based on formula (7) in the gradient information extraction model.

[0140]

[0141] In formula (7), ε represents a preset parameter.

[0142] Step 304 , performing a pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image.

[0143] In the embodiment of the present application, based on the quality score of the distorted image in formula (8), the quality of the target image is judged according to the quality score, wherein the quality score has a positive correlation with the quality of the target image.

[0144] The quality score of the distorted image is calculated based on the following formula:

[0145]

[0146] In formula (8), GSM i Represents the gradient similarity of image blocks, PSM i represents the fidelity of the visual information of the image block, and K represents the total number of pixels.

[0147] In this application, the distortion at the edge of the image can be taken into account by calculating the gradient similarity, and the gradient of the image can sensitively reflect the degree of distortion of the image. The VIF model can simulate the visual effects of the target image and the reference image in the human eye, and calculate the quality score of the distorted image based on the obtained PSM value and GSM value. Therefore, this application can simulate the human eye to obtain the visual information fidelity of the distorted image relative to the target image in real time, and achieve a more accurate evaluation of the distorted image based on the visual information fidelity and gradient similarity.

[0148] Based on the same inventive concept, the present application also provides a screen image quality assessment device 400, such as Figure 4 As shown, the device comprises:

[0149] The image acquisition module 401 is configured to acquire a target image;

[0150] The distortion module 402 is configured to obtain a distorted image of the target image according to a distortion channel; the distortion channel is obtained by modeling a display device;

[0151] The human eye visual perception channel module 403 is configured to execute the acquisition of a first visual image of the target image and a second visual image of the distorted image based on a human eye visual perception channel model.

[0152] The quality assessment module 404 is configured to determine the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image.

[0153] Optionally, to perform the step of determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image, the quality assessment module 404 is configured to perform:

[0154] determining first image mutual information between the target image and the first visual image, and second image mutual information between the target image and the second visual image;

[0155] Determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information;

[0156] Determining respective gradient information of the target image and the distorted image, and determining the gradient similarity between the target image and the distorted image according to the gradient information;

[0157] A pooling operation is performed on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image.

[0158] Optionally, the step of acquiring the distorted image of the target image according to the distorted channel is performed, and the distortion module 402 is configured to perform:

[0159] Divide the target image into n*n image blocks;

[0160] Constructing vector expressions of each image block based on the pixel value of each image block;

[0161] The vector representation of each image block is processed based on the distorted channel to obtain the distorted image.

[0162] Optionally, to perform the step of constructing the vector representation of each image block based on the pixel value of each image block, the distortion module 402 is configured to perform:

[0163] For any of the image blocks, obtaining a pixel value of the image block and constructing a pixel value vector;

[0164] The pixel value vector is processed by using a Gaussian mixture model to obtain a vector expression of the image block, wherein the Gaussian mixture model is used to convert the pixel value vector of each image block into a normally distributed vector with a covariance of a specified covariance.

[0165] Optionally, the vector expression of each image block is processed based on the distorted channel to obtain the distorted image, and the distortion module 402 is configured to perform:

[0166] The vector expression of each image block is processed based on the following distorted image model to obtain the distorted image:

[0167] d = g i ×c i ′+V i

[0168] Among them, the vector expression of the distorted image of the d-generation image block, c i ′ represents the vector expression of the i-th image block in the frequency domain, g i Represents the scalar noise gain parameter of the i-th image block, V i Represents the independent additive zero-mean Gaussian white noise of the i-th image patch.

[0169] Optionally, the human eye visual perception channel model is:

[0170]

[0171] Among them, Q i represents the i-th image block of the input, and P represents the corresponding image block Q i The corresponding image block in the visual image; if Q i represents the i-th image block in the target image, then P represents the i-th image block in the first visual image; if Q i represents the i-th image block in the distorted image, and P represents the i-th image block in the second visual image.

[0172] Optionally, the first image mutual information and the second image mutual information include mutual information of each image block, and the visual information fidelity module 405 is configured to perform:

[0173] The fidelity of the visual information is determined based on the following formula:

[0174]

[0175] Wherein, PSM represents the fidelity of the visual information, M represents the number of image blocks divided from the target image, and f i represents the mutual information of the i-th image block in the distorted image, e i Represents the mutual information of the i-th image block in the target image.

[0176] Optionally, the step of determining the gradient information of each of the target image and the distorted image, and determining the gradient similarity between the target image and the distorted image according to the gradient information, the gradient similarity module 406 is configured to execute:

[0177] The gradient similarity between the target image and the distorted image is determined based on the following formula:

[0178]

[0179] Among them, G r =r*o h *o v , G d =d*o h *o v , r represents the target image, d represents the distorted image, * represents the convolution operation, o h represents the horizontal gradient operator, o v represents the vertical gradient operator, G r represents the gradient of the target image, G d represents the gradient of the distorted image, GSM represents the gradient similarity between the target image and the distorted image, and ε represents a preset parameter.

[0180] Optionally, the pooling operation is performed on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image, and the quality assessment module 404 is configured to perform:

[0181] The quality score of the distorted image is calculated based on the following formula:

[0182]

[0183] Among them, GSM i Represents the gradient similarity of the image block, PSM i represents the visual information fidelity of the image block, and K represents the total number of pixels;

[0184] The quality score has a positive correlation with the quality of the target image.

[0185] After introducing the abnormal event detection method and electronic device according to the exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0186] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0187] In some possible implementations, the electronic device according to the present application may include at least one processor and at least one memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps in the search method for monitoring nodes according to various exemplary implementations of the present application described above in this specification. For example, the processor may execute the steps in the search method for monitoring nodes.

[0188] Refer to the following Figure 5 The electronic device 130 according to this embodiment of the present application is described. Figure 5 The electronic device 130 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0189] like Figure 5 As shown, the electronic device 130 is in the form of a general electronic device. The components of the electronic device 130 may include but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

[0190] Bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0191] The memory 132 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and / or a cache memory 1322 , and may further include a read-only memory (ROM) 1323 .

[0192] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0193] The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 130, and / or communicate with any device that enables the electronic device 130 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 135. Furthermore, the electronic device 130 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 136. As shown, the network adapter 136 communicates with other modules for the electronic device 130 via a bus 133. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0194] In some possible implementations, various aspects of a monitoring node search method provided by the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps in a monitoring method described above in this specification according to various exemplary implementations of the present application.

[0195] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0196] The program product for monitoring of the embodiment of the present application can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on an electronic device. However, the program product of the present application is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, an apparatus or a device or used in combination with it.

[0197] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0198] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0199] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user electronic device, partially on the user device, as an independent software package, partially on the user electronic device and partially on the remote electronic device, or entirely on the remote electronic device or server. In the case of a remote electronic device, the remote electronic device can be connected to the user electronic device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external electronic device (for example, using an Internet service provider to connect through the Internet).

[0200] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0201] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0202] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0203] The present application is described with reference to the flowcharts and block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and block diagram, as well as the combination of the processes and boxes in the flowchart and block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and block diagram. Figure 1 Process or multiple processes and boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0204] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 Process or multiple processes and boxes Figure 1 A function specified in one or more boxes.

[0205] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 Process or multiple processes and boxes Figure 1 The steps for the functions specified in one or more boxes.

[0206] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0207] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A screen image quality assessment method, characterized in that: The method comprises: Get the target image; Acquire a distorted image of the target image according to a distortion channel; the distortion channel is obtained by modeling a display device; Obtaining a first visual image of the target image and a second visual image of the distorted image based on a human eye visual perception channel model for the target image and the distorted image, respectively; determining the image quality of the target image after being displayed by the display device based on the difference between the first visual image and the second visual image; The determining, based on the difference between the first visual image and the second visual image, the image quality of the target image after being displayed by the display device comprises: determining first image mutual information between the target image and the first visual image, and second image mutual information between the target image and the second visual image; Determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information; Determining respective gradient information of the target image and the distorted image, and determining the gradient similarity between the target image and the distorted image according to the gradient information; A pooling operation is performed on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image.

2. The method according to claim 1, characterized in that The step of acquiring the distorted image of the target image according to the distorted channel includes: Divide the target image into n*n image blocks; Constructing vector expressions of each image block based on the pixel value of each image block; The vector representation of each image block is processed based on the distorted channel to obtain the distorted image.

3. The method according to claim 2, characterized in that The constructing of vector expressions of the image blocks based on the pixel values ​​of the image blocks respectively includes: For any of the image blocks, obtaining a pixel value of the image block and constructing a pixel value vector; The pixel value vector is processed by using a Gaussian mixture model to obtain a vector expression of the image block, wherein the Gaussian mixture model is used to convert the pixel value vector of each image block into a normally distributed vector with a covariance of a specified covariance.

4. The method according to claim 2, characterized in that: Processing the vector expression of each image block based on the distorted channel to obtain the distorted image includes: The vector expression of each image block is processed based on the following distorted image model to obtain the distorted image: d=g i ×c i ′ +V i Among them, d represents the vector expression of the distorted image of the image block, c i ′ represents the vector expression of the i-th image block in the frequency domain, g i Represents the scalar noise gain parameter of the i-th image block, V i Represents the independent additive zero-mean Gaussian white noise of the i-th image patch.

5. The method according to claim 1, characterized in that The human eye visual perception channel model is: Among them, Q i represents the i-th image block of the input, and P represents the corresponding image block Q i The corresponding image block in the visual image; if Q i represents the i-th image block in the target image, then P represents the i-th image block in the first visual image; if Q i represents the i-th image block in the distorted image, and P represents the i-th image block in the second visual image.

6. The method according to claim 1, characterized in that The first image mutual information and the second image mutual information include mutual information of each image block, and determining the visual information fidelity of the distorted image relative to the target image according to the first image mutual information and the second image mutual information includes: The fidelity of the visual information is determined based on the following formula: Wherein, PSM represents the fidelity of the visual information, M represents the number of image blocks divided from the target image, and f i represents the mutual information of the i-th image block in the distorted image, e i Represents the mutual information of the i-th image block in the target image.

7. The method according to claim 6, characterized in that The determining of the gradient information of the target image and the distorted image, and determining the gradient similarity between the target image and the distorted image according to the gradient information, comprises: The gradient similarity between the target image and the distorted image is determined based on the following formula: Among them, G r =r*o h *o v , G d =d*o h *o v , r represents the target image, d represents the distorted image, * represents the convolution operation, o h represents the horizontal gradient operator, o v represents the vertical gradient operator, G r represents the gradient of the target image, G d represents the gradient of the distorted image, GSM represents the gradient similarity between the target image and the distorted image, and ε represents a preset parameter.

8. The method according to claim 7, characterized in that The performing a pooling operation on the distorted image according to the visual information fidelity and the gradient similarity to obtain a quality score of the distorted image includes: The quality score of the distorted image is calculated based on the following formula: Among them, GSM i Represents the gradient similarity of the image block, PSM i represents the visual information fidelity of the image block, and K represents the total number of pixels; The quality score has a positive correlation with the quality of the target image.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the steps of the method as described in any one of claims 1-8.

Images