Image quality determination method, device, equipment and storage medium

Abstract

The present application discloses an image quality determination method, apparatus, device, and storage medium, which are applied to the field of image processing technology. The method first obtains a depth image and a color image corresponding to the depth image, calculates the first image weight of the jth pixel of the i-th pixel in the depth image and the second image weight of the jth pixel of the i-th pixel in the color image; calculates the parameters of the structural area of ​​the i-th pixel based on the first image weight and the second image weight; then calculates the first weight matrix of the i-th pixel in the depth image and the second weight matrix of the j-th pixel; calculates the parameters of the smooth area of ​​the i-th pixel based on the first weight matrix and the second weight matrix; finally, obtains the quality parameters of the depth image based on the parameters of the structural area of ​​the i-th pixel and the parameters of the smooth area of ​​the i-th pixel. In this way, the overall quality parameters of the depth image can be determined more comprehensively and accurately.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image quality determination method, apparatus, device and storage medium. Background Art

[0002] A depth image, also known as a range image, is an image whose pixel values ​​are the distance from the image collector to each point in the scene. Depth images can reveal the geometry of visible surfaces and are used for facial recognition and robotics' three-dimensional environmental perception. Depth images can be generated using physical sensors and stereo matching algorithms.

[0003] Currently, the depth image obtained has a certain degree of distortion. How to more accurately measure the quality of the depth image is a problem that needs to be solved. Summary of the Invention

[0004] In view of this, the present application provides an image quality determination method, apparatus, device and storage medium, which can obtain relatively accurate quality parameters of depth images and achieve relatively accurate measurement of the image quality of depth images.

[0005] To solve the above problems, the technical solutions provided by this application are as follows:

[0006] In a first aspect, the present application provides a method for determining image quality, the method comprising:

[0007] Acquire a depth image and a color image corresponding to the depth image, wherein the depth image includes a structured area and a smooth area;

[0008] Calculate a first image weight of a j-th pixel of the i-th pixel of the depth image and a second image weight of a j-th pixel of the i-th pixel of the color image, where the j-th pixel is the j-th pixel among a plurality of pixels included in a neighborhood centered on the i-th pixel, where the value of i is from 1 to less than or equal to the number of pixels included in the depth image, and the value of j is from 1 to less than or equal to the number of pixels included in the neighborhood centered on the i-th pixel;

[0009] Calculate the parameters of the structure area of ​​the i-th pixel according to the first image weight and the second image weight;

[0010] Calculate a first weight matrix for the i-th pixel and a second weight matrix for the j-th pixel of the depth image;

[0011] Calculate the parameters of the smoothing area of ​​the i-th pixel according to the first weight matrix and the second weight matrix;

[0012] A quality parameter of the depth image is obtained according to the parameters of the structural area of ​​the i-th pixel and the parameters of the smooth area of ​​the i-th pixel, where the quality parameter is used to measure the image quality of the depth image.

[0013] In a possible implementation, calculating the parameters of the structure area of ​​the i-th pixel according to the first image weight and the second image weight includes:

[0014] Calculating the product of the first image weight of the j-th pixel of the i-th pixel and the second image weight of the j-th pixel of the i-th pixel to obtain first data of the j-th pixel of the i-th pixel;

[0015] Calculate the sum of the first data of each j-th pixel point of the i-th pixel point to obtain second data;

[0016] Calculating the sum of the first image weights of each j-th pixel point of the i-th pixel point to obtain third data;

[0017] The ratio of the second data to the third data is calculated to obtain the parameters of the structural area of ​​the i-th pixel.

[0018] In a possible implementation, calculating the parameters of the smoothing area of ​​the i-th pixel according to the first weight matrix and the second weight matrix includes:

[0019] Calculating the square of the difference between the first weight matrix of the i-th pixel point and the second weight matrix of the j-th pixel point to obtain fourth data of the j-th pixel point;

[0020] The sum of the fourth data of each j-th pixel point is calculated to obtain the parameter of the smoothing area of ​​the i-th pixel point.

[0021] In a possible implementation, obtaining the quality parameter of the depth image according to the parameter of the structure area of ​​the i-th pixel and the parameter of the smooth area of ​​the i-th pixel includes:

[0022] Calculate the product of the parameter of the structure area of ​​the i-th pixel and the first weight of the i-th pixel to obtain fifth data of the i-th pixel;

[0023] Calculate the sum of the fifth data of each i-th pixel point to obtain sixth data;

[0024] Calculate the sum of the first weights of each i-th pixel point to obtain seventh data;

[0025] Calculate the product of the parameter of the smoothing area of ​​the i-th pixel and the second weight of the i-th pixel to obtain the eighth data of the i-th pixel;

[0026] Calculate the sum of the eighth data of each i-th pixel point to obtain the ninth data;

[0027] Calculate the sum of the second weights of each i-th pixel point to obtain the tenth data;

[0028] A sum of a ratio of the sixth data to the seventh data and a ratio of the ninth data to the tenth data is calculated to obtain a quality parameter of the depth image.

[0029] In one possible implementation, the first weight is the difference between 1 and the second weight. If the parameter of the structural area of ​​the i-th pixel point is greater than a threshold, the first weight is 1; if the parameter of the structural area of ​​the i-th pixel point is greater than the threshold, the first weight is 0.

[0030] In a second aspect, the present application provides an image quality determination device, the device comprising:

[0031] An acquisition unit, configured to acquire a depth image and a color image corresponding to the depth image, wherein the depth image includes a structured area and a smooth area;

[0032] a first calculation unit, configured to calculate a first image weight of a j-th pixel of an i-th pixel of the depth image and a second image weight of a j-th pixel of an i-th pixel of the color image, where the j-th pixel is the j-th pixel among a plurality of pixels included in a neighborhood centered on the i-th pixel, wherein the value of i ranges from 1 to less than or equal to the number of pixels included in the depth image, and the value of j ranges from 1 to less than or equal to the number of pixels included in the neighborhood centered on the i-th pixel;

[0033] a second calculating unit, configured to calculate a parameter of a structural region of an i-th pixel according to the first image weight and the second image weight;

[0034] A third calculation unit is used to calculate a first weight matrix of an i-th pixel point and a second weight matrix of a j-th pixel point of the depth image;

[0035] a fourth calculation unit, configured to calculate parameters of a smoothing area of ​​an i-th pixel point according to the first weight matrix and the second weight matrix;

[0036] A determination unit is configured to obtain a quality parameter of the depth image based on a parameter of the structural area of ​​the i-th pixel and a parameter of the smooth area of ​​the i-th pixel, wherein the quality parameter is used to measure the image quality of the depth image.

[0037] In one possible implementation, the second calculation unit is specifically used to calculate the product of the first image weight of the jth pixel point of the i-th pixel point and the second image weight of the jth pixel point of the i-th pixel point to obtain the first data of the j-th pixel point of the i-th pixel point; calculate the sum of the first data of each j-th pixel point of the i-th pixel point to obtain the second data; calculate the sum of the first image weights of each j-th pixel point of the i-th pixel point to obtain the third data; calculate the ratio of the second data and the third data to obtain the parameters of the structural area of ​​the i-th pixel point.

[0038] In one possible implementation, the fourth calculation unit is specifically used to calculate the square of the difference between the first weight matrix of the i-th pixel point and the second weight matrix of the j-th pixel point to obtain the fourth data of the j-th pixel point; and calculate the sum of the fourth data of each j-th pixel point to obtain the parameters of the smoothing area of ​​the i-th pixel point.

[0039] In one possible implementation, the determination unit is specifically used to calculate the product of the parameter of the structural area of ​​the i-th pixel and the first weight of the i-th pixel to obtain the fifth data of the i-th pixel; calculate the sum of the fifth data of each i-th pixel to obtain the sixth data; calculate the sum of the first weights of each i-th pixel to obtain the seventh data; calculate the product of the parameter of the smooth area of ​​the i-th pixel and the second weight of the i-th pixel to obtain the eighth data of the i-th pixel; calculate the sum of the eighth data of each i-th pixel to obtain the ninth data; calculate the sum of the second weights of each i-th pixel to obtain the tenth data; calculate the ratio of the sixth data to the seventh data, and the sum of the ratio of the ninth data to the tenth data to obtain the quality parameter of the depth image.

[0040] In one possible implementation, the first weight is the difference between 1 and the second weight. If the parameter of the structural area of ​​the i-th pixel point is greater than a threshold, the first weight is 1; if the parameter of the structural area of ​​the i-th pixel point is greater than the threshold, the first weight is 0.

[0041] In a third aspect, the present application provides an image quality determination device, comprising: a processor, a memory, and a system bus;

[0042] The processor and the memory are connected via the system bus;

[0043] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes the method described in the first aspect.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions. When the instructions are executed on a terminal device, the terminal device executes the method described in the first aspect above.

[0045] It can be seen that this application has the following beneficial effects:

[0046] The present application provides an image quality determination method, apparatus, device, and storage medium. The method first obtains a depth image and a color image corresponding to the depth image, calculates a first image weight of the jth pixel of the i-th pixel in the depth image, and a second image weight of the jth pixel of the i-th pixel in the color image, wherein the j-th pixel is the j-th pixel among a plurality of pixels in a neighborhood centered on the i-th pixel, i ranges from 1 to less than or equal to the number of pixels in the depth image, and j ranges from 1 to less than or equal to the number of pixels in the neighborhood centered on the i-th pixel; calculates parameters of a structural region of the i-th pixel based on the first image weight and the second image weight; then calculates a first weight matrix of the i-th pixel in the depth image and a second weight matrix of the j-th pixel; calculates parameters of a smooth region of the i-th pixel based on the first weight matrix and the second weight matrix; and finally, obtains a quality parameter of the depth image based on the parameters of the structural region of the i-th pixel and the parameters of the smooth region of the i-th pixel. The quality parameter is used to measure the image quality of the depth image. In this way, based on the quality of the smooth area and the quality of the structured area of ​​the depth image, the quality parameter of the entire depth image can be determined more comprehensively and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flowchart of a method for determining image quality provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the structure of an image quality determination device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to facilitate the understanding and explanation of the technical solutions provided by the embodiments of the present application, the relevant terms involved in the present application will be explained below.

[0050] A color image refers to an image in which each pixel information is composed of the three primary colors RGB (red, green, blue).

[0051] GT (Ground-truth, reference standard) image, represented as a depth image without any distortion.

[0052] Structural area: The depth information of the depth image will produce a sudden change at the edge of the object. This area is called the structural area of ​​the depth image.

[0053] Smooth area: There are a large number of areas with the same grayscale value inside the depth image, which is called the smooth area of ​​the depth image.

[0054] Distortion detection: that is, detecting the corresponding erroneous pixels in the depth image through corresponding methods. The area composed of multiple erroneous pixels is called the distorted area.

[0055] No reference: refers to the absence of a reference Ground-truth (GT) image.

[0056] The following will first explain the background technology of the relevant terms involved in this application.

[0057] There are generally two types of depth image quality assessment methods: one with a reference, namely the full-reference depth image quality assessment method and the other with a half-reference depth image quality assessment method. The reference-based depth image quality assessment method requires a reference image, i.e., an error-free reference image. By comparing the reference image with the distorted depth image, the corresponding distortion detection result can be obtained, which is then used as the quality score for evaluating the depth image.

[0058] However, the quality assessment methods for depth images with references are limited by the difficulty in obtaining true value images, making them difficult to be widely used. Current quality assessment methods for depth images without references also struggle to obtain relatively accurate quality parameters.

[0059] Based on this, the embodiments of the present application provide an image quality determination method, apparatus, device and storage medium, which first obtains a depth image and a color image corresponding to the depth image, calculates the first image weight of the jth pixel of the i-th pixel of the depth image and the second image weight of the jth pixel of the i-th pixel of the color image, wherein the j-th pixel is the j-th pixel among the multiple pixels included in the neighborhood centered on the i-th pixel, i is a value from 1 to less than or equal to the number of pixels included in the depth image, and j is a value from 1 to less than or equal to the number of pixels included in the neighborhood centered on the i-th pixel; calculates the parameters of the structural area of ​​the i-th pixel based on the first image weight and the second image weight; then calculates the first weight matrix of the i-th pixel of the depth image and the second weight matrix of the j-th pixel; calculates the parameters of the smooth area of ​​the i-th pixel based on the first weight matrix and the second weight matrix; finally, obtains the quality parameter of the depth image based on the parameters of the structural area of ​​the i-th pixel and the parameters of the smooth area of ​​the i-th pixel. The quality parameter is used to measure the image quality of the depth image. In this way, based on the quality of the smooth area and the quality of the structured area of ​​the depth image, the quality parameter of the entire depth image can be determined more comprehensively and accurately.

[0060] To facilitate understanding of the technical solution provided by the embodiment of the present application, an image quality determination method provided by the embodiment of the present application is described below with reference to the accompanying drawings.

[0061] See also Figure 1 As shown, this figure is a flow chart of an image quality determination method provided by an embodiment of the present application. The figure includes S101-S106:

[0062] S101: Acquire a depth image and a color image corresponding to the depth image, wherein the depth image includes a structure area and a smooth area.

[0063] The color image corresponding to the depth image is a color image at the same viewpoint as the depth image. The color image can be acquired by a device that acquires the depth image.

[0064] The structure of the color image and the depth image corresponding to the depth image are highly consistent. Using the edge information of the color image as a reference, the edge information error in the distorted depth image is measured to evaluate the quality of the depth image.

[0065] S102: Calculate a first image weight of a j-th pixel of an i-th pixel of the depth image and a second image weight of a j-th pixel of an i-th pixel of the color image.

[0066] The i-th pixel of the depth image is the i-th pixel included in the depth image, sorted in a preset order. The value of i ranges from 1 to less than or equal to the number of pixels included in the depth image. The position of the i-th pixel of the color image in the color image is the same as the position of the i-th pixel of the depth image in the depth image, that is, the i-th pixel of the color image is the pixel in the color image corresponding to the position of the i-th pixel of the depth image.

[0067] The jth pixel of the i-th pixel is the j-th pixel among the multiple pixels in the neighborhood centered on the i-th pixel. The value of j ranges from 1 to less than or equal to the number of pixels in the neighborhood centered on the i-th pixel.

[0068] In one possible implementation, Gaussian weights are used to obtain structural information of the color image and the depth image, and a first image weight of the jth pixel of the i-th pixel of the depth image and a second image weight of the jth pixel of the i-th pixel of the color image are obtained.

[0069] The first image weight of the jth pixel of the i-th pixel in the depth image See formula (1):

[0070]

[0071] The second image weight of the jth pixel of the i-th pixel of the color image See formula (2):

[0072]

[0073] The weight values ​​of the first image weight and the second image weight range from [0 to 1]. When the weight value is 1, it indicates that pixel i and pixel j are in the same region, that is, the spatial structures of pixel i and pixel j are consistent. When the weight value is 0, it indicates that pixel i and pixel j are in different regions, that is, the spatial structures of pixel i and pixel j are inconsistent. The relationship between the central pixel i and each pixel in its neighborhood can be used to determine regions with consistent and inconsistent structures.

[0074] For structural regions, when a pixel in the depth image is located in a different region than the pixel at the same location in the corresponding color image, this pixel is called a color-depth inconsistency point. The region consisting of the set of inconsistency points is called the distortion region.

[0075] S103: Calculate and obtain parameters of the structure area of ​​the i-th pixel according to the first image weight and the second image weight.

[0076] According to the obtained first image weight and second image weight, the first image weight and the second image weight can be quantitatively analyzed, and then the pixel points with inconsistent structures can be determined according to the first image weight and the geothermal image weight, thereby determining the distorted area.

[0077] In a possible implementation manner, a weighted sum of the first image weight and the second image weight is calculated as a parameter of the structure region of the i-th pixel.

[0078] In another possible implementation, the embodiment of the present application provides a specific implementation method for calculating the parameters of the structure area of ​​the i-th pixel point based on the first image weight and the second image weight, including the following four steps:

[0079] Calculating the product of the first image weight of the j-th pixel of the i-th pixel and the second image weight of the j-th pixel of the i-th pixel to obtain first data of the j-th pixel of the i-th pixel;

[0080] Calculate the sum of the first data of each j-th pixel point of the i-th pixel point to obtain second data;

[0081] Calculating the sum of the first image weights of each j-th pixel point of the i-th pixel point to obtain third data;

[0082] The ratio of the second data to the third data is calculated to obtain the parameters of the structural area of ​​the i-th pixel.

[0083] The parameters of the structural area of ​​the i-th pixel can be calculated using formula (3).

[0084]

[0085] Here, ω represents the neighborhood. The product of the first image weight of the jth pixel of the i-th pixel and the second image weight of the jth pixel of the i-th pixel is the first data of the j-th pixel of the i-th pixel. Then calculate the sum of the first image weights of each j-th pixel of the i-th pixel, that is, sum the first data of each pixel in the neighborhood centered on the i-th pixel to obtain the second data

[0086] Then calculate the sum of the first image weights of each j-th pixel of the i-th pixel, that is, sum the first image weights of each pixel in the neighborhood centered on the i-th pixel, and get the third data

[0087] After obtaining the second data and the third data, the ratio of the second data to the third data is calculated to obtain a weighted average value, which is the parameter W of the structural area of ​​the i-th pixel point.i Inc .

[0088] W i Inc Its range is [0,1], where W i Inc The larger the value of (not exceeding 1), the closer the structure of the depth image and the color image of the i-th pixel is, and the smaller the distortion is; on the contrary, when W i Inc The smaller the value (not less than 0), the greater the structural difference between the depth image and the color image of the i-th pixel, and the more serious the distortion.

[0089] S104: Calculate a first weight matrix for the i-th pixel point and a second weight matrix for the j-th pixel point of the depth image.

[0090] Smooth regions of depth images also exhibit various types of distortion, such as noise and artifacts. Determining the quality of smooth regions of depth images still relies on measuring the distortion in these regions. The first weight matrix for the i-th pixel and the second weight matrix for the j-th pixel in the depth image can be Gaussian weights.

[0091] S105: Calculate the parameters of the smoothing area of ​​the i-th pixel according to the first weight matrix and the second weight matrix.

[0092] The imaging principle of depth images shows that they contain a large number of homogeneous regions, or smooth regions, with identical pixel values. Pixels with identical weights in the depth image's Gaussian weights are considered smooth regions. For a pixel in a depth image, if the Gaussian weights of all pixels in a neighborhood centered on it are the same as the Gaussian weight of the center pixel, the pixel is considered smooth. Otherwise, the pixel is considered non-smooth.

[0093] In this way, based on the calculated first weight matrix and the second weight matrix, it is possible to determine whether the i-th pixel point is a smooth point, and further whether the i-th pixel point belongs to a distorted area.

[0094] In one possible implementation, the embodiment of the present application provides a specific implementation method for calculating the parameters of the smoothing area of ​​the i-th pixel point based on the first weight matrix and the second weight matrix, including:

[0095] Calculating the square of the difference between the first weight matrix of the i-th pixel point and the second weight matrix of the j-th pixel point to obtain fourth data of the j-th pixel point;

[0096] The sum of the fourth data of each j-th pixel point is calculated to obtain the parameter of the smoothing area of ​​the i-th pixel point.

[0097] The following describes a specific implementation method for calculating the parameters of the smoothing area of ​​the i-th pixel point based on the first weight matrix and the second weight matrix in conjunction with formula (4).

[0098]

[0099] in, is the parameter of the smoothing area of ​​the i-th pixel. is the square of the difference between the first weight matrix of the i-th pixel and the second weight matrix of the j-th pixel, that is, the fourth data of the j-th pixel. Calculate the sum of the fourth data of each pixel in the neighborhood centered on the i-th pixel, that is, the sum of each j-th pixel, and obtain That is, the parameter of the smoothing area of ​​the i-th pixel.

[0100] ω represents the window size. The value of ω can be determined based on the degree of distortion of the smooth area in the depth image. In one possible implementation, a smaller ω is used so that the parameters of the smooth area of ​​the i-th pixel are more accurate.

[0101] S106: Obtain a quality parameter of the depth image according to the parameter of the structural region of the i-th pixel and the parameter of the smooth region of the i-th pixel, where the quality parameter is used to measure the image quality of the depth image.

[0102] After obtaining the parameters of the structural area of ​​the i-th pixel and the parameters of the smooth area of ​​the i-th pixel, the quality parameter of the i-th pixel can be obtained. Then, the quality parameter of the depth image is obtained based on the quality parameters of each i-th pixel. According to the quality parameter of the depth image, the image quality of the depth image can be determined. In some possible implementations, if the quality parameter of the depth image indicates that the image quality of the depth image is good, the depth image can be used to perform subsequent tasks. In another possible implementation, if the quality parameter of the depth image indicates that the image quality of the depth image is good or poor, the process of obtaining the depth image can be improved, or the depth image can be deleted to avoid affecting subsequent tasks.

[0103] The embodiments of the present application provide two specific implementation methods for obtaining the quality parameters of the depth image based on the parameters of the structural area of ​​the i-th pixel point and the parameters of the smooth area of ​​the i-th pixel point. Please refer to the following for details.

[0104] Based on the above steps S101-S106, we can see that the degree of distortion is measured for both the structural and smooth regions of the depth image, and corresponding parameters are obtained. The overall quality parameter of the depth image is then determined based on the parameters for the structural and smooth regions. This allows for a more comprehensive evaluation of the depth image's quality.

[0105] In one possible implementation, the embodiment of the present application provides a specific implementation method for obtaining the quality parameter of the depth image based on the parameters of the structural area of ​​the i-th pixel point and the parameters of the smooth area of ​​the i-th pixel point, including the following steps:

[0106] Calculate the product of the parameter of the structure area of ​​the i-th pixel and the first weight of the i-th pixel to obtain fifth data of the i-th pixel;

[0107] Calculate the sum of the fifth data of each i-th pixel point to obtain sixth data;

[0108] Calculate the sum of the first weights of each i-th pixel point to obtain seventh data;

[0109] Calculate the product of the parameter of the smoothing area of ​​the i-th pixel and the second weight of the i-th pixel to obtain the eighth data of the i-th pixel;

[0110] Calculate the sum of the eighth data of each i-th pixel point to obtain the ninth data;

[0111] Calculate the sum of the second weights of each i-th pixel point to obtain the tenth data;

[0112] A sum of a ratio of the sixth data to the seventh data and a ratio of the ninth data to the tenth data is calculated to obtain a quality parameter of the depth image.

[0113] The specific implementation method of calculating the quality parameter of the depth image is described below in conjunction with formula (5).

[0114]

[0115] Among them, W i Inc is the parameter of the structural area of ​​the i-th pixel, (1-W i Inc_T ) is the first weight. (1-W i Inc_T )W i Inc It is the product of the parameter of the structure area of ​​the i-th pixel and the first weight of the i-th pixel, that is, the fifth data of the i-th pixel. It is the sixth data.

[0116] The seventh data.

[0117] W i NotSmooth is the parameter of the smoothing area of ​​the i-th pixel. i Inc_T is the second weight. (1-W i Inc_T )W i NotSmooth The eighth data of the i-th pixel. It is the ninth data.

[0118] This is the tenth data.

[0119] In one possible implementation, W i Inc_T It can be determined according to formula (6).

[0120]

[0121] Wherein, Threshold is a pre-set threshold value used to measure whether the i-th pixel point is a consistent point or an inconsistent point. i Inc is the consistency result of the i-th pixel point in the structure area of ​​the depth image. If the i-th pixel point is a consistency point, then W i Inc The weight value of is relatively large, close to 1, indicating that the distortion is relatively light; on the contrary, if the i-th pixel is an inconsistent point, then W i Inc The weight value is relatively small, close to 0.

[0122] So, W i Inc_T When the output of is 1, it means that the i-th pixel is a consistent point. i Inc_T When the output of is 0, it means that the i-th pixel is an inconsistent point.

[0123] From formula (5), we can see that due to W i Inc_T Only the distortion problem of the structure area can be detected, but the distortion problem of the smooth area cannot be detected. In some possible implementations, when there is no distortion problem in the structure area of ​​the depth image, that is, W i Inc_T The value of is 1, and the first half of formula (5) is 0, which means that in the current case, the quality of the depth image is determined by the quality of its smooth area. In other possible implementations, the structural area of ​​the depth image has distortion problems, so W i Inc _TThe value of is 0, and the quality parameters of the depth image are determined by the parameters of the structure area and the parameters of the smooth area.

[0124] Based on the image quality determination method provided in the above method embodiment, the embodiment of the present application also provides an image quality determination device, which will be described below with reference to the accompanying drawings.

[0125] See also Figure 2 As shown in FIG, this figure is a structural diagram of an image quality determination device provided by an embodiment of the present application. Figure 2 As shown, the image quality determination device includes:

[0126] An acquisition unit 201 is configured to acquire a depth image and a color image corresponding to the depth image, wherein the depth image includes a structured area and a smooth area;

[0127] A first calculation unit 202 is configured to calculate a first image weight of a j-th pixel of an i-th pixel in the depth image and a second image weight of a j-th pixel of an i-th pixel in the color image, where the j-th pixel is the j-th pixel among a plurality of pixels in a neighborhood centered on the i-th pixel, where the value of i ranges from 1 to less than or equal to the number of pixels in the depth image, and the value of j ranges from 1 to less than or equal to the number of pixels in the neighborhood centered on the i-th pixel;

[0128] A second calculation unit 203 is configured to calculate parameters of the structure area of ​​the i-th pixel according to the first image weight and the second image weight;

[0129] The third calculation unit 204 is configured to calculate a first weight matrix for the i-th pixel point and a second weight matrix for the j-th pixel point of the depth image;

[0130] A fourth calculation unit 205 is configured to calculate parameters of a smoothing area of ​​an i-th pixel point according to the first weight matrix and the second weight matrix;

[0131] The determining unit 206 is configured to obtain a quality parameter of the depth image according to the parameters of the structure region of the i-th pixel and the parameters of the smooth region of the i-th pixel, where the quality parameter is used to measure the image quality of the depth image.

[0132] In one possible implementation, the second calculation unit 203 is specifically used to calculate the product of the first image weight of the jth pixel point of the i-th pixel point and the second image weight of the jth pixel point of the i-th pixel point to obtain the first data of the j-th pixel point of the i-th pixel point; calculate the sum of the first data of each j-th pixel point of the i-th pixel point to obtain the second data; calculate the sum of the first image weights of each j-th pixel point of the i-th pixel point to obtain the third data; calculate the ratio of the second data and the third data to obtain the parameters of the structural area of ​​the i-th pixel point.

[0133] In one possible implementation, the fourth calculation unit 205 is specifically used to calculate the square of the difference between the first weight matrix of the i-th pixel point and the second weight matrix of the j-th pixel point to obtain the fourth data of the j-th pixel point; and calculate the sum of the fourth data of each j-th pixel point to obtain the parameters of the smoothing area of ​​the i-th pixel point.

[0134] In one possible implementation, the determination unit 206 is specifically used to calculate the product of the parameter of the structural area of ​​the i-th pixel and the first weight of the i-th pixel to obtain the fifth data of the i-th pixel; calculate the sum of the fifth data of each i-th pixel to obtain the sixth data; calculate the sum of the first weights of each i-th pixel to obtain the seventh data; calculate the product of the parameter of the smooth area of ​​the i-th pixel and the second weight of the i-th pixel to obtain the eighth data of the i-th pixel; calculate the sum of the eighth data of each i-th pixel to obtain the ninth data; calculate the sum of the second weights of each i-th pixel to obtain the tenth data; calculate the ratio of the sixth data to the seventh data, and the sum of the ratio of the ninth data to the tenth data to obtain the quality parameter of the depth image.

[0135] In one possible implementation, the first weight is the difference between 1 and the second weight. If the parameter of the structural area of ​​the i-th pixel point is greater than a threshold, the first weight is 1; if the parameter of the structural area of ​​the i-th pixel point is greater than the threshold, the first weight is 0.

[0136] Based on the image quality determination method provided by the above method embodiment, the present application provides an image quality determination device, including: a processor, a memory, and a system bus;

[0137] The processor and the memory are connected via the system bus;

[0138] The memory is used to store one or more programs, where the one or more programs include instructions. When the instructions are executed by the processor, the processor executes the method described in any one of the above embodiments.

[0139] Based on an image quality determination method provided by the above method embodiment, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the method described in any of the above embodiments.

[0140] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0141] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0142] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0143] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0144] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining image quality, characterized in that: The method comprises: Acquire a depth image and a color image corresponding to the depth image, wherein the depth image includes a structured area and a smooth area; Calculate a first image weight of a j-th pixel of the i-th pixel of the depth image and a second image weight of a j-th pixel of the i-th pixel of the color image, where the j-th pixel is the j-th pixel among a plurality of pixels included in a neighborhood centered on the i-th pixel, where the value of i is from 1 to less than or equal to the number of pixels included in the depth image, and the value of j is from 1 to less than or equal to the number of pixels included in the neighborhood centered on the i-th pixel; Calculate the parameters of the structure area of ​​the i-th pixel according to the first image weight and the second image weight; Calculate a first weight matrix for the i-th pixel and a second weight matrix for the j-th pixel of the depth image; Calculate the parameters of the smoothing area of ​​the i-th pixel according to the first weight matrix and the second weight matrix; Calculating a product of a parameter of the structure area of ​​the i-th pixel and a first weight of the i-th pixel to obtain fifth data of the i-th pixel; Calculating the sum of the fifth data of each of the i-th pixel points to obtain sixth data; Calculating the sum of the first weights of the i-th pixel to obtain seventh data; Calculating the product of the parameter of the smoothing area of ​​the i-th pixel and the second weight of the i-th pixel to obtain eighth data of the i-th pixel; Calculating the sum of the eighth data of each of the i-th pixel points to obtain ninth data; Calculating the sum of the second weights of each of the i-th pixel points to obtain tenth data; A ratio of the sixth data to the seventh data and a sum of a ratio of the ninth data to the tenth data are calculated to obtain a quality parameter of the depth image; the quality parameter is used to measure the image quality of the depth image.

2. The method according to claim 1, characterized in that The calculating the parameters of the structural area of ​​the i-th pixel according to the first image weight and the second image weight includes: Calculating the product of the first image weight of the j-th pixel of the i-th pixel and the second image weight of the j-th pixel of the i-th pixel to obtain first data of the j-th pixel of the i-th pixel; Calculate the sum of the first data of each j-th pixel point of the i-th pixel point to obtain second data; Calculating the sum of the first image weights of each j-th pixel point of the i-th pixel point to obtain third data; The ratio of the second data to the third data is calculated to obtain the parameters of the structural area of ​​the i-th pixel.

3. The method according to claim 1, characterized in that The calculating of the parameters of the smoothing area of ​​the i-th pixel point according to the first weight matrix and the second weight matrix includes: Calculating the square of the difference between the first weight matrix of the i-th pixel point and the second weight matrix of the j-th pixel point to obtain fourth data of the j-th pixel point; The sum of the fourth data of each of the j-th pixel points is calculated to obtain the parameter of the smoothing area of ​​the i-th pixel point.

4. The method according to claim 1, wherein The first weight is the difference between 1 and the second weight. If the parameter of the structural area of ​​the i-th pixel point is greater than the threshold, the first weight is 0. If the parameter of the structural area of ​​the i-th pixel point is less than or equal to the threshold, the first weight is 1.

5. An image quality determination device, characterized in that: The device comprises: An acquisition unit, configured to acquire a depth image and a color image corresponding to the depth image, wherein the depth image includes a structured area and a smooth area; a first calculation unit, configured to calculate a first image weight of a j-th pixel of an i-th pixel of the depth image and a second image weight of a j-th pixel of an i-th pixel of the color image, where the j-th pixel is the j-th pixel among a plurality of pixels included in a neighborhood centered on the i-th pixel, wherein the value of i ranges from 1 to less than or equal to the number of pixels included in the depth image, and the value of j ranges from 1 to less than or equal to the number of pixels included in the neighborhood centered on the i-th pixel; a second calculating unit, configured to calculate a parameter of a structural region of an i-th pixel according to the first image weight and the second image weight; A third calculation unit is used to calculate a first weight matrix of an i-th pixel point and a second weight matrix of a j-th pixel point of the depth image; a fourth calculation unit, configured to calculate parameters of a smoothing area of ​​an i-th pixel point according to the first weight matrix and the second weight matrix; A determination unit is used to calculate the product of the parameter of the structural area of ​​the i-th pixel and the first weight of the i-th pixel to obtain the fifth data of the i-th pixel; calculate the sum of the fifth data of each i-th pixel to obtain the sixth data; calculate the sum of the first weights of each i-th pixel to obtain the seventh data; calculate the product of the parameter of the smoothing area of ​​the i-th pixel and the second weight of the i-th pixel to obtain the eighth data of the i-th pixel; calculate the sum of the eighth data of each i-th pixel to obtain the ninth data; calculate the sum of the second weights of each i-th pixel to obtain the tenth data; calculate the ratio of the sixth data to the seventh data, and the sum of the ratio of the ninth data to the tenth data to obtain the quality parameter of the depth image, wherein the quality parameter is used to measure the image quality of the depth image.

6. The device according to claim 5, characterized in that The second calculation unit is specifically configured to calculate the product of the first image weight of the jth pixel of the i-th pixel and the second image weight of the jth pixel of the i-th pixel to obtain the first data of the j-th pixel of the i-th pixel; calculate the sum of the first data of each j-th pixel of the i-th pixel to obtain the second data; and calculate the sum of the first image weights of each j-th pixel of the i-th pixel to obtain the third data; The ratio of the second data to the third data is calculated to obtain the parameters of the structural area of ​​the i-th pixel.

7. The device according to claim 5, characterized in that The fourth calculation unit is specifically used to calculate the square of the difference between the first weight matrix of the i-th pixel point and the second weight matrix of the j-th pixel point to obtain the fourth data of the j-th pixel point; calculate the sum of the fourth data of each j-th pixel point to obtain the parameters of the smoothing area of ​​the i-th pixel point.

8. An image quality determination device, characterized in that include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is configured to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor is enabled to perform the method according to any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 4.

Images