Image quality detection method and device, electronic equipment and computer readable medium

By converting, standardizing and frequency domain processing of the images to be detected, combined with the pre-trained image block quality detection model, the problems of strong image quality detection dependence and information loss are solved, and the detection accuracy and completeness are improved.

CN118941510APending Publication Date: 2024-11-12GUANGDONG VIMICRO +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410943942.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing image quality detection methods rely on reference images, resulting in low detection accuracy, and the area relationship is not considered when the image to be detected is divided into image blocks, which may lead to the loss of some image area information.

Method used

By acquiring the image to be detected, performing conversion processing to obtain a brightness image block set, standardizing the process and performing frequency domain conversion, inputting it to a pre-trained image block quality detection model, fusing the image block quality detection information to obtain image quality detection results, and reducing the cost Refer to the dependence of the image and consider the image block area relationship.

Benefits of technology

This improves the accuracy and completeness of image quality detection, reduces the loss of image area information, and reduces the impact on the distortion of reference images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941510A_ABST
    Figure CN118941510A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses an image quality detection method and device, electronic equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: acquiring a to-be-detected image; performing conversion processing on the to-be-detected image to obtain a brightness image block set corresponding to the to-be-detected image; standardizing the brightness image block set to obtain a standardized image block set; performing frequency domain conversion processing on each standardized image block in the standardized image block set to obtain frequency domain matrix information of the corresponding standardized image block; inputting the frequency domain matrix information and the standardized image blocks corresponding to the frequency domain matrix information into a pre-trained image block quality detection model to obtain image block quality detection information; and performing fusion processing on the quality detection information of each image block to obtain image quality detection information corresponding to the to-be-detected image. According to the embodiment, the accuracy of image quality detection is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to an image quality detection method, device, electronic device, and computer-readable medium. Background Art

[0002] With the rapid evolution of communication technology and Internet platforms, video and image data have experienced unprecedented exponential growth. However, data acquisition, video encoding compression and transmission will inevitably cause image quality loss. Image quality detection is a technology for detecting the image quality of an image. At present, when detecting the image quality of an image, the method usually used is to detect the image quality of the image through a full reference image quality evaluation method.

[0003] However, when the above method is used to detect the image quality of an image, the following technical problems often occur:

[0004] First, the image quality of an image is tested by a full-reference image quality assessment method. In the process of image quality testing, the image needs to be compared with a reference image, and the image is highly dependent on the reference image. However, in practical applications, the reference image may be difficult to obtain or have quality problems. If the reference image itself has distortion or noise, the image quality of the image is tested by a full-reference image quality assessment method, which will result in low accuracy of image quality testing.

[0005] Continuing, in the process of adopting technical solutions to solve the above-mentioned technical problem 1, the following technical problems are often accompanied: in the process of detecting the image quality of the image to be detected, the image to be detected needs to be divided into various brightness image blocks. The solution for dividing the image to be detected into various brightness image blocks is generally: directly divide the grayscale image corresponding to the image to be detected into various brightness image blocks with image block areas of a preset area and no overlap. However, the above-mentioned conventional solution directly divides the grayscale image corresponding to the image to be detected into various brightness image blocks with image block areas of a preset area and no overlap, and still has the following problems:

[0006] Second, the grayscale image corresponding to the image to be detected is directly divided into individual brightness image blocks with a preset area and no overlap. The relationship between the area of ​​the image to be detected and the area of ​​the divided image blocks is not considered during the division process. If the total area of ​​the image to be detected cannot be divided by the preset image block area, the size of the image blocks in the last row or column may be less than the preset area. This part of the image area may not be divided into any complete image block, and the image information of the above part of the image area may be discarded, resulting in a high possibility of loss of information of part of the image area of ​​the image to be detected.

[0007] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0008] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0009] Some embodiments of the present disclosure propose image quality detection methods, devices, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the above background technology section.

[0010] In a first aspect, some embodiments of the present disclosure provide an image quality detection method, which includes: acquiring an image to be detected; performing conversion processing on the image to be detected to obtain a brightness image block set corresponding to the image to be detected; performing standardization processing on the brightness image block set to obtain a standardized image block set; performing frequency domain conversion processing on each standardized image block in the standardized image block set to obtain frequency domain matrix information corresponding to the standardized image block; for each frequency domain matrix information in the obtained frequency domain matrix information, inputting the frequency domain matrix information and the standardized image block corresponding to the frequency domain matrix information into a pre-trained image block quality detection model to obtain image block quality detection information, wherein the image block quality detection information includes a foreground and background classification identifier and an image block quality score; based on the respective foreground and background classification identifiers included in the obtained image block quality detection information, performing fusion processing on the respective image block quality detection information to obtain image quality detection information corresponding to the image to be detected.

[0011] In a second aspect, some embodiments of the present disclosure provide an image quality detection device, the device comprising: an acquisition unit, configured to acquire an image to be detected; a conversion processing unit, configured to perform conversion processing on the image to be detected to obtain a brightness image block set corresponding to the image to be detected; a standardization processing unit, configured to perform standardization processing on the brightness image block set to obtain a standardized image block set; a frequency domain conversion processing unit, configured to perform frequency domain conversion processing on each standardized image block in the standardized image block set to obtain frequency domain matrix information corresponding to the standardized image block; an input unit, configured to, for each frequency domain matrix information obtained, input the frequency domain matrix information and the standardized image block corresponding to the frequency domain matrix information into a pre-trained image block quality detection model to obtain image block quality detection information, wherein the image block quality detection information includes a foreground and background classification identifier and an image block quality score; a fusion processing unit, configured to perform fusion processing on the image block quality detection information based on the foreground and background classification identifiers included in the obtained image block quality detection information to obtain image quality detection information corresponding to the image to be detected.

[0012] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0013] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0014] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the image quality detection method of some embodiments of the present disclosure, the accuracy of image quality detection is improved. Specifically, the reason for the poor accuracy of image quality detection is that: the image quality of the image is detected by the full reference image quality evaluation method. In the process of image quality detection, the image needs to be compared with the reference image, and the dependence on the reference image is strong. However, in practical applications, the reference image may be difficult to obtain or have quality problems. If the reference image itself has distortion or noise, the image quality of the image is detected by the full reference image quality evaluation method, which will cause the accuracy of image quality detection to be low. Based on this, the image quality detection method of some embodiments of the present disclosure, first, obtains the image to be detected. Thus, the image to be detected can be obtained. Then, the above-mentioned image to be detected is converted to obtain a set of brightness image blocks corresponding to the above-mentioned image to be detected. Thus, a set of brightness image blocks for generating each frequency domain matrix information can be obtained. Then, the above-mentioned brightness image block set is standardized to obtain a standardized image block set. Thus, the image pixel values ​​in each brightness image block in the brightness image block set can be scaled to a specific range. Afterwards, each standardized image block in the above-mentioned standardized image block set is subjected to frequency domain conversion processing to obtain frequency domain matrix information corresponding to the above-mentioned standardized image block. Thus, the standardized image block can be converted from the spatial domain to the frequency domain to obtain the frequency domain matrix information of the standardized image block in the frequency domain. Then, for each frequency domain matrix information in each obtained frequency domain matrix information, the above frequency domain matrix information and the standardized image block corresponding to the above frequency domain matrix information are input into a pre-trained image block quality detection model to obtain image block quality detection information, wherein the above image block quality detection information includes a foreground background classification identifier and an image block quality score. Thus, the image quality of the image block can be detected, and the image block can be classified into a foreground background. Based on each foreground background classification identifier included in each obtained image block quality detection information, each image block quality detection information is fused to obtain the image quality detection information corresponding to the above-mentioned image to be detected. Thus, each image block quality detection information is fused according to the foreground background classification identifier of each image block to obtain the image quality detection information of the image to be detected. The differential attention to foreground and background information in the perception process of the human visual system is simulated. The accuracy of image quality detection is improved. This is also because the image to be detected is converted into a standardized image block set whose image pixel values ​​are scaled to a specific range. Then, based on the standardized image block set, each frequency domain matrix information of each standardized image block in the frequency domain is generated. Afterwards, according to each frequency domain matrix information and each standardized image block corresponding to each frequency domain matrix information, image quality detection is performed on each image block through an image block quality detection model to obtain quality detection information of each image block.Next, the differentiated attention paid to foreground and background information during the perception process of the human visual system was simulated, and the quality detection information of each image block was fused to obtain the image quality detection information of the image to be detected. No reference image is used in the image quality detection process, and the dependence on the reference image is weak, which reduces the impact of reference image distortion on the detection results and improves the accuracy of image quality detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0016] Figure 1 is a flow chart of some embodiments of the image quality detection method according to the present disclosure;

[0017] Figure 2 is a schematic structural diagram of some embodiments of the image quality detection device according to the present disclosure;

[0018] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0025] Figure 1 The process 100 of some embodiments of the image quality detection method according to the present disclosure is shown. The image quality detection method comprises the following steps:

[0026] Step 101: Acquire an image to be detected.

[0027] In some embodiments, the execution subject (eg, computing device) of the image quality detection method may obtain the image to be detected through a wired connection or a wireless connection. The image to be detected may be a compressed image.

[0028] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0029] Step 102: convert the image to be detected to obtain a set of brightness image blocks corresponding to the image to be detected.

[0030] In some embodiments, the execution entity may perform conversion processing on the image to be detected to obtain a set of brightness image blocks corresponding to the image to be detected.

[0031] In some optional implementations of some embodiments, the execution subject may perform conversion processing on the image to be detected through the following steps to obtain a set of brightness image blocks corresponding to the image to be detected:

[0032] The first step is to perform a flipping process on the image to be detected to obtain a flipped image to be detected. In practice, the execution subject may flip the image to be detected horizontally or vertically according to a random probability.

[0033] In the second step, for each pixel in the above flipped image to be detected, the following pixel processing steps are performed:

[0034] The first sub-step is to determine the three-primary color value information of the pixel point, wherein the three-primary color value information may refer to the numerical value information of the red, green and blue channels in the pixel point.

[0035] The second sub-step is to generate the brightness value of the corresponding pixel based on the above-mentioned three-primary color value information. In practice, first, the above-mentioned execution entity can determine the product of the red color value corresponding to the red channel in the three-primary color information and the first preset weight value as the first value. Then, the above-mentioned execution entity can determine the product of the green color value corresponding to the green channel in the three-primary color information and the second preset weight value as the second value. After that, the above-mentioned execution entity can determine the product of the blue color value corresponding to the blue channel in the three-primary color information and the third preset weight value as the third value. Finally, the above-mentioned execution entity can determine the sum of the above-mentioned first value, the above-mentioned second value, and the above-mentioned third value as the brightness value of the pixel.

[0036] The third sub-step is to determine the number of color channels of the pixel point as the first preset value, where the first preset value may be 1.

[0037] The fourth sub-step is to determine the brightness value as the pixel value of the pixel point.

[0038] In the third step, the to-be-detected flipped image processed by the above pixel processing steps is determined as the brightness image to be divided.

[0039] The fourth step is to determine the area of ​​the brightness image to be divided as the total area to be divided.

[0040] In the fifth step, in response to determining that the total area to be divided is divisible by the preset division area, the brightness image to be divided is divided into brightness image blocks with an area equal to the preset division area and without overlapping images.

[0041] Step 6: In response to determining that the total area to be divided cannot be divided by the preset division area, the following sliding window segmentation process is performed on the brightness image to be divided:

[0042] In the first sub-step, the length of the brightness image to be divided is determined as the length to be divided.

[0043] The second sub-step is to determine the first preset value as the sliding window length.

[0044] The third sub-step is to determine the width of the brightness image to be divided as the width to be divided.

[0045] The fourth sub-step is to determine the second preset value as the sliding window width.

[0046] The fifth sub-step is to obtain from a preset database a sliding step length corresponding to the length to be divided and the length of the sliding window as a horizontal sliding step length.

[0047] The sixth sub-step is to obtain from a preset database a sliding step length corresponding to the width to be divided and the sliding window width as the longitudinal sliding step length.

[0048] The seventh sub-step is to divide the brightness image to be divided based on the length of the sliding window, the width of the sliding window, the horizontal sliding step length, and the vertical sliding step length to obtain each brightness image block. In practice, the execution subject can execute a pre-packaged division method to divide the brightness image to be divided based on the length of the sliding window, the width of the sliding window, the horizontal sliding step length, and the vertical sliding step length to obtain each brightness image block. There are overlapping areas in each brightness image block divided by the sliding window.

[0049] In the seventh step, the obtained brightness image blocks are determined as a brightness image block set.

[0050] The above-mentioned technical scheme and its related contents, as an inventive point of an embodiment of the present disclosure, solve the second technical problem mentioned in the background technology, "directly dividing the grayscale image corresponding to the image to be detected into individual brightness image blocks with a preset area and no overlap, and not considering the relationship between the area of ​​the image to be detected and the area of ​​the divided image blocks during the division process. If the total area of ​​the image to be detected cannot be divided by the preset image block area, the size of the image blocks in the last row or the last column may be less than the preset area. This part of the image area may not be divided into any complete image block, and the image information of the above part of the image area may be discarded, resulting in a high possibility of loss of information of part of the image area of ​​the image to be detected." The factors that lead to the possibility of losing information of part of the image area of ​​the image to be detected are often as follows: directly dividing the grayscale image corresponding to the image to be detected into individual brightness image blocks with a preset area and no overlap, and not considering the relationship between the area of ​​the image to be detected and the area of ​​the divided image blocks during the division process. If the total area of ​​the image to be detected cannot be divided by the preset image block area, the size of the last row or the last column of the image block may be less than the preset area, and this part of the image area may not be divided into any complete image block, and the image information of the above part of the image area may be discarded, resulting in a greater possibility of losing information of part of the image area of ​​the image to be detected. If the above factors are solved, the effect of reducing the possibility of losing information of part of the image area of ​​the image to be detected can be achieved. In order to achieve this effect, first, the above image to be detected is flipped to obtain a flipped image to be detected. Then, for each pixel in the above flipped image to be detected, the following pixel processing steps are performed: the first sub-step is to determine the three primary color value information of the pixel. Among them, the above three primary color value information can refer to the numerical information of the red, green and blue channels in the pixel. The second sub-step is to generate the brightness value of the corresponding pixel based on the above three primary color value information. The third sub-step is to determine the number of color channels of the pixel point as the above-mentioned first preset value. The fourth sub-step is to determine the above-mentioned brightness value as the pixel value of the pixel point. The third step is to determine the flipped image to be detected after each pixel of the flipped image to be detected is processed by the above-mentioned pixel processing step as the brightness image to be divided. In this way, the flipped image to be detected can be converted into the brightness image to be divided. The fourth step is to determine the area of ​​the above-mentioned brightness image to be divided as the total area to be divided. In this way, the total area to be divided for division processing can be obtained. The fifth step is, in response to determining that the above-mentioned total area to be divided can be divided by the preset division area, dividing the above-mentioned brightness image to be divided into individual brightness image blocks with an area of ​​the above-mentioned preset division area and without overlapping images.Thus, the relationship between the area of ​​the brightness image to be divided and the area of ​​the divided image blocks corresponding to the image to be detected can be considered. When the total area of ​​the image to be detected can be divided by the preset image block area, the brightness image to be divided is divided into individual brightness image blocks with the area of ​​the preset division area and no overlapping images. Afterwards, in response to determining that the total area to be divided cannot be divided by the preset division area, the following sliding window segmentation process is performed on the brightness image to be divided: the first sub-step, the length of the brightness image to be divided is determined as the length to be divided. The second sub-step, the first preset value is determined as the sliding window length. The third sub-step, the width of the brightness image to be divided is determined as the width to be divided. The fourth sub-step, the second preset value is determined as the sliding window width. The fifth sub-step, the sliding step corresponding to the length to be divided and the sliding window length is obtained from the preset database as the horizontal sliding step. The sixth sub-step, the sliding step corresponding to the width to be divided and the sliding window width is obtained from the preset database as the vertical sliding step. The seventh sub-step is to divide the brightness image to be divided based on the length of the sliding window, the width of the sliding window, the horizontal sliding step length, and the vertical sliding step length to obtain each brightness image block. Thus, the relationship between the area of ​​the brightness image to be divided corresponding to the image to be detected and the area of ​​the divided image block is taken into account. When the total area of ​​the image to be detected cannot be divided by the preset image block area, the brightness image to be divided is divided according to the length of the sliding window, the width of the sliding window, the horizontal sliding step length, and the vertical sliding step length to obtain each brightness image block with overlapping areas. The possibility that the size of the last row or last column of the image block in the brightness image to be divided corresponding to the image to be detected is less than the preset area and is discarded is reduced. Finally, the obtained brightness image blocks are determined as a brightness image block set. Since the relationship between the area of ​​the brightness image to be divided corresponding to the image to be detected and the area of ​​the divided image block is taken into account in the process of dividing the brightness image to be divided, the possibility that the size of the last row or last column of the image block in the brightness image to be divided corresponding to the image to be detected is less than the preset area and is discarded is reduced. Furthermore, the possibility of losing information of a part of the image area of ​​the image to be detected is reduced.

[0051] Step 103, normalizing the brightness image block set to obtain a normalized image block set.

[0052] In some embodiments, the execution entity may perform standardization processing on the brightness image block set to obtain a standardized image block set.

[0053] In some optional implementations of some embodiments, the execution subject may perform standardization processing on the brightness image block set to obtain a standardized image block set by the following steps:

[0054] In the first step, for each brightness image block in the above brightness image block set, the following normalization process is performed:

[0055] The first sub-step is to determine the mean value of each pixel value of each pixel point in the brightness image block.

[0056] The second sub-step is to determine the standard deviation of each pixel value of each pixel point in the brightness image block.

[0057] In the third sub-step, for each pixel in the brightness image block, the following update steps are performed:

[0058] Sub-step one: determining the pixel value of the above pixel point as the pixel value to be updated.

[0059] Sub-step 2: Generate an updated pixel value based on the pixel value to be updated, the mean value, and the standard deviation. In practice, first, the execution subject may determine the difference between the pixel value to be updated and the mean value as the target difference value. Then, the execution subject may determine the ratio of the target difference value to the standard deviation as the updated pixel value.

[0060] Sub-step three: determining the updated pixel value as the pixel value of the pixel point to update the pixel point.

[0061] In the fourth sub-step, the brightness image block after each pixel is updated is determined as a standardized image block.

[0062] In the second step, the determined standardized image blocks are determined as a standardized image block set.

[0063] The above technical solution, combined with steps 104 to 105 and their related contents, is an inventive point of an embodiment of the present disclosure, and solves the technical problem three: "directly inputting the brightness image block that has not been standardized into the pre-trained image block quality detection model, because the brightness image block has not been standardized, the feature distribution of the brightness image block varies greatly, which may cause a large difference in data distribution between the layers of the subsequent network, and easily cause the gradient explosion phenomenon, thereby causing the subsequent image block quality detection model to have poor detection accuracy". If the above factors are solved, the effect of improving the accuracy of the detection results of the subsequent image block quality detection model can be achieved. In order to achieve this effect, first, for each brightness image block in the above brightness image block set, the following standardization process is performed: determine the mean of each pixel value of each pixel point in the above brightness image block. Thus, the mean value for generating the updated pixel value can be obtained. Then, determine the standard deviation of each pixel value of each pixel point in the above brightness image block. Thus, the standard deviation for generating the updated pixel value can be obtained. Afterwards, for each pixel point in the above brightness image block, the following update step is performed: determine the pixel value of the above pixel point as the pixel value to be updated. Then, based on the pixel value to be updated, the mean value, and the standard deviation, an updated pixel value is generated. Next, the updated pixel value is determined as the pixel value of the pixel point to update the pixel point. Thus, the pixel point can be updated and the pixel value can be scaled to a specific range. Finally, the brightness image block after each pixel point is updated is determined as a standardized image block. Thus, the image pixel value in the brightness image block can be scaled to a specific range to reduce the variation of the feature distribution of the brightness image block. Combined with step 104, each standardized image block in the standardized image block set is subjected to frequency domain conversion processing to obtain frequency domain matrix information of the corresponding standardized image block. Combined with step 105, for each frequency domain matrix information in each frequency domain matrix information obtained, the frequency domain matrix information and the standardized image block corresponding to the frequency domain matrix information are input into a pre-trained image block quality detection model to obtain image block quality detection information. Thus, the frequency domain matrix information of the corresponding standardized image block and the standardized image block can be input into a pre-trained image block quality detection model to obtain image block quality detection information. The data distribution differences between network layers of the image block quality detection model are reduced, the probability of gradient explosion phenomenon is reduced, and thus the accuracy of the detection results of the image block quality detection model is improved.

[0064] Step 104 , performing frequency domain conversion processing on each standardized image block in the standardized image block set to obtain frequency domain matrix information corresponding to the standardized image block.

[0065] In some embodiments, the execution subject may perform frequency domain conversion processing on each standardized image block in the standardized image block set to obtain frequency domain matrix information corresponding to the standardized image block. In practice, the execution subject may perform frequency domain conversion processing on each standardized image block in the standardized image block set by DCT transformation technology to obtain frequency domain matrix information corresponding to the standardized image block. The frequency domain matrix information may be a discrete cosine transform coefficient matrix.

[0066] Step 105 , for each frequency domain matrix information obtained, the frequency domain matrix information and the standardized image block corresponding to the frequency domain matrix information are input into a pre-trained image block quality detection model to obtain image block quality detection information.

[0067] In some embodiments, the execution subject may input the frequency domain matrix information and the standardized image block corresponding to the frequency domain matrix information into a pre-trained image block quality detection model for each of the obtained frequency domain matrix information to obtain image block quality detection information, wherein the image block quality detection information includes a foreground and background classification identifier and an image block quality score. The foreground and background classification identifier may be an identifier that characterizes the foreground or the background.

[0068] In some optional implementations of some embodiments, the execution subject may input the frequency domain matrix information and the standardized image block corresponding to the frequency domain matrix information into a pre-trained image block quality detection model to obtain the image block quality detection information through the following steps:

[0069] The first step is to input the frequency domain matrix information into the image block quality detection sub-model included in the image block quality detection model to obtain the image block quality score.

[0070] In the second step, the standardized image block is input into the foreground-background detection sub-model included in the image block quality detection model to obtain the foreground-background classification identification.

[0071] In the third step, the image block quality score and the foreground and background classification identifier are determined as image block quality detection information.

[0072] In some optional implementations of some embodiments, the execution subject may input the frequency domain matrix information into the image block quality detection sub-model included in the image block quality detection model to obtain the image block quality score through the following steps:

[0073] In the first step, the frequency domain matrix information is input into the first feature extraction layer to obtain initial feature extraction information. The first feature extraction layer may be a convolution layer including two 3×3 convolution kernels with a step size of 1 and one 1×1 convolution kernel with a step size of 1. The initial feature extraction information may be a feature map of the frequency domain matrix information.

[0074] In the second step, the initial feature extraction information is input into the second feature extraction layer to obtain the first global feature extraction information. The second feature extraction layer may be a convolution layer including four 3×3 convolution kernels with a step size of 2 and one 1×1 convolution kernel with a step size of 2. The first global extraction information may be a downsampled feature map.

[0075] In the third step, the global feature extraction information is input into the third feature extraction layer to obtain deep feature extraction information. The third feature extraction layer may be a convolution layer including two 3×3 convolution kernels with a step size of 1 and one 1×1 convolution kernel with a step size of 1. The deep feature extraction information may be a feature map obtained by further feature extraction of the downsampled feature map.

[0076] In the fourth step, the depth feature extraction information is output to the fourth feature extraction layer to obtain the second global feature extraction information. The fourth feature extraction layer may be a convolution layer including four 3×3 convolution kernels with a step size of 2 and one 1×1 convolution kernel with a step size of 2.

[0077] In a fifth step, the initial feature extraction information, the first global feature extraction information, the deep feature extraction information, and the second global feature extraction information are input into the feature fusion layer to obtain fused feature information. The feature fusion layer may be a fusion layer. The fused feature information may be a spliced ​​feature map obtained by splicing the initial feature extraction information, the first global feature extraction information, the deep feature extraction information, and the second global feature extraction information.

[0078] In the sixth step, the fused feature information is input into the pooling layer to obtain the pooled fused feature information, wherein the pooled fused feature information may be a feature map after pooling.

[0079] In the seventh step, the pooled fusion feature information is input into the quality detection layer to obtain the image block quality score. The quality detection layer may be a fully connected layer that predicts the image quality of the image block based on the pooled fusion feature information. The image block quality score may represent the image quality rating.

[0080] In some optional implementations of some embodiments, the execution subject may input the standardized image block into the foreground-background detection sub-model included in the image block quality detection model to obtain the foreground-background classification identifier through the following steps:

[0081] In the first step, the standardized image block is input into the image block feature extraction layer to obtain image block feature extraction information. The image block feature extraction layer may be a convolution layer. The image block feature extraction information may be a feature vector representing the standardized image block.

[0082] In the second step, the image block feature extraction information is input into the pooling layer to obtain pooling feature information, wherein the pooling feature information may be a feature vector obtained by performing pooling processing on the feature vector.

[0083] The third step is to input the pooled feature information into the foreground background detection layer to obtain the foreground background probability distribution information. The foreground background detection layer can be a fully connected layer that classifies the foreground background of the standardized image block according to the pooled feature vector information. The foreground background probability distribution information can represent the probability distribution of the foreground or background.

[0084] The fourth step is to input the foreground and background probability distribution information into the activation function layer to obtain the foreground and background classification identification.

[0085] Step 106: Based on the foreground and background classification identifiers included in the obtained quality detection information of each image block, the quality detection information of each image block is fused to obtain image quality detection information corresponding to the image to be detected.

[0086] In some embodiments, the execution entity may perform fusion processing on the quality detection information of each image block based on the foreground and background classification identifiers included in the quality detection information of each image block, so as to obtain image quality detection information corresponding to the image to be detected.

[0087] In some optional implementations of some embodiments, the execution subject may perform fusion processing on the quality detection information of each image block based on each foreground and background classification identifier included in the quality detection information of each image block obtained, to obtain image quality detection information corresponding to the image to be detected through the following steps:

[0088] In the first step, for each piece of image block quality detection information in the above-mentioned image block quality detection information, the following processing steps are performed:

[0089] In the first sub-step, the foreground and background classification identifiers in the above-mentioned image block quality detection information are determined as target classification identifiers.

[0090] The second sub-step is to obtain a preset weight value corresponding to the above target classification identifier from a preset database.

[0091] In a third sub-step, the image block quality score in the above-mentioned image block quality detection information is determined as a target score value.

[0092] The fourth sub-step is to generate an image block quality detection value based on the preset weight value and the target score value. In practice, the execution subject may determine the image block quality detection value as the product of the preset weight value and the target score value.

[0093] The second step is to generate a quality score corresponding to the image to be detected based on the quality detection values ​​of each image block generated. In practice, the execution subject may determine the average of the quality detection values ​​of each image block as the quality score corresponding to the image to be detected.

[0094] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the image quality detection method of some embodiments of the present disclosure, the accuracy of image quality detection is improved. Specifically, the reason for the poor accuracy of image quality detection is that: the image quality of the image is detected by the full reference image quality evaluation method. In the process of image quality detection, the image needs to be compared with the reference image, and the dependence on the reference image is strong. However, in practical applications, the reference image may be difficult to obtain or have quality problems. If the reference image itself has distortion or noise, the image quality of the image is detected by the full reference image quality evaluation method, which will cause the accuracy of image quality detection to be low. Based on this, the image quality detection method of some embodiments of the present disclosure, first, obtains the image to be detected. Thus, the image to be detected can be obtained. Then, the above-mentioned image to be detected is converted to obtain a set of brightness image blocks corresponding to the above-mentioned image to be detected. Thus, a set of brightness image blocks for generating each frequency domain matrix information can be obtained. Then, the above-mentioned brightness image block set is standardized to obtain a standardized image block set. Thus, the image pixel values ​​in each brightness image block in the brightness image block set can be scaled to a specific range. Afterwards, each standardized image block in the above-mentioned standardized image block set is subjected to frequency domain conversion processing to obtain frequency domain matrix information corresponding to the above-mentioned standardized image block. Thus, the standardized image block can be converted from the spatial domain to the frequency domain to obtain the frequency domain matrix information of the standardized image block in the frequency domain. Then, for each frequency domain matrix information in each obtained frequency domain matrix information, the above frequency domain matrix information and the standardized image block corresponding to the above frequency domain matrix information are input into a pre-trained image block quality detection model to obtain image block quality detection information, wherein the above image block quality detection information includes a foreground background classification identifier and an image block quality score. Thus, the image quality of the image block can be detected, and the image block can be classified into a foreground background. Based on each foreground background classification identifier included in each obtained image block quality detection information, each image block quality detection information is fused to obtain the image quality detection information corresponding to the above-mentioned image to be detected. Thus, each image block quality detection information is fused according to the foreground background classification identifier of each image block to obtain the image quality detection information of the image to be detected. The differential attention to foreground and background information in the perception process of the human visual system is simulated. The accuracy of image quality detection is improved. This is also because the image to be detected is converted into a standardized image block set whose image pixel values ​​are scaled to a specific range. Then, based on the standardized image block set, each frequency domain matrix information of each standardized image block in the frequency domain is generated. Afterwards, according to each frequency domain matrix information and each standardized image block corresponding to each frequency domain matrix information, image quality detection is performed on each image block through an image block quality detection model to obtain quality detection information of each image block.Next, the differentiated attention paid to foreground and background information during the perception process of the human visual system was simulated, and the quality detection information of each image block was fused to obtain the image quality detection information of the image to be detected. No reference image is used in the image quality detection process, and the dependence on the reference image is weak, which reduces the impact of reference image distortion on the detection results and improves the accuracy of image quality detection.

[0095] Further references Figure 2 As an implementation of the methods shown in the figures, the present disclosure provides some embodiments of an image quality detection device, and these device embodiments are Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0096] like Figure 2 As shown, the image quality detection device 200 of some embodiments includes: an acquisition unit 201, a conversion processing unit 202, a standardization processing unit 203, a frequency domain conversion processing unit 204, an input unit 205 and a fusion processing unit 206. Among them, the acquisition unit 201 is configured to acquire the image to be detected; the conversion processing unit 202 is configured to perform conversion processing on the above-mentioned image to be detected to obtain a brightness image block set corresponding to the above-mentioned image to be detected; the standardization processing unit 203 is configured to perform standardization processing on the above-mentioned brightness image block set to obtain a standardized image block set; the frequency domain conversion processing unit 204 is configured to perform frequency domain conversion processing on each standardized image block in the above-mentioned standardized image block set to obtain frequency domain matrix information corresponding to the above-mentioned standardized image block; the input unit 205 is configured to, for each frequency domain matrix information obtained in each frequency domain matrix information, input the above frequency domain matrix information and the standardized image block corresponding to the above frequency domain matrix information into a pre-trained image block quality detection model to obtain image block quality detection information, wherein the above-mentioned image block quality detection information includes foreground and background classification identifiers and image block quality scores; the fusion processing unit 206 is configured to perform fusion processing on the above-mentioned image block quality detection information based on the various foreground and background classification identifiers included in the obtained image block quality detection information to obtain image quality detection information corresponding to the above-mentioned image to be detected.

[0097] It is understood that the units described in the device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units contained therein, and will not be described in detail here.

[0098] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0099] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0100] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0101] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the functions defined in the method of some embodiments of the present disclosure are executed.

[0102] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0103] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0104] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains an image to be detected; performs conversion processing on the image to be detected to obtain a brightness image block set corresponding to the image to be detected; performs standardization processing on the brightness image block set to obtain a standardized image block set; performs frequency domain conversion processing on each standardized image block in the standardized image block set to obtain frequency domain matrix information corresponding to the standardized image block; for each frequency domain matrix information in each obtained frequency domain matrix information, the frequency domain matrix information and the standardized image block corresponding to the frequency domain matrix information are input into a pre-trained image block quality detection model to obtain image block quality detection information, wherein the image block quality detection information includes a foreground and background classification identifier and an image block quality score; based on each foreground and background classification identifier included in each obtained image block quality detection information, each image block quality detection information is fused to obtain image quality detection information corresponding to the image to be detected.

[0105] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0106] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0107] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, a conversion processing unit, a standardization processing unit, a frequency domain conversion processing unit, an input unit, and a fusion processing unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring an image to be detected".

[0108] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0109] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of technical features, but should also cover other technical solutions formed by any combination of technical features or their equivalent features without departing from the inventive concept. For example, a technical solution formed by replacing the features with (but not limited to) the technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A method for detecting image quality, comprising: Acquire the image to be detected; Performing conversion processing on the image to be detected to obtain a set of brightness image blocks corresponding to the image to be detected; Performing standardization processing on the brightness image block set to obtain a standardized image block set; Performing frequency domain conversion processing on each standardized image block in the standardized image block set to obtain frequency domain matrix information corresponding to the standardized image block; For each of the obtained frequency domain matrix information, the frequency domain matrix information and the standardized image block corresponding to the frequency domain matrix information are input into a pre-trained image block quality detection model to obtain image block quality detection information, wherein the image block quality detection information includes a foreground and background classification identifier and an image block quality score; Based on each foreground and background classification identifier included in each obtained image block quality detection information, the image block quality detection information is fused to obtain image quality detection information corresponding to the image to be detected.

2. The method according to claim 1, wherein: The step of inputting the frequency domain matrix information and the standardized image block corresponding to the frequency domain matrix information into a pre-trained image block quality detection model to obtain the image block quality detection information includes: Inputting the frequency domain matrix information into the image block quality detection sub-model included in the image block quality detection model to obtain an image block quality score; Inputting the standardized image block into a foreground-background detection submodel included in an image block quality detection model to obtain a foreground-background classification identifier; The image block quality score and the foreground and background classification identifier are determined as image block quality detection information.

3. The method according to claim 2, wherein: The image block quality detection sub-model includes a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, a feature fusion layer, a pooling layer, and a quality detection layer; And the step of inputting the frequency domain matrix information into the image block quality detection sub-model included in the image block quality detection model to obtain an image block quality score comprises: Inputting the frequency domain matrix information into the first feature extraction layer to obtain initial feature extraction information; Inputting the initial feature extraction information into the second feature extraction layer to obtain first global feature extraction information; Inputting the global feature extraction information into the third feature extraction layer to obtain deep feature extraction information; Outputting the depth feature extraction information to the fourth feature extraction layer to obtain second global feature extraction information; Inputting the initial feature extraction information, the first global feature extraction information, the deep feature extraction information, and the second global feature extraction information into the feature fusion layer to obtain fused feature information; Inputting the fused feature information into the pooling layer to obtain pooled fused feature information; The pooled fusion feature information is input into the quality detection layer to obtain the image block quality score.

4. The method according to claim 2, wherein: The foreground and background detection sub-model includes an image block feature extraction layer, a pooling layer, a foreground and background detection layer, and an activation function layer; and the standardized image block is input into the foreground and background detection sub-model included in the image block quality detection model to obtain a foreground and background classification identifier, including: Inputting the standardized image block into the image block feature extraction layer to obtain image block feature extraction information; Inputting the image block feature extraction information into the pooling layer to obtain pooling feature information; Inputting the pooled feature information into the foreground-background detection layer to obtain foreground-background probability distribution information; The foreground-background probability distribution information is input into the activation function layer to obtain the foreground-background classification identification.

5. The method according to claim 1, wherein: The method of fusing the quality detection information of each image block based on each foreground and background classification identifier included in the quality detection information of each image block to obtain image quality detection information corresponding to the image to be detected includes: For each piece of image block quality detection information in the respective image block quality detection information, the following processing steps are performed: Determining the foreground and background classification identifiers in the image block quality detection information as target classification identifiers; Acquire a preset weight value corresponding to the target classification identifier from a preset database; Determine the image block quality score in the image block quality detection information as a target score value; Based on the preset weight value and the target score value, generating an image block quality detection value; Generating a quality score corresponding to the image to be detected based on the generated quality detection values ​​of each image block; The quality score is determined as image quality detection information corresponding to the image to be detected.

6. An image quality detection device, comprising: An acquisition unit, configured to acquire an image to be detected; A conversion processing unit, configured to perform conversion processing on the image to be detected to obtain a set of brightness image blocks corresponding to the image to be detected; A standardization processing unit, configured to perform standardization processing on the brightness image block set to obtain a standardized image block set; A frequency domain conversion processing unit is configured to perform frequency domain conversion processing on each standardized image block in the standardized image block set to obtain frequency domain matrix information corresponding to the standardized image block; An input unit is configured to input each frequency domain matrix information in each frequency domain matrix information obtained and a standardized image block corresponding to the frequency domain matrix information into a pre-trained image block quality detection model to obtain image block quality detection information, wherein the image block quality detection information includes a foreground and background classification identifier and an image block quality score; The fusion processing unit is configured to perform fusion processing on the quality detection information of each image block based on the foreground and background classification identifiers included in the quality detection information of each image block obtained, so as to obtain image quality detection information corresponding to the image to be detected.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.