Image quality evaluation method and device and computer equipment
By performing the method of color and texture feature extraction, arithmetic encoding and fusion feature evaluation on images, the problem of inaccurate image quality evaluation in the prior art is solved, and more accurate image quality evaluation is achieved.
Patent Information
- Application Number
- CN202411595634.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2025-03-28
AI Technical Summary
Existing image quality evaluation methods are evaluated directly from the image level, resulting in inaccurate evaluation results.
By obtaining the original image to be evaluated, color feature extraction and texture feature extraction are performed separately, arithmetic encoding is performed, the color features and texture features after arithmetic encoding are obtained, the fusion features are synthesized, and the fusion features are quality evaluated.
The accuracy of image quality evaluation is improved by analyzing image quality from both the color and texture levels of the image, and removing redundant information from the features using arithmetic encoding.
Smart Images
Figure CN119723303A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of image assessment, and in particular to an image quality assessment method, device and computer equipment. Background Art
[0002] With the development of electronic technology, the application of image evaluation technology in the medical field has gradually received attention. For example, in the endoscopy scene, the image captured by the endoscope may contain artifacts, such as highlights, a large amount of bubbles and mucus, etc., and also due to the problem of manual shooting, some images may have large-scale light spots or shadows. In addition, the movement of the camera may cause motion blur. These artifacts will seriously affect the image quality of the endoscope, which will not only affect the doctor's judgment of the patient's condition, but also affect artificial intelligence-assisted diagnosis. Therefore, both medical judgment and scientific research require high-quality endoscopic images. Based on this, it is necessary to evaluate the quality of the images captured by the endoscope.
[0003] The existing image quality assessment methods mainly segment the endoscopic image, calculate the mean of each segmented block in the HSV (Hue, Saturation, Value) color space, compare the mean calculation results of the HSV color space of each block with the HSV threshold range of the normal image, and obtain the proportion of poor quality areas, so as to judge the quality of the endoscopic image as a whole. However, the above image quality assessment methods mainly perform image quality assessment directly at the image level, which has the problem of inaccurate image quality assessment results.
[0004] With regard to the existing image quality assessment methods, image quality assessment is performed directly at the image level, which results in inaccurate image quality assessment results. Currently, no effective solution has been proposed. Summary of the invention
[0005] Based on this, it is necessary to provide an image quality assessment method, device and computer equipment to address the above technical issues.
[0006] In a first aspect, the present application provides an image quality assessment method. The method comprises the following steps:
[0007] Obtain the original image to be evaluated;
[0008] Performing color feature extraction and texture feature extraction on the original image respectively to obtain color features and texture features of the original image;
[0009] Performing arithmetic coding on the color features and the texture features of the original image respectively to obtain the arithmetic coded color features and the arithmetic coded texture features of the original image;
[0010] Determining a fusion feature of the original image based on the color feature after the arithmetic coding and the texture feature after the arithmetic coding;
[0011] A quality assessment is performed on the fusion features of the original image to determine a quality assessment result of the original image.
[0012] In one embodiment, respectively extracting color features and texture features from the original image to obtain color features and texture features of the original image comprises the following steps:
[0013] Using a preset color feature extraction model, color feature extraction is performed on the original image to obtain color features of the original image;
[0014] The texture features of the original image are extracted using a preset texture feature extraction model to obtain the texture features of the original image.
[0015] In one of the embodiments, the arithmetically encoding the color features and the texture features of the original image respectively to obtain the arithmetically encoded color features and the arithmetically encoded texture features of the original image comprises the following steps:
[0016] quantizing the color features and the texture features of the original image respectively to obtain the quantized color features and the quantized texture features of the original image;
[0017] The quantized color features and the quantized texture features of the original image are arithmetically encoded respectively to obtain the arithmetically encoded color features and the arithmetically encoded texture features of the original image.
[0018] In one embodiment, the quantizing the color features and the texture features of the original image respectively to obtain the quantized color features and the quantized texture features of the original image comprises the following steps:
[0019] The color features and the texture features of the original image are quantized respectively by using a preset quantization model to obtain the quantized color features and the quantized texture features of the original image.
[0020] In one embodiment, the performing arithmetic coding on the quantized color features and the quantized texture features of the original image respectively to obtain the arithmetically coded color features and the arithmetically coded texture features of the original image comprises the following steps:
[0021] The quantized color features and the quantized texture features of the original image are arithmetically encoded respectively by using a preset arithmetic coding model to obtain the arithmetically encoded color features and the arithmetically encoded texture features of the original image.
[0022] In one of the embodiments, determining the fusion feature of the original image based on the color feature after the arithmetic coding and the texture feature after the arithmetic coding comprises the following steps:
[0023] A matrix corresponding to the color feature after the arithmetic coding and a matrix corresponding to the texture feature after the arithmetic coding are cross-multiplied to obtain a matrix corresponding to the fusion feature of the original image.
[0024] In one embodiment, the performing quality assessment on the fusion features of the original image to determine the quality assessment result of the original image includes the following steps:
[0025] Performing arithmetic coding on the fused features of the original image to obtain the fused features after arithmetic coding;
[0026] The quality of the fused features after arithmetic coding is evaluated by using a preset evaluation model to obtain a quality evaluation result of the original image.
[0027] In one embodiment, performing arithmetic coding on the fusion features of the original image to obtain the fusion features after arithmetic coding includes the following steps:
[0028] Encoding the fusion features of the original image to obtain encoded fusion features;
[0029] quantizing the encoded fusion features to obtain quantized fusion features;
[0030] Performing arithmetic coding on the quantized fused features to obtain the arithmetic coded fused features.
[0031] In a second aspect, the present application also provides an image quality assessment device. The device comprises:
[0032] An image acquisition module, used to acquire an original image to be evaluated;
[0033] A feature extraction module, used to extract color features and texture features from the original image, to obtain color features and texture features of the original image;
[0034] An encoding module, used for performing arithmetic encoding on the color features and texture features of the original image respectively, to obtain the arithmetic encoded color features and the arithmetic encoded texture features of the original image;
[0035] A fusion feature determination module, used to determine the fusion feature of the original image based on the color feature after the arithmetic coding and the texture feature after the arithmetic coding;
[0036] and an evaluation module, which is used to perform quality evaluation on the fusion features of the original image and determine the quality evaluation result of the original image.
[0037] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the image quality assessment method described in the first aspect is implemented.
[0038] The above-mentioned image quality assessment method, device and computer equipment obtain the color features and texture features of the original image by acquiring the original image to be assessed, and perform arithmetic coding on the color features and texture features of the original image respectively to obtain the color features and texture features of the original image after arithmetic coding, and then determine the fusion features of the original image based on the color features and texture features after arithmetic coding, and finally perform quality assessment on the fusion features of the original image to determine the quality assessment result of the original image. It analyzes the image quality from the two levels of color and texture of the image, and uses arithmetic coding, that is, uses information entropy to encode the color features and texture features of the original image, removes redundant information in the features, and makes use of the fused features to perform quality assessment on the original image, and the assessment result is more accurate, which solves the problem that the existing image quality assessment method directly performs image quality assessment from the image level, and the image quality assessment result is inaccurate.
[0039] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0041] Figure 1 A hardware structure block diagram of a terminal for an image quality assessment method provided in an embodiment of the present application;
[0042] Figure 2 A flowchart of an image quality assessment method provided by an embodiment of the present application;
[0043] Figure 3 A flowchart of an image quality assessment method provided in a preferred embodiment of the present application;
[0044] Figure 4 This is a structural block diagram of an image quality assessment device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0045] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.
[0046] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.
[0047] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the image quality assessment method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1More or fewer components as shown, or with Figure 1 Different configurations shown.
[0048] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the image quality assessment method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0049] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.
[0050] In this embodiment, a method for evaluating image quality is provided. Figure 2 is a flow chart of the image quality assessment method of this embodiment. Figure 2 As shown, the process includes the following steps:
[0051] Step S210: obtaining an original image to be evaluated.
[0052] The original image to be evaluated may be an image captured by an endoscope and requiring image quality evaluation. The original image may be an image directly output by an endoscope device. Specifically, it may be an image in a video stream in a YUV (a color space, in which Y represents brightness, and U and V represent chrominance) format directly output by an endoscope device. For example, YUV data may be directly obtained through the SDK (Software Development Kit) or API (Application Programming Interface) of the endoscope device. In addition, if the image directly output by the endoscope device is not a YUV image, the image directly output by the endoscope device needs to be converted into a YUV image. For example, if the image directly output by the endoscope device is an RGB (Red, Green, Blue) image, it is necessary to use image processing software or programming languages (such as Python and OpenCV) to convert the RGB image into a YUV image.
[0053] Step S220 , performing color feature extraction and texture feature extraction on the original image respectively to obtain color features and texture features of the original image.
[0054] Before color feature extraction and texture feature extraction are performed on the original image, it is necessary to ensure that the original image is a YUV image, and it is necessary to extract the UV (chrominance) component and the Y (brightness) component from the original image. The above-mentioned extraction of the UV (chrominance) component and the Y (brightness) component from the original image can be based on the format of the YUV image, extracting the UV (chrominance) component and the Y (brightness) component. The format of the above-mentioned YUV image can include one or more formats such as YUV420, YUV422, and YUV444. For example, YUV420, where 4, 2, and 0 represent the sampling rates of Y, U, and V, respectively, means that every 4 Y components correspond to 1 U component and 1 V component. Therefore, the above-mentioned extraction of the UV (chrominance) component and the Y (brightness) component according to the format of the YUV image can be based on the format of the YUV image, respectively determine the sampling rates of the Y, U, and V components of the YUV image, and separate the Y, U, and V components of the YUV image based on the sampling rates of the Y, U, and V components of the YUV image.
[0055] The above-mentioned color feature extraction and texture feature extraction are respectively performed on the original image to obtain the color feature and texture feature of the original image. The color feature of the original image can be extracted by using the UV component of the original image to obtain the color feature of the original image, and the texture feature of the original image can be extracted by using the Y component of the original image to obtain the texture feature of the original image.
[0056] In addition, the color features of the original image may be the distribution and attribute features of the colors in the original image. For example, the color features of the original image may be represented by color histogram, color moment, color clustering, etc. The texture features of the original image may be the repetitive and regular features of the local patterns in the original image. For example, the texture features of the original image may be represented by gray level co-occurrence matrix, local binary pattern or wavelet transform.
[0057] Step S230 , performing arithmetically coding on the color features and texture features of the original image respectively, to obtain the arithmetically coded color features and arithmetically coded texture features of the original image.
[0058] The above-mentioned arithmetically encoding the color features and texture features of the original image respectively to obtain the arithmetically encoded color features and arithmetically encoded texture features of the original image can be performed by first quantizing the color features and the texture features of the original image respectively to obtain the quantized color features and the quantized texture features of the original image, and then arithmetically encoding the quantized color features and the quantized texture features of the original image respectively to obtain the arithmetically encoded color features and the arithmetically encoded texture features of the original image.
[0059] Step S240, determining the fusion features of the original image based on the arithmetically coded color features and the arithmetically coded texture features.
[0060] In this step, the fusion feature of the original image may be a feature vector used to characterize the relationship between the color feature of the original image and the texture feature of the original image, and specifically may be a comprehensive feature vector formed by combining the feature vector of the color feature with the feature vector of the texture feature.
[0061] The above-mentioned determination of the fusion features of the original image based on the color features after arithmetic coding and the texture features after arithmetic coding can be performed by cross-multiplying the matrix corresponding to the color features after arithmetic coding and the matrix corresponding to the texture features after arithmetic coding, and using the obtained covariance matrix as the matrix corresponding to the fusion features of the original image.
[0062] The specific process of obtaining the fusion features of the original image is as follows:
[0063] m=k a k b
[0064] Among them, k a is the color feature after arithmetic coding, k b is the texture feature after arithmetic coding, and m is the fusion feature of the original image. If k a The size of k is 256×1×1,b The size of is 256×1×1, then the dimension of m is 256×256.
[0065] Step S250, performing quality assessment on the fusion features of the original image to determine a quality assessment result of the original image.
[0066] The above-mentioned quality assessment of the fusion features of the original image to determine the quality assessment result of the original image can be to perform arithmetic coding on the fusion features of the original image to obtain the arithmetically coded fusion features; and perform quality assessment on the arithmetically coded fusion features to obtain the quality assessment result of the original image.
[0067] The above-mentioned quality assessment of the fusion features of the original image may be to assess the quality of the original image using the fusion features of the original image. The above-mentioned quality assessment result may be a category of the output image quality, which may specifically include three categories: high quality, medium quality and low quality.
[0068] In the above steps S210 to S250, the color features and texture features of the original image are obtained by acquiring the original image to be evaluated, and the color features and texture features of the original image are arithmetically encoded respectively to obtain the color features and texture features of the original image after arithmetically encoding, and then the fusion features of the original image are determined based on the color features and texture features after arithmetically encoding, and finally, the fusion features of the original image are quality evaluated to determine the quality evaluation result of the original image. The image quality is analyzed from the two levels of color and texture of the image, and arithmetic coding is used, that is, information entropy is used to encode the color features and texture features of the original image, and redundant information in the features is removed, so that the quality of the original image is evaluated by using the fused features, and the evaluation result is more accurate, which solves the problem that the existing image quality evaluation method directly evaluates the image quality from the image level and the image quality evaluation result is inaccurate.
[0069] In one embodiment, the above step S220, respectively extracting color features and texture features from the original image to obtain color features and texture features of the original image, includes:
[0070] Step S222: Using a preset color feature extraction model, color features of the original image are extracted to obtain color features of the original image.
[0071] The above-mentioned preset color feature extraction model can be a pre-set model for encoding the UV component of an image and extracting the color features of the encoded image. The above-mentioned preset color feature extraction model can be a model for encoding a color image and finally generating a one-dimensional feature vector. Specifically, the above-mentioned preset color feature extraction model can be composed of N downsampling modules and a global pooling module, and each downsampling module and the global pooling module are connected in series. Each downsampling module includes a convolutional layer with a size of 3×3 and a step size of 2, a normalization layer and an activation function. The spatial dimensions of the color features of the original image are reduced in turn through each downsampling module, and the color features of the original image are converted into a one-dimensional feature vector through global pooling (such as global average pooling). The above N is a positive integer, for example, N is 3.
[0072] Among them, the process of extracting color features from the original image using the preset color feature extraction model to obtain the color features of the original image can be expressed as:
[0073] y a =Encoder1(I a )
[0074] Among them, I a is the UV component of the original image, i.e., the input signal of the preset color feature extraction model, and Encoder1() is the preset color feature extraction model. a The size of can be 128×128×2, and the network channels of the 3 downsampling modules can be 16, 64, and 256 respectively.
[0075] Step S224, using a preset texture feature extraction model, extracting texture features from the original image to obtain texture features of the original image.
[0076] The above-mentioned preset texture feature extraction model can be a pre-set model for encoding the Y component of the image and extracting the texture features of the encoded image. The above-mentioned preset texture feature extraction model can be a model for encoding the texture image and finally generating a one-dimensional feature vector. Specifically, the above-mentioned preset texture feature extraction model can be composed of N+1 downsampling modules and a global pooling module, and each downsampling module and the global pooling module are connected in series. Each downsampling module includes a convolution layer with a size of 3×3 and a step size of 2, a normalization layer and an activation function. The spatial dimensions of the texture features of the original image are reduced in turn by each downsampling module, and the texture features of the original image are converted into a one-dimensional feature vector by global pooling (such as global average pooling). It should be noted that because the format of conventional YUV images is generally YUV420, that is, the data volume of the UV component is only half of the Y component, therefore, the preset texture feature extraction model needs to set N+1 downsampling modules to achieve the same size of the one-dimensional feature vector output by the preset texture feature extraction model and the one-dimensional feature vector output by the preset color feature extraction model. If the format of the YUV image is YUV422, the data volume of the UV component is equal to the data volume of the Y component. At this time, the preset texture feature extraction model needs to set N downsampling modules to achieve the same size of the one-dimensional feature vector output by the preset texture feature extraction model and the one-dimensional feature vector output by the preset color feature extraction model. If the format of the YUV image is YUV444, the data volume of the UV component is twice the data volume of the Y component. At this time, the preset texture feature extraction model needs to set N-1 downsampling modules to achieve the same size of the one-dimensional feature vector output by the preset texture feature extraction model and the one-dimensional feature vector output by the preset color feature extraction model.
[0077] The process of extracting texture features from the original image using a preset texture feature extraction model to obtain the texture features of the original image can be expressed as:
[0078] y b =Encoder2(I b )
[0079] Among them, I b is the Y component of the original image, i.e., the input signal of the preset texture feature extraction model, and Encoder2() is the preset texture feature extraction model. b The size of can be 256×256×1, and the network channels of the four sampling modules can be 16, 64, 128 and 256 respectively.
[0080] In the above steps S222 to S224, a preset color feature extraction model is used to extract color features of the original image to obtain the color features of the original image, and a preset texture feature extraction model is used to extract texture features of the original image to obtain the texture features of the original image. By acquiring the color features and texture features of the original image, it is convenient to acquire the fusion features of the original image based on the color features and texture features of the original image.
[0081] Specifically, in one embodiment, based on step S230, the color features and texture features of the original image are arithmetically encoded respectively to obtain the arithmetically encoded color features and arithmetically encoded texture features of the original image, including:
[0082] Step S232, quantizing the color features and texture features of the original image respectively to obtain the quantized color features and quantized texture features of the original image.
[0083] In order to simplify the calculation amount of subsequent arithmetic coding and simplify the calculation process, the color features and texture features of the original image can be quantized so that the color features and texture features of the original image are converted from floating point numbers to integers. The above-mentioned quantization of the color features and texture features of the original image can be performed by using a preset quantization model to quantize the color features and texture features of the original image. The above-mentioned preset quantization model can be one or more of a rounding model, a floor model, an ceiling model, a nearest even integer model, a truncation model, etc. By quantizing the color features and texture features of the original image, the data of the quantized color features and the quantized texture features of the original image are both integers.
[0084] Step S234, performing arithmetically coding on the quantized color features and the quantized texture features of the original image respectively, to obtain the arithmetically coded color features and the arithmetically coded texture features of the original image.
[0085] The arithmetic coding can reduce the size of the arithmetically coded color features and the arithmetically coded texture features by using the statistical characteristics of symbols. The quantized color features and the quantized texture features of the original image are arithmetically coded respectively, that is, the quantized color features and the quantized texture features of the original image are compressed respectively by arithmetic coding to obtain the arithmetically coded color features and the arithmetically coded texture features of the original image, so as to reduce feature redundancy.
[0086] Among them, information entropy is an important concept to measure the amount of information and uncertainty. Therefore, information entropy is involved when arithmetically encoding the quantized color features and quantized texture features of the original image respectively. Information entropy is directly related to the compression efficiency and optimality of the encoding of the quantized color features and quantized texture features of the original image.
[0087] Specifically, information entropy is a measure that describes the uncertainty of a random variable. For the pixel values in an image, information entropy can tell us the average amount of information per pixel value. If a pixel value has a high probability of occurring, then the amount of information it carries is relatively small; conversely, if a pixel value rarely occurs, then the amount of information it carries is relatively large. However, the pixel values of an image usually have statistical properties, that is, some pixel values occur more frequently than other values. Arithmetic coding can use these statistical properties to optimize the encoding process so that more common pixel values are assigned shorter codes, while uncommon pixel values are assigned longer codes. Further, according to Shannon's first theorem, the encoding is close to optimal when the code length of the encoding is close to the information entropy. This means that if we can assign the code length according to the probability distribution of each pixel value so that the average code length is close to the information entropy, then we can achieve the optimal performance of near-lossless compression. Arithmetic coding adapts to the probability distribution of different pixel values by dynamically adjusting the size of the encoding interval. In this way, pixel values with high information entropy (i.e. values with low probability of occurrence) will be assigned longer coding intervals, while pixel values with low information entropy (i.e. values with high probability of occurrence) will be assigned shorter coding intervals. This coding method enables arithmetic coding to more effectively utilize coding space and improve compression efficiency.
[0088] Therefore, information entropy plays a core role in arithmetic coding, which not only guides the optimality of coding, but also affects the efficiency and compression ratio of coding. By minimizing the difference between the code length and information entropy, arithmetic coding can achieve efficient image compression of the quantized color features and quantized texture features of the original image, and obtain the arithmetic coded color features and arithmetic coded texture features of the original image.
[0089] In the above steps S232 to S234, the color features and texture features of the original image are quantized to obtain the quantized color features and quantized texture features of the original image, and the color features and texture features of the original image are encoded using information entropy to remove redundant information in the features. The image quality is analyzed from two levels, namely, the color and texture of the image, so that the quality assessment of the original image using the fused features is more accurate.
[0090] In addition, in one embodiment, based on step S232, the color features and texture features of the original image are quantized respectively to obtain the quantized color features and quantized texture features of the original image, including:
[0091] Step S2322: quantize the color features and texture features of the original image using a preset quantization model to obtain quantized color features and quantized texture features of the original image.
[0092] In one embodiment, the above step S234, respectively performing arithmetic coding on the quantized color features and the quantized texture features of the original image to obtain the arithmetic coded color features and the arithmetic coded texture features of the original image, includes:
[0093] Step S2342: using a preset arithmetic coding model, respectively perform arithmetic coding on the quantized color features and the quantized texture features of the original image to obtain the arithmetically coded color features and the arithmetically coded texture features of the original image.
[0094] The above-mentioned preset arithmetic coding model can be one or more of a Gaussian mixture-attention model, a neural network-based image compression model, a checkerboard context model, and a channel adaptive arithmetic coding model. Among them, the arithmetic coding model adopted in this implementation is not specifically limited in this embodiment.
[0095] In addition, in one embodiment, the above step S240, based on the color features after arithmetic coding and the texture features after arithmetic coding, determines the fusion features of the original image, including:
[0096] Step S242, cross-multiply the matrix corresponding to the color feature after arithmetic coding and the matrix corresponding to the texture feature after arithmetic coding to obtain the matrix corresponding to the fusion feature of the original image.
[0097] Furthermore, in one embodiment, the above step S250, performing quality assessment on the fusion features of the original image to determine the quality assessment result of the original image, includes:
[0098] Step S252, performing arithmetic coding on the fusion features of the original image to obtain fusion features after arithmetic coding.
[0099] Specifically, a preset arithmetic coding model may be used to perform arithmetic coding on the fusion features of the original image to obtain the fusion features after arithmetic coding.
[0100] Step S254, using a preset evaluation model, performs quality evaluation on the fused features after arithmetic coding to obtain a quality evaluation result of the original image.
[0101] The above-mentioned preset evaluation model can be a pre-set model for performing quality evaluation on the fused features after arithmetic coding to obtain the quality evaluation result of the original image. Among them, the above-mentioned preset evaluation model can be a classifier that takes the fused features after arithmetic coding as input and outputs the classification of image quality. Specifically, the above-mentioned classifier can be composed of 3 layers of MPL (Multilayer Perceptron), and the channels of the three layers of MPL are 128, 64, and 3 respectively. The classifier finally outputs one of the 3 categories of high image quality, medium image quality and low image quality.
[0102] In the above steps S252 to S254, the fusion features of the original image are arithmetically encoded to obtain the fusion features after arithmetically encoding, and then the quality assessment of the fusion features after arithmetically encoding is performed to obtain the quality assessment result of the original image, thereby achieving quality assessment of the original image.
[0103] In one embodiment, based on step S252, the fusion features of the original image are arithmetically encoded to obtain the fusion features after arithmetically encoding, including:
[0104] Step S2522, encode the fusion features of the original image to obtain the encoded fusion features.
[0105] The encoding of the fusion features of the original image may be performed by re-extracting the fusion features of the original image to obtain the encoded fusion features. The re-extraction of the fusion features of the original image may be performed by extracting features from the fusion features of the original image using a preset feature extraction model.
[0106] Step S2524, quantize the encoded fusion features to obtain quantized fusion features.
[0107] In this step, the encoded fusion features are quantized to obtain the quantized fusion features, which may be performed by using a preset quantization model to quantize the encoded fusion features to obtain the quantized fusion features.
[0108] Step S2526, performing arithmetic coding on the quantized fused features to obtain the arithmetically coded fused features.
[0109] The arithmetic coding is performed on the quantized fused features to obtain the arithmetically coded fused features. This may be performed on the quantized fused features by using a preset arithmetic coding model to obtain the arithmetically coded fused features.
[0110] In the above steps S2522 to S2526, the fusion features of the original image are encoded to obtain the encoded fusion features, and then the encoded fusion features are quantized to obtain the quantized fusion features. Then, the quantized fusion features are arithmetically encoded to obtain the arithmetically coded fusion features, and redundant information is removed again by information entropy coding, so that the quality assessment of the original image using the arithmetically coded fusion features is more accurate.
[0111] Preferably, the above-mentioned preset quantization model, preset arithmetic coding model and preset evaluation model can use a cross entropy loss function to train the model.
[0112] The present embodiment is described and illustrated below through preferred embodiments.
[0113] Figure 3 FIG. 1 is a flow chart of an image quality assessment method provided by a preferred embodiment of the present application. Figure 3 As shown, the image quality assessment method includes the following steps:
[0114] Step S301, obtaining an original image to be evaluated;
[0115] Step S302, using a preset color feature extraction model to extract color features from the original image to obtain color features of the original image;
[0116] Step S303, using a preset texture feature extraction model to extract texture features from the original image to obtain texture features of the original image;
[0117] Step S304, quantizing the color features and texture features of the original image respectively to obtain quantized color features and quantized texture features of the original image;
[0118] Step S305, performing arithmetic coding on the quantized color features and the quantized texture features of the original image respectively to obtain the arithmetic coded color features and the arithmetic coded texture features of the original image;
[0119] Step S306, determining the fusion features of the original image based on the color features after arithmetic coding and the texture features after arithmetic coding;
[0120] Step S307, encoding the fusion features of the original image to obtain encoded fusion features;
[0121] Step S308, quantizing the encoded fusion features to obtain quantized fusion features;
[0122] Step S309, performing arithmetic coding on the quantized fused features to obtain arithmetically coded fused features;
[0123] Step S310, using a preset evaluation model, performs quality evaluation on the fused features after arithmetic coding to obtain a quality evaluation result of the original image.
[0124] In the above steps S301 to S310, the color features and texture features of the original image are obtained by acquiring the original image to be evaluated, and the color features and texture features of the original image are arithmetically encoded respectively to obtain the color features and texture features of the original image after arithmetically encoding, and then the fusion features of the original image are determined based on the color features and texture features after arithmetically encoding, and finally, the fusion features of the original image are quality evaluated to determine the quality evaluation result of the original image. The image quality is analyzed from the two levels of color and texture of the image, and arithmetic coding is used, that is, information entropy is used to encode the color features and texture features of the original image, and redundant information in the features is removed, so that the quality of the original image is evaluated by using the fused features, and the evaluation result is more accurate, which solves the problem that the existing image quality evaluation method directly performs image quality evaluation from the image level, and the image quality evaluation result is inaccurate.
[0125] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0126] Based on the same inventive concept, an image quality assessment device is also provided in this embodiment, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0127] In one embodiment, Figure 4 is a structural block diagram of an image quality assessment device provided by an embodiment of the present application, such as Figure 4 As shown, the image quality assessment device comprises:
[0128] An image acquisition module 41 is used to acquire an original image to be evaluated;
[0129] The feature extraction module 42 is used to extract color features and texture features of the original image to obtain color features and texture features of the original image;
[0130] The encoding module 43 is used to perform arithmetic encoding on the color features and texture features of the original image respectively to obtain the arithmetic encoded color features and the arithmetic encoded texture features of the original image;
[0131] A fusion feature determination module 44, used to determine the fusion feature of the original image based on the color feature after arithmetic coding and the texture feature after arithmetic coding;
[0132] And an evaluation module 45 is used to perform quality evaluation on the fusion features of the original image and determine the quality evaluation result of the original image.
[0133] The above-mentioned image quality assessment device obtains the color features and texture features of the original image by acquiring the original image to be assessed, and performs arithmetic coding on the color features and texture features of the original image respectively to obtain the color features and texture features of the original image after arithmetic coding, and then determines the fusion features of the original image based on the color features and texture features after arithmetic coding, and finally performs quality assessment on the fusion features of the original image to determine the quality assessment result of the original image. It analyzes the image quality from the two levels of color and texture of the image, and uses arithmetic coding, that is, uses information entropy to encode the color features and texture features of the original image, removes redundant information in the features, and makes the quality assessment of the original image using the fused features, and the assessment result is more accurate, which solves the problem that the existing image quality assessment method directly performs image quality assessment from the image level, and the image quality assessment result is inaccurate.
[0134] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.
[0135] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, any one of the image quality assessment methods in the above embodiments is implemented.
[0136] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0137] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0138] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A method for evaluating image quality, characterized in that: The method comprises: Obtain the original image to be evaluated; Performing color feature extraction and texture feature extraction on the original image respectively to obtain color features and texture features of the original image; Performing arithmetic coding on the color features and the texture features of the original image respectively to obtain the arithmetic coded color features and the arithmetic coded texture features of the original image; Determining a fusion feature of the original image based on the color feature after the arithmetic coding and the texture feature after the arithmetic coding; A quality assessment is performed on the fusion features of the original image to determine a quality assessment result of the original image.
2. The image quality assessment method according to claim 1, characterized in that: The step of respectively performing color feature extraction and texture feature extraction on the original image to obtain the color feature and texture feature of the original image includes: Using a preset color feature extraction model, color feature extraction is performed on the original image to obtain color features of the original image; The texture features of the original image are extracted using a preset texture feature extraction model to obtain the texture features of the original image.
3. The image quality assessment method according to claim 1, characterized in that: The method of performing arithmetic coding on the color features and the texture features of the original image respectively to obtain the arithmetic coded color features and the arithmetic coded texture features of the original image comprises: quantizing the color features and the texture features of the original image respectively to obtain the quantized color features and the quantized texture features of the original image; The quantized color features and the quantized texture features of the original image are arithmetically encoded respectively to obtain the arithmetically encoded color features and the arithmetically encoded texture features of the original image.
4. The image quality assessment method according to claim 3, characterized in that: The step of quantizing the color features and the texture features of the original image to obtain the quantized color features and the quantized texture features of the original image comprises: The color features and the texture features of the original image are quantized respectively by using a preset quantization model to obtain the quantized color features and the quantized texture features of the original image.
5. The image quality assessment method according to claim 3, characterized in that: The performing arithmetic coding on the quantized color features and the quantized texture features of the original image respectively to obtain the arithmetically coded color features and the arithmetically coded texture features of the original image comprises: The quantized color features and the quantized texture features of the original image are arithmetically encoded respectively by using a preset arithmetic coding model to obtain the arithmetically encoded color features and the arithmetically encoded texture features of the original image.
6. The image quality assessment method according to claim 1, characterized in that: The determining the fusion feature of the original image based on the color feature after the arithmetic coding and the texture feature after the arithmetic coding includes: A matrix corresponding to the color feature after the arithmetic coding and a matrix corresponding to the texture feature after the arithmetic coding are cross-multiplied to obtain a matrix corresponding to the fusion feature of the original image.
7. The image quality assessment method according to any one of claims 1 to 6, characterized in that: The performing quality assessment on the fusion features of the original image to determine the quality assessment result of the original image includes: Performing arithmetic coding on the fused features of the original image to obtain the fused features after arithmetic coding; The quality of the fused features after arithmetic coding is evaluated by using a preset evaluation model to obtain a quality evaluation result of the original image.
8. The image quality assessment method according to claim 7, characterized in that: The performing arithmetic coding on the fusion features of the original image to obtain the fusion features after arithmetic coding includes: Encoding the fusion features of the original image to obtain encoded fusion features; quantizing the encoded fusion features to obtain quantized fusion features; Performing arithmetic coding on the quantized fused features to obtain the arithmetic coded fused features.
9. An image quality assessment device, characterized in that: The device comprises: An image acquisition module, used to acquire an original image to be evaluated; A feature extraction module, used to extract color features and texture features from the original image, to obtain color features and texture features of the original image; An encoding module, used for performing arithmetic encoding on the color features and texture features of the original image respectively, to obtain the arithmetic encoded color features and the arithmetic encoded texture features of the original image; A fusion feature determination module, used to determine the fusion feature of the original image based on the color feature after the arithmetic coding and the texture feature after the arithmetic coding; And an evaluation module, used for performing quality evaluation on the fusion features of the original image to determine the quality evaluation result of the original image.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the image quality assessment method according to any one of claims 1 to 8 are implemented.