Image quality evaluation method and device, electronic equipment and medium

Abstract

The present disclosure relates to an image quality evaluation method and device, electronic equipment and medium, wherein the method comprises: acquiring an image to be processed; performing feature extraction processing on the image to obtain original image features and low-frequency image features in the image; determining a quality evaluation result of the image according to the original image features and the low-frequency image features, wherein the low-frequency image features are related features of semantic content in the image; and according to the original image features and the low-frequency image features, related features of noise in the image can be determined, interference of semantic content in image quality evaluation is avoided, accuracy of the quality evaluation result obtained by evaluation is ensured, and image quality evaluation accuracy is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image quality assessment method, apparatus, electronic device, and medium. Background Technology

[0002] Current image quality assessment methods, for example, involve inputting a face image into a feature extraction network to obtain feature data; then inputting the feature data into a fully connected layer for processing to obtain a face quality score for the face image.

[0003] In the above scheme, the face image includes noise and semantic content, which makes it difficult to accurately reflect the noise in the face image when the face quality score is determined by combining noise and semantic content. This results in poor accuracy of the face quality score and poor image quality assessment. Summary of the Invention

[0004] This disclosure provides an image quality assessment method, apparatus, electronic device, and medium.

[0005] According to a first aspect of the present disclosure, an image quality assessment method is provided, the method comprising: acquiring an image to be processed; performing feature extraction processing on the image to acquire original image features and low-frequency image features in the image; and determining a quality assessment result of the image based on the original image features and the low-frequency image features.

[0006] In one embodiment of this disclosure, the original image features include original image sub-features at multiple scales; the low-frequency image features include the low-frequency image sub-features at the multiple scales.

[0007] In one embodiment of this disclosure, the image feature extraction network for extracting features from the original image includes image feature sub-extraction networks at multiple scales; the image feature sub-extraction network includes a first convolutional layer, a ReLU activation network, a second convolutional layer, and an L2 pooling layer connected in sequence.

[0008] In one embodiment of this disclosure, feature extraction processing is performed on the image to obtain low-frequency image features in the image, including: performing at least one of max pooling feature extraction processing, average pooling feature extraction processing, image content feature extraction processing, and perceptual content feature extraction processing on the image to obtain at least one intermediate low-frequency image feature; the intermediate low-frequency image feature includes intermediate low-frequency image sub-features at multiple scales; and the intermediate low-frequency image sub-features at multiple scales in the at least one intermediate low-frequency image feature are fused to obtain the low-frequency image feature.

[0009] In one embodiment of this disclosure, the low-frequency feature extraction network for extracting the low-frequency image features includes a feature fusion network, and a max pooling feature extraction network, an average pooling feature extraction network, an image content feature extraction network, and a perceptual content feature extraction network respectively connected to the feature extraction network; the perceptual content feature extraction network is a deep convolutional neural network VGG-19.

[0010] In one embodiment of this disclosure, determining the quality assessment result of the image based on the original image features and the low-frequency image features includes: for each of the plurality of scales, determining the mean square error between the original image sub-features and the low-frequency image sub-features at that scale; performing a weighted summation of the mean square error at each scale according to the weight of each scale to obtain a quality score; and using the quality score as the quality assessment result of the image.

[0011] In one embodiment of this disclosure, the weights of each of the plurality of scales and the feature extraction network for performing feature extraction processing on the image are trained based on a plurality of sample noise images and sample quality scores of the sample noise images.

[0012] In one embodiment of this disclosure, the quality assessment result is a quality score; the quality score indicates the noise level in the image; the processing scenario to which the image belongs includes at least one of the following: image capture scenario, image ranking and recommendation scenario, and image generation scenario; the quality score is used to guide the processing algorithm or processing parameters in the processing scenario.

[0013] According to a second aspect of the present disclosure, an image quality assessment apparatus is also provided. The apparatus includes: an acquisition module for acquiring an image to be processed; a feature extraction module for performing feature extraction processing on the image to acquire original image features and low-frequency image features in the image; and a determination module for determining the quality assessment result of the image based on the original image features and the low-frequency image features.

[0014] In one embodiment of this disclosure, the original image features include original image sub-features at multiple scales; the low-frequency image features include the low-frequency image sub-features at the multiple scales.

[0015] In one embodiment of this disclosure, the image feature extraction network for extracting features from the original image includes image feature sub-extraction networks at multiple scales; the image feature sub-extraction network includes a first convolutional layer, a ReLU activation network, a second convolutional layer, and an L2 pooling layer connected in sequence.

[0016] In one embodiment of this disclosure, the feature extraction module is specifically used to perform at least one of the following processes on the image: max pooling feature extraction, average pooling feature extraction, image content feature extraction, and perceptual content feature extraction, to obtain at least one intermediate low-frequency image feature; the intermediate low-frequency image feature includes intermediate low-frequency image sub-features at multiple scales; and to perform fusion processing on the intermediate low-frequency image sub-features at multiple scales in the at least one intermediate low-frequency image feature to obtain the low-frequency image feature.

[0017] In one embodiment of this disclosure, the low-frequency feature extraction network for extracting the low-frequency image features includes a feature fusion network, and a max pooling feature extraction network, an average pooling feature extraction network, an image content feature extraction network, and a perceptual content feature extraction network respectively connected to the feature extraction network; the perceptual content feature extraction network is a deep convolutional neural network VGG-19.

[0018] In one embodiment of this disclosure, the determining module is specifically configured to: determine the mean squared error between the original image sub-features and the low-frequency image sub-features at each of the plurality of scales; perform weighted summation of the mean squared errors at each scale according to the weight of each of the plurality of scales to obtain a quality score; and use the quality score as the quality assessment result of the image.

[0019] In one embodiment of this disclosure, the weights of each of the plurality of scales and the feature extraction network for performing feature extraction processing on the image are trained based on a plurality of sample noise images and sample quality scores of the sample noise images.

[0020] In one embodiment of this disclosure, the quality assessment result is a quality score; the quality score indicates the noise level in the image; the processing scenario to which the image belongs includes at least one of the following: image capture scenario, image ranking and recommendation scenario, and image generation scenario; the quality score is used to guide the processing algorithm or processing parameters in the processing scenario.

[0021] According to a third aspect of the present disclosure, an electronic device is also provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: implement the steps of the image quality assessment method as described above.

[0022] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is also provided, which, when executed by a processor, enables the processor to perform the image quality assessment method as described above.

[0023] According to a fifth aspect of the present disclosure, a chip is also provided, including one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from a memory of an electronic device and send the signals to the processors, the signals including computer instructions stored in the memory, wherein when the processor executes the computer instructions, the electronic device performs the image quality assessment method as described above.

[0024] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0025] The process involves acquiring the image to be processed; performing feature extraction on the image to obtain the original image features and low-frequency image features; determining the image quality assessment result based on the original image features and low-frequency image features, where the low-frequency image features are the semantic content-related features in the image; and determining the noise-related features in the image based on the original image features and low-frequency image features, thus avoiding interference from semantic content in the image quality assessment, ensuring the accuracy of the obtained quality assessment result, and improving the accuracy of image quality assessment.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0028] Figure 1 This is a flowchart of an image quality assessment method according to an embodiment of the present disclosure;

[0029] Figure 2 This is a flowchart of an image quality assessment method according to another embodiment of the present disclosure;

[0030] Figure 3 This is a schematic diagram of the structure of an image feature extraction network;

[0031] Figure 4 This is a schematic diagram of the low-frequency feature extraction network structure;

[0032] Figure 5 This is a schematic diagram of the structure of an image quality assessment device according to an embodiment of the present disclosure;

[0033] Figure 6 This is a structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure;

[0034] Figure 7 This is a schematic diagram of the structure of a chip according to an embodiment of the present disclosure. Detailed Implementation

[0035] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0036] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0037] Current image quality assessment methods, for example, involve inputting a face image into a feature extraction network to obtain feature data; then inputting the feature data into a fully connected layer for processing to obtain a face quality score for the face image.

[0038] In the above scheme, the face image includes noise and semantic content, which makes it difficult to accurately reflect the noise in the face image when the face quality score is determined by combining noise and semantic content. This results in poor accuracy of the face quality score and poor image quality assessment.

[0039] Figure 1 This is a flowchart of an image quality assessment method according to an embodiment of the present disclosure. It should be noted that the image quality assessment method of this embodiment can be applied to an image quality assessment device, which can be configured in an electronic device or chip to enable the electronic device or chip to perform image quality assessment functions.

[0040] Among them, electronic devices can be any device with computing capabilities, such as personal computers (PCs), mobile terminals, servers, vehicle controllers, etc. Mobile terminals can be, for example, in-vehicle devices, mobile phones, tablets, personal digital assistants, wearable devices, and other hardware devices with various operating systems, touch screens, and / or displays.

[0041] In addition, the image quality assessment device can also be software in electronic devices. The following embodiments use an electronic device as an example for explanation.

[0042] like Figure 1 As shown, the method includes the following steps:

[0043] Step 101: Obtain the image to be processed.

[0044] In this embodiment of the disclosure, the scene to which the image belongs may include at least one of the following: an image capture scene, an image ranking and recommendation scene, and an image generation scene. In the image capture scene, the image to be processed can be a captured image. In the image ranking and recommendation scene, the image to be processed can be an image to be ranked and recommended. In the image generation scene, the image to be processed can be a generated image.

[0045] Step 102: Perform feature extraction processing on the image to obtain the original image features and low-frequency image features.

[0046] In this embodiment of the disclosure, the electronic device may perform step 102 as follows: perform image feature extraction processing on the image to obtain the original image features in the image; and perform low-frequency feature extraction processing on the image to obtain the low-frequency image features in the image.

[0047] The image can include noise and semantic content. Correspondingly, the original image features can include noise-related features and semantic content-related features. Semantic content is generally low-frequency; therefore, semantic content-related features can be obtained by performing low-frequency feature extraction on the image. Thus, low-frequency image features can be extracted to distinguish between noise-related features and semantic content-related features in the image.

[0048] The original image features can be extracted using an image feature extraction network. Low-frequency image features can be extracted using a low-frequency feature extraction network. The electronic device can be configured with a quality assessment model, which includes the image feature extraction network, the low-frequency feature extraction network, and the quality assessment network. The quality assessment network is used to determine the image quality assessment result based on the original image features and the low-frequency image features.

[0049] The quality assessment model can be trained based on multiple sample noisy images and the sample quality assessment results of these noisy images. The sample noisy images can be images carrying noise, or images without noise that have undergone noise addition processing. The quality assessment results can be, for example, quality scores.

[0050] Step 103: Determine the image quality assessment result based on the original image features and low-frequency image features.

[0051] In this embodiment of the disclosure, the electronic device performing step 103 may, for example, determine the mean square error between the original image features and the low-frequency image features; use this mean square error as a quality score; and use this quality score as the image quality assessment result. The quality score may indicate the noise level in the image, such as the intensity and / or type of noise. The type of noise may include, for example, Gaussian noise, Poisson noise, impulse noise, etc.

[0052] The original image features can include features of multiple dimensions; the low-frequency image features can also include features of multiple dimensions. The process by which an electronic device determines the mean squared error between the original image features and the low-frequency image features can be, for example, as follows: for each dimension of the original image features, determine the squared difference between the feature in that dimension and the feature in the corresponding dimension of the low-frequency image features; sum and average the squared differences across multiple dimensions to obtain the calculated result; and determine this calculated result as the mean squared error between the original image features and the low-frequency image features.

[0053] In this embodiment of the disclosure, when the scene to which the image belongs is an image capture scene, the image capture parameters can be adjusted based on the quality assessment results of the captured image to improve the quality of the captured image. Furthermore, when the scene to which the image belongs is an image ranking and recommendation scene, multiple images can be ranked according to their quality assessment results, and the images ranked higher can be recommended.

[0054] In cases where the scene to which the image belongs is the image generation scene, the image generation parameters can be adjusted based on the image quality assessment results to improve the quality of the generated image.

[0055] In the image quality assessment method of this disclosure, an image to be processed is acquired; feature extraction processing is performed on the image to obtain original image features and low-frequency image features; the image quality assessment result is determined based on the original image features and low-frequency image features, wherein the low-frequency image features are semantic content-related features in the image; based on the original image features and low-frequency image features, the noise-related features in the image can be determined, avoiding interference from semantic content in the image quality assessment, ensuring the accuracy of the obtained quality assessment result, and improving the accuracy of image quality assessment.

[0056] Figure 2 This is a flowchart illustrating another embodiment of an image quality assessment method disclosed herein. It should be noted that the image quality assessment method of this embodiment can be applied to an image quality assessment device, which can be configured in an electronic device or chip to enable the electronic device or chip to perform image quality assessment functions.

[0057] Among them, electronic devices can be any device with computing capabilities, such as personal computers (PCs), mobile terminals, servers, vehicle controllers, etc. Mobile terminals can be, for example, in-vehicle devices, mobile phones, tablets, personal digital assistants, wearable devices, and other hardware devices with various operating systems, touch screens, and / or displays.

[0058] In addition, the image quality assessment device can also be software in electronic devices. The following embodiments use an electronic device as an example for explanation.

[0059] like Figure 2 As shown, the method includes the following steps:

[0060] Step 201: Obtain the image to be processed.

[0061] Step 202: Perform feature extraction processing on the image to obtain the original image features in the image; the original image features include original image sub-features at multiple scales.

[0062] In this embodiment of the disclosure, the image feature extraction network for extracting features from the original image includes image feature sub-extraction networks at multiple scales. Each image feature sub-extraction network includes a first convolutional layer, a ReLU activation network, a second convolutional layer, and an L2 pooling layer connected sequentially. A schematic diagram of the image feature extraction network structure can be shown, for example, as shown below. Figure 3 As shown in the figure. The first and second convolutional layers can be 3×3 convolutional networks.

[0063] Step 203: Perform at least one of the following processing on the image: max pooling feature extraction, average pooling feature extraction, image content feature extraction, and perceptual content feature extraction, to obtain at least one intermediate low-frequency image feature; the intermediate low-frequency image feature includes intermediate low-frequency image sub-features at multiple scales.

[0064] In this embodiment, the max pooling feature extraction network includes max pooling sub-extraction networks at multiple scales. Each max pooling sub-feature network comprises a third convolutional layer, a ReLU activation network, a fourth convolutional layer, and a max pooling layer connected in sequence. The third and fourth convolutional layers can be 3×3 convolutional networks.

[0065] The average pooling feature network includes average pooling feature extraction sub-networks at multiple scales. The average pooling feature extraction sub-network consists of a fifth convolutional layer, a ReLU activation network, a sixth convolutional layer, and an average pooling layer (Avg Pooling), connected sequentially. The fifth and sixth convolutional layers can be 3×3 convolutional networks.

[0066] The structure of the image content feature extraction network is the same as that of the image feature extraction network. The perceptual content feature extraction network can be a deep convolutional neural network, VGG-19.

[0067] The low-frequency feature extraction network for extracting low-frequency image features includes a feature fusion network, and max pooling feature extraction network, average pooling feature extraction network, image content feature extraction network, and perceptual content feature extraction network connected to the feature extraction network, respectively. The feature fusion network may include a seventh convolutional layer, a ReLU activation network, an eighth convolutional layer, and a spatial attention mechanism network connected in sequence. The spatial attention mechanism network can be composed of two convolutional layers, a ReLU activation network, and a Sigmoid activation network. The first convolutional layer reduces the number of channels to 1 / 4 of the original number, and the second convolutional layer restores the original number of channels; a ReLU activation network is used between the two convolutional layers; and a Sigmoid activation network is used after the second convolutional layer to obtain the output of the spatial attention mechanism network.

[0068] Step 204: Perform fusion processing on the intermediate low-frequency image sub-features at multiple scales in at least one intermediate low-frequency image feature to obtain low-frequency image features.

[0069] The fusion of intermediate low-frequency image sub-features at multiple scales within at least one intermediate low-frequency image feature can be achieved through a feature fusion network within a low-frequency feature extraction network. A schematic diagram of the low-frequency feature extraction network can be shown as follows: Figure 4 As shown. In Figure 4 In this context, the max pooling feature extractor refers to the max pooling feature extraction network; the average pooling feature extractor refers to the average pooling feature extraction network; the image content feature extractor refers to the image content feature extraction network; and the perceptual content feature extractor refers to the perceptual content feature extraction network.

[0070] Step 205: Determine the image quality assessment result based on the original image features and low-frequency image features.

[0071] In this embodiment of the disclosure, the electronic device may perform step 205 as follows: for each of the multiple scales, determine the mean square error between the original image sub-features and the low-frequency image sub-features at that scale; perform weighted summation of the mean square error at each scale according to the weight of each scale to obtain a quality score; and use the quality score as the image quality assessment result.

[0072] For example, with 5 scales, the formulas for calculating the quality score can be as shown in formulas (1), (2) and (3).

[0073] F e1 ,...,F e5 =f E (I) (1)

[0074] F l1 ,...,F l5 =f L (I) (2)

[0075]

[0076] Where I represents the image; f E (I) represents the original image features; F e1 f represents the original image sub-features at the first scale; L (I) represents low-frequency image features; F l1 Represents low-frequency image sub-features at the first scale; score represents the quality score; w i MSE(F) represents the weight of the i-th scale; ei -F li ) represents the mean squared error at the i-th scale.

[0077] The weights for each scale across multiple scales, and the feature extraction network used for image feature extraction, are trained based on multiple sample noisy images and sample quality scores for these noisy images. The feature extraction network may include an image feature extraction network and a low-frequency feature extraction network.

[0078] It should be noted that for detailed explanations of steps 201 and 205, please refer to [link / reference needed]. Figure 1 Steps 101 and 103 in the illustrated embodiment will not be described in detail here.

[0079] In the image quality assessment method of this disclosure, an image to be processed is acquired; feature extraction processing is performed on the image to obtain original image features; the original image features include original image sub-features at multiple scales; at least one of the following processing methods is performed on the image: max pooling feature extraction, average pooling feature extraction, image content feature extraction, and perceptual content feature extraction, to obtain at least one intermediate low-frequency image feature; the intermediate low-frequency image feature includes intermediate low-frequency image sub-features at multiple scales; the intermediate low-frequency image sub-features at multiple scales in the at least one intermediate low-frequency image feature are fused to obtain low-frequency image features; and the image quality assessment result is determined based on the original image features and the low-frequency image features. The extraction processing of at least one intermediate low-frequency image feature can obtain a sufficient number of low-frequency image features through multiple low-frequency image feature extraction methods, and then, by combining the low-frequency image features and the original image features, the relevant features of noise in the image can be further accurately determined, thereby further improving the accuracy of image quality assessment.

[0080] Figure 5 This is a schematic diagram of the structure of an image quality assessment device according to an embodiment of the present disclosure.

[0081] like Figure 5 As shown, the image quality assessment device may include: an acquisition module 501, a feature extraction module 502, and a determination module 503.

[0082] The acquisition module 501 is used to acquire the image to be processed; the feature extraction module 502 is used to perform feature extraction processing on the image to acquire the original image features and low-frequency image features in the image; and the determination module 503 is used to determine the quality assessment result of the image based on the original image features and the low-frequency image features.

[0083] In one embodiment of this disclosure, the original image features include original image sub-features at multiple scales; the low-frequency image features include the low-frequency image sub-features at the multiple scales.

[0084] In one embodiment of this disclosure, the image feature extraction network for extracting features from the original image includes image feature sub-extraction networks at multiple scales; the image feature sub-extraction network includes a first convolutional layer, a ReLU activation network, a second convolutional layer, and an L2 pooling layer connected in sequence.

[0085] In one embodiment of this disclosure, the feature extraction module 502 is specifically used to perform at least one of the following processes on the image: max pooling feature extraction, average pooling feature extraction, image content feature extraction, and perceptual content feature extraction, to obtain at least one intermediate low-frequency image feature; the intermediate low-frequency image feature includes intermediate low-frequency image sub-features at multiple scales; and to perform fusion processing on the intermediate low-frequency image sub-features at multiple scales in the at least one intermediate low-frequency image feature to obtain the low-frequency image feature.

[0086] In one embodiment of this disclosure, the low-frequency feature extraction network for extracting the low-frequency image features includes a feature fusion network, and a max pooling feature extraction network, an average pooling feature extraction network, an image content feature extraction network, and a perceptual content feature extraction network respectively connected to the feature extraction network; the perceptual content feature extraction network is a deep convolutional neural network VGG-19.

[0087] In one embodiment of this disclosure, the determining module 503 is specifically configured to: determine the mean squared error between the original image sub-features and the low-frequency image sub-features at each of the plurality of scales; perform weighted summation of the mean squared errors at each scale according to the weight of each of the plurality of scales to obtain a quality score; and use the quality score as the quality assessment result of the image.

[0088] In one embodiment of this disclosure, the weights of each of the plurality of scales and the feature extraction network for performing feature extraction processing on the image are trained based on a plurality of sample noise images and sample quality scores of the sample noise images.

[0089] In one embodiment of this disclosure, the quality assessment result is a quality score; the quality score indicates the noise level in the image; the processing scenario to which the image belongs includes at least one of the following: image capture scenario, image ranking and recommendation scenario, and image generation scenario; the quality score is used to guide the processing algorithm or processing parameters in the processing scenario.

[0090] In the image quality assessment apparatus of this disclosure, an image to be processed is acquired; feature extraction processing is performed on the image to obtain original image features and low-frequency image features; the image quality assessment result is determined based on the original image features and low-frequency image features, wherein the low-frequency image features are semantic content-related features in the image; based on the original image features and low-frequency image features, the noise-related features in the image can be determined, avoiding interference from semantic content in the image quality assessment, ensuring the accuracy of the obtained quality assessment result, and improving the accuracy of image quality assessment.

[0091] According to a third aspect of the present disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor is configured to: implement the image quality assessment method as described above.

[0092] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium.

[0093] When the instructions in the storage medium are executed by the processor, the processor is able to perform the image quality assessment method described above.

[0094] To implement the above embodiments, this disclosure also provides a computer program product.

[0095] When the computer program product is executed by the processor of the electronic device, it enables the electronic device to perform the above-described method.

[0096] Figure 6 This is a structural block diagram of an electronic device according to an exemplary embodiment. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0097] like Figure 6 As shown, the electronic device 1000 includes a processor 111, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 112 or a program loaded from memory 116 into a random access memory (RAM) 113. The RAM 113 also stores various programs and data required for the operation of the electronic device 1000. The processor 111, ROM 112, and RAM 113 are interconnected via a bus 114. An input / output (I / O) interface 115 is also connected to the bus 114.

[0098] The following components are connected to I / O interface 115: memory 116 including hard disks, etc.; and communication section 117 including network interface cards such as local area network (LAN) cards, modems, etc., communication section 117 performs communication processing via a network such as the Internet; and driver 118 is also connected to I / O interface 115 as needed.

[0099] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 117. When the computer program is executed by processor 111, it performs the functions defined in the methods of this disclosure.

[0100] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory including instructions, which can be executed by the processor 111 of the electronic device 1000 to perform the above-described method. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0101] In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0102] Figure 7 This is a schematic diagram of the structure of a chip according to an embodiment of this disclosure. Figure 7 As shown, the chip includes a processor 701 and an interface circuit 702. The number of processors 701 and the number of interface circuits 702 can be one or more.

[0103] Optionally, the chip also includes a memory 703 for storing necessary computer programs and data; an interface circuit 702 for receiving signals from the memory 703 and sending signals to the processor 701, the signals including computer instructions stored in the memory 703, which, when executed by the processor 701, cause the electronic device to perform the image quality assessment method described in the above embodiments of this disclosure.

[0104] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0105] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0106] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0107] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image quality assessment method, characterized in that, The method includes: Obtain the image to be processed; The image is subjected to feature extraction processing to obtain the original image features and low-frequency image features; wherein, the low-frequency image features are semantic content-related features in the image; The quality assessment result of the image is determined based on the original image features and the low-frequency image features; The original image features include original image sub-features at multiple scales; the low-frequency image features include low-frequency image sub-features at multiple scales. The image feature extraction network used to extract features from the original image includes image feature sub-extraction networks at multiple scales; the image feature sub-extraction network includes a first convolutional layer, a ReLU activation network, a second convolutional layer, and an L2 pooling layer connected in sequence. The step of determining the image quality assessment result based on the original image features and the low-frequency image features includes: For each of the plurality of scales, determine the mean square error between the original image sub-features and the low-frequency image sub-features at that scale; Based on the weight of each of the multiple scales, the mean squared errors under each scale are weighted and summed to obtain the quality score; The quality score is used as the quality assessment result of the image.

2. The method according to claim 1, characterized in that, The image is subjected to feature extraction processing to obtain low-frequency image features, including: The image is subjected to at least one of the following processing methods: max pooling feature extraction, average pooling feature extraction, image content feature extraction, and perceptual content feature extraction, to obtain at least one intermediate low-frequency image feature; the intermediate low-frequency image feature includes intermediate low-frequency image sub-features at the multiple scales. The low-frequency image features are obtained by fusing multiple intermediate low-frequency image sub-features at multiple scales from at least one intermediate low-frequency image feature.

3. The method according to claim 2, characterized in that, The low-frequency feature extraction network for extracting the low-frequency image features includes a feature fusion network, and a max pooling feature extraction network, an average pooling feature extraction network, an image content feature extraction network, and a perceptual content feature extraction network, which are respectively connected to the feature extraction network. The network for extracting the perceived content features is a deep convolutional neural network, VGG-19.

4. The method according to claim 1, characterized in that, The weights of each of the multiple scales, and the feature extraction network used for feature extraction processing of the image, are trained based on multiple sample noisy images and the sample quality scores of the sample noisy images.

5. The method according to claim 1, characterized in that, The quality assessment result is a quality score; the quality score indicates the noise level in the image. The processing scenario to which the image belongs includes at least one of the following: image capture scenario, image ranking and recommendation scenario, and image generation scenario; the quality score is used to guide the processing algorithm or processing parameters in the processing scenario.

6. An image quality assessment device, characterized in that, The device includes: The acquisition module is used to acquire the image to be processed; The feature extraction module is used to perform feature extraction processing on the image to obtain the original image features and low-frequency image features in the image; wherein, the low-frequency image features are semantic content-related features in the image; The determining module is used to determine the quality assessment result of the image based on the original image features and the low-frequency image features; The original image features include original image sub-features at multiple scales; the low-frequency image features include low-frequency image sub-features at multiple scales. The image feature extraction network used to extract features from the original image includes image feature sub-extraction networks at multiple scales; the image feature sub-extraction network includes a first convolutional layer, a ReLU activation network, a second convolutional layer, and an L2 pooling layer connected in sequence. The determining module is further configured to: For each of the plurality of scales, determine the mean square error between the original image sub-features and the low-frequency image sub-features at that scale; Based on the weight of each of the multiple scales, the mean squared errors under each scale are weighted and summed to obtain the quality score; The quality score is used as the quality assessment result of the image.

7. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured as follows: The steps of implementing the image quality assessment method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor, the processor is able to perform the image quality assessment method as described in any one of claims 1 to 5.

9. A chip, characterized in that, The device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of the electronic device and send the signals to the processors, the signals including computer instructions stored in the memory, which, when executed by the processors, cause the electronic device to perform the image quality assessment method according to any one of claims 1 to 5.

Images