Image quality determination method and device, electronic equipment and storage medium

Abstract

The invention provides an image quality determination method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed image and a set cue word, inputting the to-be-processed image and the cue word into a recognition model obtained through training, obtaining the quality description information of the to-be-processed image, and obtaining the quality description information of the to-be-processed image according to the quality description information of the to-be-processed image. The scoring model obtained through training is input, the quality score of the to-be-processed image is obtained, the quality description information and the quality score of the to-be-processed image serve as the quality evaluation result of the to-be-processed image, the quality description information of the image is generated firstly, and then the quality score is generated based on the quality description information, so that the accuracy of the quality score is improved; meanwhile, the quality description information of the image can be obtained, the quality score can be obtained, the information amount of the obtained image quality evaluation is increased, and the quality evaluation result can meet the quality analysis requirements of different scenes.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to a method, apparatus, electronic device, and storage medium for determining image quality. Background Art

[0002] Image quality assessment is a fundamental topic in the fields of computer vision and image processing, which tends to evaluate the perceptual quality of images like humans. With the development of the mobile Internet and the artificial intelligence industry, image quality assessment is not only very useful for evaluating user-generated content in people's daily lives, but also widely required in other computer vision tasks.

[0003] However, in related technologies, there are problems of inaccuracy and insufficient information in the image quality assessment results. Summary of the Invention

[0004] This application aims to solve at least one of the technical problems in related technologies to some extent.

[0005] To this end, this application proposes a method, apparatus, electronic device, and storage medium for determining image quality. By first generating quality description information of an image and then generating a quality score based on the quality description information, the accuracy of the quality score is improved. At the same time, not only can the quality description information of the image be obtained, but also the quality score can be obtained, increasing the amount of information obtained in the image quality assessment.

[0006] An embodiment of one aspect of this application proposes a method for determining image quality, including:

[0007] Obtain an image to be processed and a set prompt;

[0008] Input the image to be processed and the prompt into a trained recognition model to obtain quality description information of the image to be processed;

[0009] Input the quality description information of the image to be processed into a trained scoring model to obtain a quality score of the image to be processed;

[0010] Use the quality description information and the quality score of the image to be processed as the quality assessment result of the image to be processed.

[0011] An embodiment of another aspect of this application proposes an apparatus for determining image quality, including:

[0012] An acquisition module, configured to acquire an image to be processed and a set prompt;

[0013] A recognition module, configured to input the image to be processed and the prompt into a trained recognition model to obtain quality description information of the image to be processed;

[0014] A scoring module, configured to input the quality description information of the to-be-processed image into a trained scoring model to obtain the quality score of the to-be-processed image;

[0015] A determination module, configured to use the quality description information and the quality score of the to-be-processed image as the quality assessment result of the to-be-processed image.

[0016] Another embodiment of this application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the foregoing aspect is implemented.

[0017] Another embodiment of this application provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the foregoing aspect is implemented.

[0018] Another embodiment of this application provides a computer program product, on which a computer program is stored. When the program is executed by a processor, the method described in the foregoing aspect is implemented.

[0019] For the method, device, electronic device, and storage medium for determining image quality provided by this application, a to-be-processed image and a set prompt word are obtained, the to-be-processed image and the prompt word are input into a trained recognition model to obtain the quality description information of the to-be-processed image, the quality description information of the to-be-processed image is input into a trained scoring model to obtain the quality score of the to-be-processed image, and the quality description information and the quality score of the to-be-processed image are used as the quality assessment result of the to-be-processed image. By first generating the quality description information of the image and then generating the quality score based on the quality description information, the accuracy of the quality score is improved. At the same time, not only the quality description information of the image can be obtained, but also the quality score can be obtained, increasing the amount of information for the obtained image quality evaluation. The quality assessment result can meet the quality analysis requirements of different scenarios.

[0020] Some of the additional aspects and advantages of this application will be given in the following description, some will become apparent from the following description, or be understood through the practice of this application. Description of the Drawings

[0021] The above and / or additional aspects and advantages of this application will become apparent and be easily understood from the following description of the embodiments in conjunction with the drawings, where:

[0022] Figure 1 is a schematic flowchart of a method for determining image quality provided by an embodiment of this application;

[0023] Figure 2Schematic flowchart of another method for determining image quality provided by an embodiment of this application;

[0024] Figure 3 Schematic structural diagram of an identification model provided by an embodiment of this application;

[0025] Figure 4 Schematic diagram of a scenario for determining image quality provided by an embodiment of this application;

[0026] Figure 5 Schematic structural diagram of an apparatus for determining image quality provided by an embodiment of this application;

[0027] Figure 6 Schematic structural diagram of an electronic device provided by an embodiment of this application. Detailed implementation manners

[0028] The embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as a limitation to the present application.

[0029] The method, apparatus, electronic device, and storage medium for determining image quality according to the embodiments of the present application will be described below with reference to the accompanying drawings.

[0030] Figure 1 Schematic flowchart of a method for determining image quality provided by an embodiment of the present application.

[0031] In the embodiments of the present application, it is exemplified that the method for determining image quality is configured in an apparatus for determining image quality. The apparatus for determining image quality can be applied to any electronic device so that the electronic device can perform the function of determining image quality.

[0032] Among them, the electronic device can be any device with computing capabilities. For example, it can be a mobile terminal. The mobile terminal can be, for example, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, etc., which are hardware devices with various operating systems, touch screens, and / or display screens.

[0033] As Figure 1 shown, the method may include the following steps:

[0034] Step 101, obtain the image to be processed and the set prompt word.

[0035] Among them, the set prompt words can be set based on the requirements of the scenario. In the scenario of quality assessment, the prompt words are, for example: Please output the quality description information of the image. Among them, the quality description information can be a quality evaluation report, and the quality evaluation report includes the quality information of each quality dimension of the image quality. Among them, the quality dimensions at least include: image brightness, image color, image saturation, image contrast, and image noise, etc.

[0036] Step 102: Input the image to be processed and the prompt words into the trained recognition model to obtain the quality description information of the image to be processed.

[0037] Among them, the recognition model can be a model based on the Transform architecture. The trained recognition model has learned to determine the quality description information of the image to be processed under the prompt of the prompt words.

[0038] In an implementation manner of the embodiment of the present application, the image to be processed is input into the feature extraction module of the recognition model to obtain image features. The image features and the prompt words are subjected to feature fusion, specifically, the vectors of the image features and the prompt words are subjected to feature fusion to obtain fused features. The fused features are input into the prediction module of the recognition model to obtain the quality description information of the image. Among them, the fusion can be splicing and summation, which is not limited in this embodiment. Among them, the prediction module includes multiple layers of Transform modules, and each layer of the Transform module includes an encoding sub-module and a decoding sub-module. The encoding sub-module extracts the image features of the corresponding level to increase the amount of information of the extracted image features and reduce the feature dimension to reduce the data processing amount of subsequent feature decoding by the decoding sub-module, realizing the improvement of data processing efficiency without reducing the amount of information.

[0039] Step 103: Input the quality description information of the image to be processed into the trained scoring model to obtain the quality score of the image to be processed.

[0040] Among them, the scoring model can be a model based on the Transform architecture. The trained scoring model has learned the corresponding relationship between the quality description information of the image to be processed and the quality score of the image to be processed, realizing the determination of the quality score based on the trained scoring model and automatically generating the quality score. Since the quality score is generated based on the quality description information, and the quality description information includes the text description related to the image quality score, it realizes the conversion of the abstract image feature information into the text description information that is easier to understand. Furthermore, based on the quality description information, the prediction of the quality score is carried out, so that the model can more easily obtain the semantic information, global information, and local information of the quality description to improve the accuracy of the quality score prediction.

[0041] Step 104: Use the quality description information and the quality score as the quality assessment result of the image to be processed.

[0042] In the embodiment of the present application, the quality assessment result includes the quality description information and the quality score of the image to be processed. Compared with the related art where only the quality score can be output and the quality description information cannot be output simultaneously, the amount of information included in the quality assessment is increased, which can meet the quality analysis requirements of different scenarios.

[0043] In the method for determining the image quality of the embodiment of the present application, the image to be processed and the set prompt word are obtained, the image to be processed and the prompt word are input into the trained recognition model to obtain the quality description information of the image to be processed, the quality description information of the image to be processed is input into the trained scoring model to obtain the quality score of the image to be processed, and the quality description information and the quality score of the image to be processed are used as the quality assessment result of the image to be processed. By first generating the quality description information of the image and then generating the quality score based on the quality description information, the accuracy of the quality score is improved. At the same time, not only can the quality description information of the image be obtained, but also the quality score can be obtained, increasing the amount of information obtained for the image quality evaluation. The quality assessment result can meet the quality analysis requirements of different scenarios.

[0044] Based on the above embodiments, Figure 2 is a schematic flowchart of another method for determining the image quality provided by the embodiment of the present application. As Figure 2 shown, the method includes the following steps:

[0045] Step 201: Obtain the image to be processed and the set prompt word.

[0046] Among them, the image to be processed is an image.

[0047] Step 202: Input the image to be processed and the prompt word into the trained recognition model to obtain the quality description information of the image to be processed.

[0048] Among them, the relevant explanations in the foregoing method embodiments are also applicable to this embodiment, with the same principle and will not be elaborated here.

[0049] Among them, the recognition model includes a feature extraction module and a prediction module. The training method of the recognition model can be implemented in the following manner:

[0050] As an implementation manner, Figure 3 is a schematic structural diagram of a recognition model provided by the embodiment of the present application. As Figure 3As shown, obtain the first sample image and the quality description information of the first sample image. The quality description information of the first sample image can be artificially generated and has high accuracy. Furthermore, input the first sample image into the feature extraction module of the recognition model for feature extraction to obtain the image features of the first sample image, where the image features include global features and local features. Perform word segmentation on the quality description information of the first sample image to obtain the total character sequence. Each character in the character sequence is the smallest unit. For example, in Chinese, one character is a Chinese character or a word. Use the target character sequence in the quality description information of the first sample image as the training prompt word. The target character sequence is a part of the total character sequence. After fusing the image features of the first sample image and the target character sequence, input them into the prediction module of the recognition model to predict the next character of the target character sequence. For example, if the target character sequence is the first K words in the total character sequence and K is a natural number greater than 1, then the predicted character is the (K + 1)-th word.

[0051] Furthermore, according to the difference between the predicted next character of the target character sequence and the actual next character of the target character sequence in the quality description information of the first sample image, adjust the parameters of the recognition model to obtain the trained recognition model.

[0052] Furthermore, iterate multiple times to realize the prediction of each character in the total character sequence. Based on each predicted character, adjust the parameters of the recognition model to finally obtain the trained recognition model. During the training process, train the recognition model based on the artificially generated quality description information, so that the quality description information of the image output by the trained recognition model is more relevant to the subjective quality evaluation, and a more interpretable and accurate quality evaluation result can be obtained.

[0053] Optionally, the loss function of the recognition model can also be determined in another way. Use the first K characters in the total character sequence as the target character sequence, that is, the first to the K-th character sequence is the first target character sequence, the second to the (K + 1)-th is the second target character sequence, the third to the (K + 2)-th is the third target character sequence, and so on. Each target character sequence has a length of K characters. After predicting the corresponding next character based on multiple target character sequences, determine multiple sub-loss functions for the difference between each predicted character and the actual character. Perform weighted averaging on the multiple sub-loss functions to obtain the total loss function. Adjust the parameters of the recognition model according to the total loss function to obtain the trained recognition model.

[0054] Step 203: Input the quality description information of the image to be processed into the first scoring module of the scoring model to obtain the quality scores of multiple quality dimensions corresponding to the image to be processed.

[0055] Step 204: Input the quality scores of multiple quality dimensions into the second scoring module of the scoring model to obtain the comprehensive quality score corresponding to the image to be processed.

[0056] In an implementation manner of the embodiment of the present application, the scoring model includes a first scoring module and a second scoring module. An application scenario of the embodiment of the present application is the design and parameter debugging of an Image Signal Processing (ISP) algorithm. During the ISP algorithm design process, it is necessary to adjust the algorithm and update the parameters according to the image quality score results. The current Image Quality Assessment (IQA) method can only give a rough quality result, cannot perform fine-grained content analysis, and cannot perform quality scoring based on fine-grained content. In order to improve the granularity of the quality score to obtain more quality score information. Thus, in an implementation manner of the embodiment of the present application, the quality score includes the quality scores of multiple quality dimensions and the comprehensive quality score. Input the quality description information of the image to be processed into the first scoring module of the scoring model. The first scoring module is a module based on the Transform structure. The first scoring module includes a scoring sub-module and a fully connected layer. The scoring sub-module outputs the probability values of multiple preset quality dimensions. Input the probability values of multiple quality dimensions into the fully connected layer for fitting to obtain the quality scores of each quality dimension, thereby obtaining the quality scores of multiple quality dimensions corresponding to the image to be processed, realizing the evaluation of the image quality in multiple dimensions, with finer evaluation granularity and more quality evaluation information can be obtained. Furthermore, input the quality scores of multiple quality dimensions into the second scoring module of the scoring model. The second scoring module has learned the weights of the quality scores of each quality dimension and obtains the comprehensive quality score corresponding to the image to be processed through non-linear averaging, improving the accuracy of the comprehensive quality score. Then, use the quality scores of multiple quality dimensions and the comprehensive quality score as the quality score of the image to be processed.

[0057] Among them, the training method of the scoring model can be implemented through the following implementation manner:

[0058] Obtain the sample quality description information, where the sample quality description information is the quality evaluation report of the second sample image. Here, the second sample image and the first sample image are used to identify and distinguish different sample images. The first sample image and the second sample image can be the same sample image or different sample images. The quality evaluation report of the second sample image can refer to the generation method of the quality evaluation report of the first sample image, with the same principle, which will not be elaborated here. Input the quality evaluation report of the second sample image into the scoring model to obtain the predicted quality scores and the predicted comprehensive quality score of multiple quality dimensions of the second sample image. Determine the loss function based on the differences between the predicted quality scores of multiple quality dimensions and the true quality scores of multiple quality dimensions of the second sample image, and the differences between the predicted comprehensive quality score and the true comprehensive quality score of the second sample image. As an implementation, the difference between the predicted quality scores of multiple quality dimensions and the true quality scores of multiple quality dimensions of the second sample image is called the first scoring difference, and the difference between the predicted comprehensive quality score and the true comprehensive quality score of the second sample image is called the second scoring difference. The first scoring difference and the second scoring difference are weighted and added to obtain the loss function. Adjust the parameters of the scoring model according to the loss function to obtain the trained scoring model, which predicts the quality scores and comprehensive scores of multiple dimensions of the image through the quality evaluation report, increasing the information content and accuracy of the quality scores.

[0059] Among them, the label information of the second sample image includes the true quality scores and the true comprehensive quality score of each quality dimension. As an implementation, display the sample quality description information on the interaction interface of the electronic device, and in response to the user's operation on the interaction interface, determine the multiple quality dimensions of the second sample image, as well as the true quality scores and the true comprehensive quality score of each quality dimension, so that the quality scores and comprehensive scores of multiple dimensions output by the trained scoring model are more in line with the requirements of the real scenario.

[0060] Among them, the scoring model is a pre-trained model, and the pre-training data is the quality description information of the images crawled from the Internet, which realizes predicting the (K + 1)-th word based on the first K words, such as movie reviews / photography reviews, etc., and realizes the adjustment of the parameters of the pre-trained model, so that when using the quality description information of the second sample image for model training, the parameters of the scoring model are the pre-trained parameters instead of randomly determined model parameters, improving the effect and efficiency of further training of the scoring model.

[0061] Step 205, use the quality description information, the quality scores and the comprehensive quality score of multiple quality dimensions as the quality evaluation result of the image to be processed.

[0062] Specifically, reference can be made to the explanation in the foregoing embodiments, with the same principle, which will not be elaborated here.

[0063] It should be noted that each model training method in the embodiments of the present application can be repeatedly executed multiple times with multiple training samples, and different training samples can be used each time. Thus, training is stopped when the loss function is less than the threshold, or when the number of repeated executions is greater than the threshold. The model obtained after adjusting the model parameters for the last time is used as the trained model.

[0064] In the method for determining the image quality according to the embodiments of the present application, a to-be-processed image and a set prompt word are obtained. The to-be-processed image and the prompt word are input into the trained recognition model to obtain the quality description information of the to-be-processed image. The quality description information of the to-be-processed image is input into the trained scoring model to obtain the quality scores of the to-be-processed image in multiple quality dimensions and the comprehensive quality score. The quality description information, the quality scores in multiple quality dimensions, and the comprehensive quality score are used as the quality evaluation result of the to-be-processed image. By first generating the quality description information of the image and then generating multi-dimensional quality scores based on the quality description information, the accuracy and granularity of the quality scores are improved. At the same time, not only the quality description information of the image can be obtained, but also the quality scores can be obtained, increasing the amount of information obtained for the image quality evaluation. The quality evaluation result can meet the quality analysis requirements of different scenarios.

[0065] Based on the above embodiments, Figure 4 FIG. is a schematic diagram of a scenario for determining the image quality provided by the embodiments of the present application. As Figure 4 shown, the to-be-processed image is input into the trained recognition model to obtain the quality evaluation report of the to-be-processed image. The quality evaluation report is input into the scoring model for scoring to obtain the quality scores in multiple dimensions and the comprehensive score. Among them, the score of item A is 100, the score of item B is 90, and the score of item C is 85, which are the quality scores in multiple dimensions, and the comprehensive score is 88 points.

[0066] To implement the above embodiments, the embodiments of the present application also propose an apparatus for determining the image quality.

[0067] Figure 5 FIG. is a schematic structural diagram of an apparatus for determining the image quality provided by the embodiments of the present application.

[0068] As Figure 5 shown, the apparatus may include:

[0069] An acquisition module 51, configured to acquire a to-be-processed image and a set prompt word.

[0070] A recognition module 52, configured to input the to-be-processed image and the prompt word into the trained recognition model to obtain the quality description information of the to-be-processed image.

[0071] A scoring module 53, configured to input the quality description information of the image to be processed into a trained scoring model, and obtain the quality score of the image to be processed.

[0072] A determination module 54, configured to use the quality description information and the quality score of the image to be processed as the quality assessment result of the image to be processed.

[0073] Furthermore, in an implementation manner of the embodiment of the present application, an identification module 52 is configured to:

[0074] Input the image to be processed into the feature extraction module of the identification model to obtain image features;

[0075] Perform feature fusion on the image features and the prompt words to obtain fused features;

[0076] Input the fused features into the prediction module of the identification model to obtain the quality description information of the image.

[0077] In an implementation manner of the embodiment of the present application, a scoring module 53 is configured to:

[0078] Input the quality description information of the image to be processed into the first scoring module of the scoring model to obtain the quality scores of multiple quality dimensions corresponding to the image to be processed;

[0079] Input the quality scores of the multiple quality dimensions into the second scoring module of the scoring model to obtain the comprehensive quality score corresponding to the image to be processed;

[0080] Use the quality scores of the multiple quality dimensions and the comprehensive quality score as the quality score of the image to be processed.

[0081] In an implementation manner of the embodiment of the present application, the device further includes a first training module, configured to:

[0082] Obtain a first sample image and the quality description information of the first sample image;

[0083] Input the first sample image into the feature extraction module of the identification model for feature extraction to obtain the image features of the first sample image;

[0084] Use the target character sequence in the quality description information of the first sample image as the training prompt word;

[0085] Input the image features of the first sample image and the training prompt word into the prediction module of the identification model, and predict the next character of the target character sequence;

[0086] Adjust the parameters of the recognition model according to the difference between the next character of the target character sequence obtained by prediction and the next actual character of the target character sequence in the quality description information of the first sample image, so as to obtain the trained recognition model.

[0087] In an implementation manner of the embodiment of the present application, the device further includes a second training module, configured to:

[0088] Obtain sample quality description information; wherein, the sample quality description information is a quality evaluation report of a second sample image;

[0089] Input the quality evaluation report of the second sample image into the scoring model to obtain the predicted quality scores and the predicted comprehensive quality score of multiple quality dimensions of the second sample image;

[0090] Determine a loss function according to the difference between the predicted quality scores of the multiple quality dimensions and the true quality scores of the multiple quality dimensions of the second sample image, and the difference between the predicted comprehensive quality score and the true comprehensive quality score of the second sample image;

[0091] Adjust the parameters of the scoring model according to the loss function to obtain the trained scoring model.

[0092] In an implementation manner of the embodiment of the present application, the device further includes a calibration module, configured to:

[0093] Display the sample quality description information on the interaction interface of the electronic device;

[0094] In response to an operation of the user on the interaction interface, determine multiple quality dimensions of the second sample image, as well as the true quality scores and the true comprehensive quality score of each quality dimension.

[0095] It should be noted that the foregoing explanation of the method embodiment also applies to the device of this embodiment, and will not be repeated here.

[0096] In the image quality determination device according to the embodiments of the present application, a to-be-processed image and a set prompt are obtained, the to-be-processed image and the prompt are input into a trained recognition model to obtain quality description information of the to-be-processed image, and the quality description information of the to-be-processed image is input into a trained scoring model to obtain quality scores of the to-be-processed image in multiple quality dimensions and a comprehensive quality score. The quality description information, the quality scores in multiple quality dimensions, and the comprehensive quality score are used as the quality evaluation result of the to-be-processed image. By first generating the quality description information of the image and then generating multi-dimensional quality scores based on the quality description information, the accuracy and granularity of the quality scores are improved. At the same time, not only the quality description information of the image can be obtained, but also the quality scores can be obtained, increasing the amount of information obtained for the image quality evaluation. The quality evaluation result can meet the quality analysis requirements of different scenarios. To implement the above embodiments, the present application also proposes an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the foregoing method embodiment is implemented.

[0097] To implement the above embodiments, the present application also proposes a non-transitory computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method described in the foregoing method embodiment is implemented.

[0098] To implement the above embodiments, the present application also proposes a computer program product, on which a computer program is stored. When the computer program is executed by a processor, the method described in the foregoing method embodiment is implemented.

[0099] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0100] Referring to Figure 6 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0101] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0102] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, or optical disks.

[0103] The power component 806 provides power to various components of the electronic device 800. The power component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0104] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0105] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0106] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, and the peripheral interface module may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0107] The sensor component 814 includes one or more sensors for providing an assessment of various aspects of the status of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor component 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0108] The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices in a wired or wireless manner. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0109] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0110] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by a processor 820 of the electronic device 800 to complete the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0111] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0112] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0113] Any process or method description in the flowchart or described in other ways herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of the present application includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art of the embodiments of the present application.

[0114] The logic and / or steps represented in the flowchart or otherwise described herein can, for example, be considered as a definable list of executable instructions for implementing logical functions, and can be embodied specifically in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion having one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or otherwise processing as appropriate, and then storing it in a computer memory.

[0115] It should be understood that various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0116] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0117] In addition, each functional unit in various embodiments of the present application may be integrated into one processing module, may exist separately physically for each unit, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0118] The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present application.

Claims

1. A method for determining image quality, characterized in that Including: Obtain the image to be processed and the set prompt words; Input the image to be processed and the prompt words into the trained recognition model to obtain the quality description information of the image to be processed; Input the quality description information of the image to be processed into the trained scoring model to obtain the quality score of the image to be processed; Use the quality description information and the quality score of the image to be processed as the quality assessment result of the image to be processed.

2. The method according to claim 1, characterized in that, The step of inputting the image to be processed and the prompt words into the trained recognition model to obtain the quality description information of the image includes: Input the image to be processed into the feature extraction module of the recognition model to obtain image features; Perform feature fusion on the image features and the prompt words to obtain fused features; Input the fused features into the prediction module of the recognition model to obtain the quality description information of the image.

3. The method according to claim 1, wherein The step of inputting the quality description information of the image to be processed into the trained scoring model to obtain the quality score of the image to be processed includes: Input the quality description information of the image to be processed into the first scoring module of the scoring model to obtain the quality scores of multiple quality dimensions corresponding to the image to be processed; Input the quality scores of the multiple quality dimensions into the second scoring module of the scoring model to obtain the comprehensive quality score corresponding to the image to be processed; Use the quality scores of the multiple quality dimensions and the comprehensive quality score as the quality score of the image to be processed.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Obtain the first sample image and the quality description information of the first sample image; Input the first sample image into the feature extraction module of the recognition model for feature extraction to obtain the image features of the first sample image; Use the target character sequence in the quality description information of the first sample image as the training prompt words; Input the image features of the first sample image and the training prompt words into the prediction module of the recognition model to predict the next character of the target character sequence; Adjust the parameters of the recognition model according to the difference between the predicted next character of the target character sequence and the actual next character of the target character sequence in the quality description information of the first sample image to obtain the trained recognition model.

5. The method according to any one of claims 1-3, characterized in that, The method further includes: Obtain sample quality description information; wherein, the sample quality description information is the quality evaluation report of the second sample image; Input the quality evaluation report of the second sample image into the scoring model to obtain the predicted quality scores of multiple quality dimensions and the predicted comprehensive quality score of the second sample image; Determine the loss function according to the difference between the predicted quality scores of the multiple quality dimensions and the true quality scores of the multiple quality dimensions of the second sample image, and the difference between the predicted comprehensive quality score and the true comprehensive quality score of the second sample image; Adjust the parameters of the scoring model according to the loss function to obtain the trained scoring model.

6. The method according to claim 5, wherein The method further includes: Display the sample quality description information on the interaction interface of the electronic device; In response to an operation of a user on the interaction interface, determine multiple quality dimensions of the second sample image, as well as the true quality score and the true comprehensive quality score of each quality dimension.

7. An apparatus for determining image quality, characterized in that Including: An acquisition module, configured to acquire an image to be processed and a set prompt word; An identification module, configured to input the image to be processed and the prompt word into a trained identification model to obtain quality description information of the image to be processed; A scoring module, configured to input the quality description information of the image to be processed into a trained scoring model to obtain a quality score of the image to be processed; A determination module, configured to use the quality description information and the quality score of the image to be processed as a quality assessment result of the image to be processed.

8. An electronic device, characterized in that, Including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method according to any one of claims 1-6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1-6 is implemented.

10. A computer program product, characterized in that, Including a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.

Images