Model training method, image-based processing method, device, equipment and medium

Abstract

The present disclosure provides a model training method, an image-based processing method, an apparatus, a device and a medium, and relates to the field of artificial intelligence technology, and in particular to the field of computer vision, machine learning and image processing technology. The specific implementation scheme is: obtaining preset sample feature information corresponding to the training sample pair and obtaining a sample label, wherein the preset sample feature information includes the statistical values ​​of the preset visual indicators corresponding to the basic sample image data and the control sample image data, respectively, and the difference information of the statistical values ​​of the same preset visual indicators, and the sample label is used to represent the subjective evaluation score of the image quality of the control sample image data compared to the basic sample image data, and the preset sample feature information is input into the preset image quality evaluation model, and the preset image quality evaluation model is trained according to the output results and the sample label. By adopting the above technical scheme, it is possible to accurately predict the image quality score, reduce the cost of manual evaluation, and improve the scoring efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of computer vision, machine learning, and image processing technology. Background Art

[0002] Currently, watching media content such as pictures, videos, and live broadcasts has become an important way for users to obtain information, and the image quality of the above media content has also become an important factor affecting user experience. In order to improve the image quality, it is necessary to accurately quantify the user's perception of the image, that is, to accurately evaluate the image quality. Summary of the invention

[0003] The present disclosure provides a model training method, an image-based processing method, an apparatus, a device and a medium.

[0004] According to one aspect of the present disclosure, a method for training an image quality assessment model is provided, comprising:

[0005] Obtaining preset sample feature information corresponding to a training sample pair, and obtaining a sample label corresponding to the training sample pair, wherein the training sample pair includes basic sample image data and control sample image data, the image data includes video data and / or picture data, the preset sample feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic sample image data and the control sample image data, and first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicator, and the sample label is used to represent a subjective evaluation score of the image quality of the control sample image data compared with the basic sample image data;

[0006] Inputting the preset sample feature information into a preset image quality evaluation model to obtain an output result of the preset image quality evaluation model;

[0007] The preset image quality assessment model is trained according to the output result of the preset image quality assessment model and the sample label.

[0008] According to another aspect of the present disclosure, there is provided an image-based processing method, comprising:

[0009] Acquire preset feature information corresponding to an image data pair, wherein the image data pair includes basic image data and reference image data, the image data includes video data and / or picture data, and the preset feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic image data and the reference image data, and first difference information of the statistical values ​​of the basic image data and the reference image data for the same preset visual indicator;

[0010] The preset feature information is input into an image quality evaluation model to obtain an output result of the image quality evaluation model, wherein the image quality evaluation model is obtained using the training method described in any embodiment of the present disclosure, and the output result of the image quality evaluation model is used to represent a predicted evaluation score of the image quality of the control image data compared to the basic image data.

[0011] According to another aspect of the present disclosure, there is provided an image-based processing method, comprising:

[0012] Acquiring original feature information of the image data to be processed, wherein the original feature information includes a statistical value of a preset visual indicator corresponding to the image data to be processed;

[0013] Constructing information to be input according to the original feature information, wherein the information to be input includes the original feature information and also includes first difference information of statistical values ​​of the original feature information and candidate feature information for the same preset visual indicator, and the candidate feature information includes feature information obtained after adjustment based on the original feature information;

[0014] Inputting the information to be input into an image quality evaluation model to obtain an output result of the image quality evaluation model, wherein the image quality evaluation model is obtained by using the training method described in any embodiment of the present disclosure;

[0015] Filtering target feature information from the information to be input according to the output result of the image quality evaluation model;

[0016] The target feature information is used to perform corresponding processing on the image data to be processed, so that the target image data obtained after the processing satisfies the target feature information.

[0017] According to another aspect of the present disclosure, a training device for an image quality evaluation model is provided, comprising:

[0018] A feature information acquisition module, used to acquire preset sample feature information corresponding to a training sample pair, wherein the training sample pair includes basic sample image data and control sample image data, the image data includes video data and / or picture data, and the preset sample feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic sample image data and the control sample image data, and first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicator;

[0019] A sample label acquisition module, used to acquire a sample label corresponding to the training sample pair, wherein the sample label is used to represent a subjective evaluation score of the image quality of the control sample image data compared with the basic sample image data;

[0020] A feature information input module, used to input the preset sample feature information into a preset image quality evaluation model to obtain an output result of the preset image quality evaluation model;

[0021] The model training module is used to train the preset image quality evaluation model according to the output result of the preset image quality evaluation model and the sample label.

[0022] According to another aspect of the present disclosure, there is provided an image-based processing device, comprising:

[0023] An information acquisition module, configured to acquire preset feature information corresponding to an image data pair, wherein the image data pair includes basic image data and reference image data, the image data includes video data and / or picture data, and the preset feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic image data and the reference image data, and first difference information between the statistical values ​​of the basic image data and the reference image data for the same preset visual indicator;

[0024] An information input module is used to input the preset feature information into an image quality evaluation model to obtain an output result of the image quality evaluation model, wherein the image quality evaluation model is obtained using the training device described in any embodiment of the present disclosure, and the output result of the image quality evaluation model is used to represent the predicted evaluation score of the image quality of the control image data compared to the basic image data.

[0025] According to another aspect of the present disclosure, there is provided an image-based processing device, comprising:

[0026] An original feature information acquisition module, used to acquire original feature information of the image data to be processed, wherein the original feature information includes a statistical value of a preset visual indicator corresponding to the image data to be processed;

[0027] An information construction module, configured to construct information to be input according to the original feature information, wherein the information to be input includes the original feature information and first difference information of statistical values ​​of the original feature information and candidate feature information for the same preset visual indicator, and the candidate feature information includes feature information obtained after adjustment based on the original feature information;

[0028] An information input module for inputting the information to be input into an image quality evaluation model to obtain an output result of the image quality evaluation model, wherein the image quality evaluation model is obtained using the training device described in any embodiment of the present disclosure;

[0029] A target feature information screening module, used for screening target feature information from the information to be input according to the output result of the image quality evaluation model;

[0030] The image processing module is used to use the target feature information to perform corresponding processing on the image data to be processed so that the target image data obtained after the processing satisfies the target feature information.

[0031] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0032] at least one processor; and

[0033] a memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in any embodiment of the present disclosure.

[0035] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in any embodiment of the present disclosure.

[0036] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method described in any embodiment of the present disclosure is implemented.

[0037] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0039] Figure 1 is a flowchart of a method for training an image quality evaluation model provided according to an embodiment of the present disclosure;

[0040] Figure 2 is a flowchart of another method for training an image quality assessment model provided according to an embodiment of the present disclosure;

[0041] Figure 3 is a flowchart of an image-based processing method provided according to an embodiment of the present disclosure;

[0042] Figure 4 is a schematic diagram of an application scenario provided according to an embodiment of the present disclosure;

[0043] Figure 5 is a flowchart of another image-based processing method provided according to an embodiment of the present disclosure;

[0044] Figure 6 It is a structural schematic diagram of a training device for an image quality evaluation model provided according to an embodiment of the present disclosure;

[0045] Figure 7 is a structural schematic diagram of an image-based processing device provided according to an embodiment of the present disclosure;

[0046] Figure 8 is a structural schematic diagram of another image-based processing device provided according to an embodiment of the present disclosure;

[0047] Fig. 9 The block diagram is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0049] Figure 1 The present invention is a flowchart of a method for training an image quality assessment model according to an embodiment of the present invention. The present invention can be applied to the case of training an image quality assessment model. The method can be executed by a training device for an image quality assessment model, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 , the method specifically includes the following:

[0050] S101, obtaining preset sample feature information corresponding to a training sample pair, and obtaining a sample label corresponding to the training sample pair, wherein the training sample pair includes basic sample image data and control sample image data, the image data includes video data and / or picture data, the preset sample feature information includes statistical values ​​of preset visual indicators corresponding to the basic sample image data and the control sample image data, respectively, and first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicator, and the sample label is used to represent a subjective evaluation score of the image quality of the control sample image data compared with the basic sample image data;

[0051] S102, inputting preset sample feature information into a preset image quality evaluation model to obtain an output result of the preset image quality evaluation model;

[0052] S103: training a preset image quality evaluation model according to the output result and sample labels of the preset image quality evaluation model.

[0053] The evaluation of image quality mainly includes two ways to quantify user perception: subjective evaluation and objective evaluation. Among them, subjective evaluation can be, for example, to use human visual senses to let observers make intuitive judgments on the picture quality of the images involved in the comparison through evaluation tools, and calculate the mean subjective score (MOS). The higher the MOS, the better the transformed image is than the original image. The value range is generally between 0 and 100, and 50 points means that the image quality of the two is basically the same. However, different people may have different judgment criteria for the quality of the image, which is prone to sensory deviation and consumes a lot of manpower costs. In the objective evaluation scheme, objective indicators are usually used to fit user perception, such as the Visual Multimethod Assessment Fusion (VMAF) indicator, which comprehensively evaluates the visual information fidelity (VIF), detail loss measure (DLM) and temporal motion index / average related position pixel difference (TI), and it is difficult to evaluate the intuitive impact of a single visual information on the MOS score. The inventor has discovered through research that after adjusting the visual indicators of an image, the image quality perceived by the user may be improved, thereby obtaining a higher MOS score. Therefore, in the embodiments of the present disclosure, information related to visual indicators is used to construct a model input, and the model is used to fit the correlation between information related to visual indicators and subjective evaluation scores, so that the model can be used to replace human visual senses to predict image quality scores.

[0054] In the embodiment of the present disclosure, a certain number of pictures or videos can be pre-selected to determine the training sample pair. The training sample pair includes basic sample image data and control sample image data. The control sample image data can be obtained by adjusting the value of one or more visual indicators based on the basic sample image data. The basic sample image data and the control sample image data can correspond to the same image content. Visual indicators can be understood as indicators that affect the subjective feelings of users when viewing images, for example, they can include hue, saturation, lightness, contrast, color, and texture. Hue can be understood as the brightness of the primary color in various image color modes, with a level range from 0 to 255. When the hue level is 255, it is white, and when the level is 0, it is black. Saturation can be understood as the concentration of the image color. The higher the saturation, the fuller the color, and the lower the saturation, the older the color will appear. Brightness can be understood as the brightness of the light shining on the scene or image. Contrast can be understood as the difference between different colors. The greater the contrast, the greater the contrast between different colors, that is, the so-called black and white. If the contrast is too large, the image will appear very dazzling. The smaller the contrast, the smaller the contrast between different colors. Color can be understood as the value of each color channel in various image color modes. Texture can be understood as a visual feature that reflects the homogeneous phenomenon in the image, reflecting the surface structure organization and arrangement properties of the object surface with slow or periodic changes. It can be used to describe the surface properties of the scene corresponding to the image or image area.

[0055] Exemplarily, after determining the training sample pair, the sample label corresponding to the training sample pair can be determined manually, and the sample label is used to represent the subjective evaluation score of the image quality of the control sample image data compared with the basic sample image data, which can be the above-mentioned MOS score. For example, for a training sample pair, a certain number of relevant personnel (such as 5-7) score, and then calculate the average of the scores to obtain the corresponding MOS score, that is, the sample label.

[0056] Exemplarily, the preset sample feature information used to input the preset image quality evaluation model includes the statistical values ​​of the preset visual indicators corresponding to the basic sample image data and the control sample image data, respectively. Optionally, the preset visual indicators include at least one of hue, saturation, brightness, contrast, color and texture. The advantage of such a setting is that visual indicators that are easy to quantify and easy to adjust are selected to quantify the correlation with the MOS score, improve the accuracy of model fitting, and facilitate the subsequent use of the output results of the model as a reference for image adjustment, so as to adjust the image with a higher image quality score. Optionally, the preset visual indicators include hue, saturation, brightness, contrast, color and texture. The advantage of such a setting is that the visual features of the image can be measured more comprehensively. Among them, the type of statistical value may include at least one of the maximum value, the average value and the minimum value, and may also include variance and standard deviation. Optionally, the maximum value, the average value and the minimum value may be included at the same time, which can take into account the richness of the features and the computational efficiency. For video data, the statistical value may be the statistical value of multiple video frames. Taking hue as an example, it may be the average value of the hue of multiple video frames, etc.; for image data, the statistical value may be the statistical value of pixels or image areas. Taking brightness as an example, it may be the maximum value of the brightness of multiple pixels or multiple image areas, etc.

[0057] In the disclosed embodiment, the preset sample feature information also includes first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual index, which can enrich the dimension of the model input feature and improve the accuracy of model fitting. The first difference information may include a difference value, which may be the difference value of the same type of statistical values ​​of the basic sample image data and the control sample image data for the same preset visual index. Taking brightness as an example, it may be the difference value between the maximum value of the brightness of the basic sample image data and the maximum value of the brightness of the control sample image data.

[0058] Exemplarily, model fitting can be abstracted as a regression problem, and the preset image quality evaluation model can be a machine learning model, specifically a decision tree model or a neural network model, etc. Optionally, the preset image quality evaluation model includes an extreme gradient boosting tree (eXtreme Gradient Boosting, XGBoost) model, and the XGBoost model has the advantages of good learning effect and fast training speed.

[0059] The training scheme of the image quality evaluation model provided by the embodiment of the present disclosure is used to input preset sample feature information in the preset image quality evaluation model, including statistical values ​​of preset visual indicators corresponding to basic sample image data and control sample image data, respectively, and first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicators. The preset sample feature information is used to fully characterize the visual features of the two groups of sample data and the differences in visual features, so that the preset image quality evaluation model can accurately fit the correlation between the feature information and the subjective evaluation score, and then can accurately predict the image quality score of the video or picture, reduce the cost of manual evaluation, and improve the scoring efficiency.

[0060] In an optional embodiment, the preset sample feature information also includes at least one of the following: second difference information of the statistical values ​​of different types of the basic sample image data for the same preset visual indicator; third difference information of the statistical values ​​of different types of the control sample image data for the same preset visual indicator; fourth difference information of the second difference information and the third difference information. The advantage of such a setting is that the dimension of the model input feature can be further enriched, thereby further improving the accuracy of model fitting. Optionally, each of the above-mentioned difference information can specifically be a difference value. For example, still taking brightness as an example, the second difference information may include the difference of the maximum brightness of the basic sample image data; the third difference information may include the difference of the maximum brightness of the control sample image data; the fourth difference information may include the difference of the maximum brightness of the basic sample image data and the difference of the maximum brightness of the control sample image data, and the difference of the two differences.

[0061] In an optional implementation, the model setting parameters of the XGBoost model include at least one of the following: booster uses gbtree; the objective function is reg:gamma; the loss function is mean absolute error loss; the gamma value is 0.1; the maximum depth of the decision tree is 8; the lambda value is 3; and the seed value is 1000. The advantage of this setting is that by reasonably setting the model setting parameters of the XGBoost model, the training efficiency and training effect of the model can be better balanced, thereby improving the accuracy of model prediction.

[0062] In an optional implementation, the obtaining of preset sample feature information corresponding to the training sample pair includes: determining the image type to be trained, and obtaining preset sample feature information corresponding to the training sample pair corresponding to the image type; wherein, inputting the preset sample feature information into a preset image quality evaluation model to obtain the output result of the preset image quality evaluation model includes: inputting the preset sample feature information into the preset image quality evaluation model corresponding to the image type to obtain the output result of the preset image quality evaluation model. The advantage of this setting is that targeted training for different types of image data can further improve the accuracy of model prediction.

[0063] For example, the classification method of image types is not limited and can be set according to actual needs. For example, the images can be classified according to the objects contained in the images, such as people, still life, and landscape, etc.; the images can be classified according to their uses, such as film and television, daily shooting, monitoring, or news, etc.

[0064] In an optional implementation, the preset sample feature information includes multiple items; after the preset image quality evaluation model is trained according to the output result of the preset image quality evaluation model and the sample label, it also includes: determining the target image quality evaluation model according to the training result; determining the importance ranking of each item of preset sample feature information according to the order from shallow to deep of the tree nodes in the target image quality evaluation model. The advantage of this setting is that the model can not only output the predicted image quality score, but also provide richer information based on the trained model structure, so as to have a more comprehensive understanding of the factors affecting the image quality, which is conducive to assisting the adjustment of the preset visual indicators, and thus obtaining image data with a higher MOS score.

[0065] Figure 2 is a flowchart of another method for training an image quality evaluation model according to an embodiment of the present disclosure. Taking image data including video data as an example, optimization is performed on the basis of the above optional embodiments, such as Figure 2 As shown, the method may include:

[0066] S201, determining the image type to be trained, and obtaining preset sample feature information corresponding to the training sample pair corresponding to the image type.

[0067] The training sample pair includes basic sample image data and control sample image data, and the image data includes video data. The preset sample feature information includes statistical values ​​of preset visual indicators corresponding to the basic sample image data and the control sample image data, first difference information of the same type of statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicator, second difference information of different types of statistical values ​​of the basic sample image data for the same preset visual indicator, third difference information of different types of statistical values ​​of the control sample image data for the same preset visual indicator, and fourth difference information of the second difference information and the third difference information. The preset visual indicators include hue, saturation, brightness, contrast, color and texture, and the types of statistical values ​​include maximum value, average value and minimum value.

[0068] Exemplarily, Table 1 shows the specific content of the preset sample feature information.

[0069] Table 1 Preset sample feature information

[0070]

[0071] Among them, _average represents the average value, _max represents the maximum value, _min represents the minimum value, -DIFF represents the first difference information (difference), _DIFF1 represents the second difference information, _DIFF2 represents the third difference information, and _gap represents the fourth difference information.

[0072] S202: Obtain sample labels corresponding to the training sample pairs.

[0073] The sample label is used to represent the subjective evaluation score of the image quality of the control sample image data compared with the basic sample image data, that is, the MOS value.

[0074] S203: Input the preset sample feature information into a preset image quality evaluation model corresponding to the image type to obtain an output result of the preset image quality evaluation model.

[0075] Among them, the preset image quality evaluation model is the XGBoost model. The model setting parameters of the XGBoost model include: booster uses gbtree; the objective function is reg:gamma; the loss function is the mean absolute error loss; the gamma value is 0.1; the maximum depth of the decision tree is 8; the lambda value is 3; and the seed value is 1000.

[0076] S204: training a preset image quality evaluation model according to the output results and sample labels of the preset image quality evaluation model.

[0077] For example, during the training process, the preset image quality evaluation model can be trained with the goal of minimizing the objective function until a preset training cutoff condition is met. The specific training cutoff condition can be set according to actual needs.

[0078] S205. Determine a target image quality evaluation model according to the training results.

[0079] Exemplarily, after the preset training cutoff condition is met, the obtained model can be determined as the trained target image quality evaluation model. In one embodiment, the obtained MAE value can reach 6.74, which shows that the error between the model prediction value and the true value is very small. Taking MOS score of 20 as a level as an example, the predicted value can basically be in the same level as the true value, verifying the accuracy of the model.

[0080] S206 , determining the importance ranking of each item of preset sample feature information according to the order of the tree nodes in the target image quality evaluation model from shallow to deep.

[0081] Exemplarily, in the tree structure of the trained XGBoost model, the upper tree nodes, that is, the shallower tree nodes, can be considered as tree nodes corresponding to features with a higher degree of influence on the MOS score, and the importance ranking of each item of preset sample feature information is determined according to the order of the tree nodes from shallow to deep.

[0082] For example, taking a trained target image quality evaluation model corresponding to an image type as an example, the top 7 preset sample feature information items ranked by importance are: hue_max-DIFF, hue_average-DIFF, saturation_max-DIFF, colorfulness_average-DIFF, hue_min-DIFF, contrast_average-DIFF, and colorfulness_max-DIFF. It can be seen that the features further constructed on the basis of the statistical values ​​of the preset visual indicators have a greater impact on user perception and can be beneficial to model fitting. For a single video, saturation and hue also have a strong degree of differentiation for MOS scores.

[0083] The training scheme of the image quality evaluation model provided by the embodiment of the present disclosure is used to input preset sample feature information in the preset image quality evaluation model corresponding to the image type, including statistical values ​​of preset visual indicators corresponding to the basic sample image data and the control sample image data, respectively, first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicator, difference information of statistical values ​​of different types of the same preset visual indicator for the single sample image data, and the difference between the difference information. The preset sample feature information is used to fully characterize the visual features of the two groups of sample data and the difference in visual features, so that the preset image quality evaluation model can accurately fit the correlation between the feature information of the corresponding image type and the subjective evaluation score, and then can accurately predict the image quality score of the video or picture, reduce the cost of manual evaluation, and improve the scoring efficiency. Moreover, more abundant information can be given according to the trained model structure, so that the factors affecting the image quality can be more comprehensively understood, which is conducive to assisting the adjustment of the preset visual indicators, and then obtaining image data with higher MOS scores.

[0084] Figure 3 This is a flowchart of an image-based processing method provided according to an embodiment of the present disclosure. The present disclosure embodiment can be applied to the case where the image quality is evaluated using an image quality evaluation model. The method can be executed by an image-based processing device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 3 , the method specifically includes the following:

[0085] S301, obtaining preset feature information corresponding to an image data pair, wherein the image data pair includes basic image data and reference image data, the image data includes video data and / or picture data, and the preset feature information includes statistical values ​​of preset visual indicators corresponding to the basic image data and the reference image data, respectively, and first difference information of the statistical values ​​of the basic image data and the reference image data for the same preset visual indicator;

[0086] S302: Input the preset feature information into the image quality evaluation model to obtain an output result of the image quality evaluation model, where the output result of the image quality evaluation model is used to represent a predicted evaluation score of the image quality of the control image data compared to the basic image data.

[0087] Among them, the image quality evaluation model is obtained using the training method described in any embodiment of the present disclosure.

[0088] For example, during the production stage, transmission stage, and consumption stage of image data, adjustments to the image data may be involved, and the image quality of the adjusted image data may be evaluated. Figure 4 is a schematic diagram of an application scenario provided according to an embodiment of the present disclosure, such as Figure 4As shown, taking video data as an example, parameters (corresponding to preset visual indicators) can be adjusted in the shooting stage, and the video frames captured in the shooting stage can be changed by adjusting the values ​​of one or more preset visual indicators (such as adjusting by adding beauty effects or filters using a camera, and the beauty effects or filters can have a mapping relationship with the values ​​of preset visual indicators) to obtain the adjusted shot video, that is, the basic image data includes the shot image data, and the control image data includes the image data after the shot image is adjusted; the image quality can be optimized in the encoding and decoding stage, and the video shot by the camera is generally stored or transmitted after encoding, and then transmitted to the consumer after decoding, and can also be optimized in the encoding and decoding process. Optimize and adjust the preset visual indicators, that is, the basic image data includes the image data before encoding and decoding, and the reference image data includes the image data after adjusting the image data before encoding and decoding; the image quality can be enhanced in the consumption stage, and the video after encoding and decoding can be presented to the user through a player. Before presenting to the user, the image data can also be adjusted. For example, the problem of deterioration of image quality that may occur after multiple transmissions or transfers can be addressed by adjusting the parameters on the player to restore or adjust the color or contrast, so as to improve the image quality. That is, the basic image data includes the image data before display, and the reference image data includes the image data after adjusting the image data before display.

[0089] Optionally, if in the model training stage, the preset sample feature information also includes at least one of the second difference information, the third difference information and the fourth difference information, then the preset feature information may include at least one of the following: the second difference information of the statistical values ​​of different types of the basic image data for the same preset visual indicator; the third difference information of the statistical values ​​of different types of the control image data for the same preset visual indicator; and the fourth difference information between the second difference information and the third difference information.

[0090] Optionally, if the image quality evaluation model is classified and trained according to image types during the training phase, the preset feature information is input into the image quality evaluation model corresponding to the image type to which the image data belongs.

[0091] The image-based processing solution provided by the embodiments of the present disclosure adopts an image quality evaluation model to accurately evaluate the image quality of adjusted image data. It can accurately determine at each stage of the image data whether the adjusted image data has obtained better image quality, whether the image quality enhancement effect meets expectations, or whether image quality loss is caused due to the adjustment processing, etc., thereby reducing the cost of manual evaluation and accurately evaluating the impact of image adjustment on user perception.

[0092] Figure 5is a flowchart of another image-based processing method provided according to an embodiment of the present disclosure. The embodiment of the present disclosure can be applied to the case where an image quality evaluation model is used to guide image adjustment. The method can be executed by an image-based processing device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 5 , the method specifically includes the following:

[0093] S501, obtaining original feature information of the image data to be processed, wherein the original feature information includes statistical values ​​of preset visual indicators corresponding to the image data to be processed;

[0094] S502, constructing information to be input according to the original feature information, wherein the information to be input includes the original feature information and first difference information of statistical values ​​of the original feature information and candidate feature information for the same preset visual indicator, and the candidate feature information includes feature information obtained after adjustment based on the original feature information;

[0095] S503, inputting the information to be input into the image quality evaluation model to obtain the output result of the image quality evaluation model;

[0096] Among them, the image quality evaluation model is obtained using the training method described in any embodiment of the present disclosure.

[0097] S504, filtering target feature information from the information to be input according to the output result of the image quality evaluation model;

[0098] S505: Use the target feature information to perform corresponding processing on the image data to be processed, so that the target image data obtained after the processing satisfies the target feature information.

[0099] As mentioned above, image data may be adjusted during the production, transmission and consumption stages of image data. Accordingly, the image data to be processed may include captured image data, image data before encoding and decoding, or image data before display, etc. Before adjustment, multiple groups of candidate feature information can be determined based on the original feature information of the image data to be processed, that is, the possible adjusted feature information can be estimated, and the input information to be input to the image quality evaluation model can be constructed (the number of groups of information to be input is generally consistent with the number of groups of candidate feature information). The target feature information to be finally adopted is selected through the image quality score of the image quality evaluation model. The target feature information can be understood as the input information with the highest corresponding image quality score in the output result of the model or the image quality score higher than the preset score threshold. The values ​​of the corresponding preset visual indicators in the image data to be processed are adjusted so that the feature information determined based on the image data to be processed and the adjusted target image data is consistent with the target feature information, or the gap with the target feature information is within the preset gap range, thereby obtaining image data with better image quality.

[0100] Optionally, a method such as grid search may be used to search within a preset adjustment range corresponding to the original feature information to determine the candidate feature information. Different preset visual indicators may correspond to different preset adjustment ranges, which may be preset according to actual needs.

[0101] The image-based processing solution provided by the disclosed embodiments can use an image quality evaluation model to accurately pre-evaluate the image quality of adjusted image data, and can accurately estimate at each stage of the image data what kind of adjusted image data can obtain better image quality, and then select adjustment parameters whose image quality enhancement effect meets the expected requirements, so as to make targeted adjustments to the image to be adjusted. On the basis of ensuring that the image quality of the adjusted image data is improved, repeated adjustment operations can be reduced, image adjustment efficiency can be improved, and parameter adjustment costs can be reduced.

[0102] In an optional embodiment, if in the model training stage, the preset sample feature information also includes at least one of the second difference information, the third difference information and the fourth difference information, then the information to be input may include at least one of the following: the second difference information of the statistical values ​​of different types of the image data to be processed for the same preset visual indicator; the third difference information of the statistical values ​​of different types of the candidate feature information for the same preset visual indicator; the fourth difference information between the second difference information and the third difference information.

[0103] In an optional implementation, if the image quality assessment model is classified and trained according to image types during the training phase, the information to be input is input into the image quality assessment model corresponding to the image type to which the image data to be processed belongs.

[0104] In an optional implementation, the constructing the information to be input according to the original feature information includes: constructing the information to be input according to the importance ranking of each item of information to be input corresponding to the image quality evaluation model, the preset adjustment range corresponding to each item of information to be input, and the original feature information. The advantage of such a setting is that the feature parameters to be adjusted first can be guided according to the importance ranking, so that the target feature information can be quickly found, further improving the efficiency of image adjustment.

[0105] Exemplarily, as in the above example, the information item to be input that ranks highest in importance is hue_max-DIFF. Then, when constructing the information to be input, hue_max-DIFF may be adjusted first, so that the MOS score output by the model can be quickly improved.

[0106] Figure 6The present invention is a schematic diagram of a training device for an image quality evaluation model according to an embodiment of the present invention. The present invention can be applied to the case of training an image quality evaluation model. The device can be implemented in hardware and / or software and can be configured in an electronic device. Figure 6 The training device 600 of the image quality evaluation model specifically includes:

[0107] The feature information acquisition module 601 is used to acquire preset sample feature information corresponding to a training sample pair, wherein the training sample pair includes basic sample image data and control sample image data, the image data includes video data and / or picture data, and the preset sample feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic sample image data and the control sample image data, and first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicator;

[0108] A sample label acquisition module 602 is used to acquire a sample label corresponding to the training sample pair, wherein the sample label is used to represent a subjective evaluation score of the image quality of the control sample image data compared with the basic sample image data;

[0109] The feature information input module 603 is used to input the preset sample feature information into the preset image quality evaluation model to obtain the output result of the preset image quality evaluation model;

[0110] The model training module 604 is used to train the preset image quality evaluation model according to the output result of the preset image quality evaluation model and the sample label.

[0111] The training scheme of the image quality evaluation model provided by the embodiment of the present disclosure is used to input preset sample feature information in the preset image quality evaluation model, including statistical values ​​of preset visual indicators corresponding to basic sample image data and control sample image data, respectively, and first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicators. The preset sample feature information is used to fully characterize the visual features of the two groups of sample data and the differences in visual features, so that the preset image quality evaluation model can accurately fit the correlation between the feature information and the subjective evaluation score, and then can accurately predict the image quality score of the video or picture, reduce the cost of manual evaluation, and improve the scoring efficiency.

[0112] In an optional implementation, the preset visual indicator includes at least one of hue, saturation, brightness, contrast, color and texture.

[0113] In an optional implementation, the type of the statistical value includes at least one of a maximum value, an average value, and a minimum value.

[0114] In an optional implementation manner, the preset sample feature information further includes at least one of the following:

[0115] Second difference information of statistical values ​​of different types of the basic sample image data for the same preset visual index;

[0116] third difference information of different types of statistical values ​​of the control sample image data for the same preset visual index;

[0117] fourth difference information between the second difference information and the third difference information.

[0118] In an optional implementation, the preset image quality evaluation model includes an extreme gradient boosting tree XGBoost model.

[0119] In an optional implementation, the model setting parameters of the XGBoost model include at least one of the following:

[0120] The booster uses gbtree; the objective function is reg:gamma; the loss function is the mean absolute error loss; the gamma value is 0.1; the maximum depth of the decision tree is 8; the lambda value is 3; and the seed value is 1000.

[0121] In an optional implementation, the feature information acquisition module is specifically used to: determine the image type to be trained, and acquire preset sample feature information corresponding to the training sample pair corresponding to the image type;

[0122] The feature information input module is specifically used to: input the preset sample feature information into a preset image quality evaluation model corresponding to the image type to obtain an output result of the preset image quality evaluation model.

[0123] In an optional implementation manner, the preset sample characteristic information includes multiple items; the device further includes:

[0124] A model determination module, configured to determine a target image quality evaluation model according to the output result of the preset image quality evaluation model and the sample label after training the preset image quality evaluation model;

[0125] The ranking determination module is used to determine the importance ranking of each item of preset sample feature information according to the order from shallow to deep of the tree nodes in the target image quality evaluation model.

[0126] Figure 7The present invention is a schematic diagram of a structure of an image-based processing device provided according to an embodiment of the present invention. The present invention can be applied to the case where the image quality is evaluated by using an image quality evaluation model. The device can be implemented in hardware and / or software and can be configured in an electronic device. Figure 7 , the image-based processing device 700 specifically includes:

[0127] The information acquisition module 701 is used to acquire preset feature information corresponding to an image data pair, wherein the image data pair includes basic image data and reference image data, the image data includes video data and / or picture data, and the preset feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic image data and the reference image data, and first difference information between the statistical values ​​of the basic image data and the reference image data for the same preset visual indicator;

[0128] The information input module 702 is used to input the preset feature information into the image quality evaluation model to obtain the output result of the image quality evaluation model, wherein the image quality evaluation model is obtained using the training device described in any embodiment of the present disclosure, and the output result of the image quality evaluation model is used to represent the predicted evaluation score of the image quality of the control image data compared to the basic image data.

[0129] The image-based processing solution provided by the embodiments of the present disclosure adopts an image quality evaluation model to accurately evaluate the image quality of adjusted image data. It can accurately determine at each stage of the image data whether the adjusted image data has obtained better image quality, whether the image quality enhancement effect meets expectations, or whether image quality loss is caused due to the adjustment processing, etc., thereby reducing the cost of manual evaluation and accurately evaluating the impact of image adjustment on user perception.

[0130] Figure 8 is a schematic diagram of the structure of another image-based processing device provided according to an embodiment of the present disclosure. The embodiment of the present disclosure can be applied to the case where an image quality evaluation model is used to guide image adjustment. The device can be implemented in hardware and / or software and can be configured in an electronic device. Figure 8 , the image-based processing device 800 specifically includes:

[0131] The original feature information acquisition module 801 is used to acquire the original feature information of the image data to be processed, wherein the original feature information includes the statistical value of the preset visual index corresponding to the image data to be processed;

[0132] An information construction module 802 is used to construct information to be input according to the original feature information, wherein the information to be input includes the original feature information and also includes first difference information of statistical values ​​of the original feature information and candidate feature information for the same preset visual indicator, and the candidate feature information includes feature information obtained after adjustment based on the original feature information;

[0133] An information input module 803 for inputting the information to be input into an image quality evaluation model to obtain an output result of the image quality evaluation model, wherein the image quality evaluation model is obtained using the training device described in any embodiment of the present disclosure;

[0134] A target feature information screening module 804 is used to screen target feature information from the information to be input according to the output result of the image quality evaluation model;

[0135] The image processing module 805 is used to perform corresponding processing on the to-be-processed image data using the target feature information, so that the target image data obtained after the processing satisfies the target feature information.

[0136] The image-based processing solution provided by the disclosed embodiments can use an image quality evaluation model to accurately pre-evaluate the image quality of adjusted image data, and can accurately estimate at each stage of the image data what kind of adjusted image data can obtain better image quality, and then select adjustment parameters whose image quality enhancement effect meets the expected requirements, so as to make targeted adjustments to the image to be adjusted. On the basis of ensuring that the image quality of the adjusted image data is improved, repeated adjustment operations can be reduced, image adjustment efficiency can be improved, and parameter adjustment costs can be reduced.

[0137] In an optional implementation, the information construction module is specifically used to:

[0138] The information to be input is constructed according to the importance ranking of each item of information to be input corresponding to the image quality evaluation model, the preset adjustment range corresponding to each item of information to be input, and the original feature information.

[0139] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0140] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0141] Fig. 9A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0142] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0143] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0144] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the training method of the image quality assessment model or the image-based processing method. For example, in some embodiments, the training method of the image quality assessment model or the image-based processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the training method of the image quality assessment model or the image-based processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute a training method of an image quality assessment model or an image-based processing method in any other appropriate manner (eg, by means of firmware).

[0145] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0147] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0149] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0150] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services. The server may also be a server of a distributed system, or a server combined with a blockchain.

[0151] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, knowledge graph technology, and other major directions.

[0152] Cloud computing refers to a technology system that uses network access to elastically scalable shared physical or virtual resource pools. Resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for technical applications such as artificial intelligence and blockchain, as well as model training.

[0153] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution provided by this disclosure can be achieved, and this document does not limit this.

[0154] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for an image quality evaluation model, comprising: Obtaining preset sample feature information corresponding to a training sample pair, and obtaining a sample label corresponding to the training sample pair, wherein the training sample pair includes basic sample image data and control sample image data, the image data includes video data and / or picture data, the preset sample feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic sample image data and the control sample image data, and first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicator, and the sample label is used to represent a subjective evaluation score of the image quality of the control sample image data compared with the basic sample image data; Inputting the preset sample feature information into a preset image quality evaluation model to obtain an output result of the preset image quality evaluation model; Training the preset image quality evaluation model according to the output result of the preset image quality evaluation model and the sample label; The control sample image data is obtained by adjusting the values ​​of one or more visual indicators based on the basic sample image data; the control sample image data and the basic sample image data correspond to the same image content.

2. The method according to claim 1, wherein: The preset visual index includes at least one of hue, saturation, brightness, contrast, color and texture.

3. The method according to any one of claims 1-2, wherein: The type of the statistical value includes at least one of a maximum value, an average value, and a minimum value.

4. The method according to any one of claims 1-2, wherein: The preset sample feature information also includes at least one of the following: second difference information of different types of statistical values ​​of the basic sample image data for the same preset visual index; third difference information of different types of statistical values ​​of the control sample image data for the same preset visual index; fourth difference information between the second difference information and the third difference information.

5. The method according to any one of claims 1-2, wherein: The preset image quality evaluation model includes an extreme gradient boosting tree XGBoost model.

6. The method according to claim 5, wherein: The model setting parameters of the XGBoost model include at least one of the following: The booster uses gbtree; the objective function is reg:gamma; the loss function is the mean absolute error loss; the gamma value is 0.1; the maximum depth of the decision tree is 8; the lambda value is 3; and the seed value is 1000.

7. The method according to any one of claims 1-2, wherein: The obtaining of preset sample feature information corresponding to the training sample pair includes: Determine the image type to be trained, and obtain preset sample feature information corresponding to the training sample pair corresponding to the image type; The step of inputting the preset sample feature information into a preset image quality evaluation model to obtain an output result of the preset image quality evaluation model includes: The preset sample feature information is input into a preset image quality evaluation model corresponding to the image type to obtain an output result of the preset image quality evaluation model.

8. The method according to claim 6, wherein: The preset sample feature information includes multiple items; after the preset image quality evaluation model is trained according to the output result of the preset image quality evaluation model and the sample label, it also includes: Determine the target image quality evaluation model based on the training results; The importance ranking of each item of preset sample feature information is determined according to the order from shallow to deep of the tree nodes in the target image quality evaluation model.

9. An image-based processing method comprising: Acquire preset feature information corresponding to an image data pair, wherein the image data pair includes basic image data and reference image data, the image data includes video data and / or picture data, and the preset feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic image data and the reference image data, and first difference information of the statistical values ​​of the basic image data and the reference image data for the same preset visual indicator; The preset feature information is input into an image quality evaluation model to obtain an output result of the image quality evaluation model, wherein the image quality evaluation model is obtained using the training method described in any one of claims 1-8, and the output result of the image quality evaluation model is used to represent a predicted evaluation score of the image quality of the control image data compared to the basic image data.

10. An image-based processing method comprising: Acquiring original feature information of the image data to be processed, wherein the original feature information includes a statistical value of a preset visual indicator corresponding to the image data to be processed; Constructing information to be input according to the original feature information, wherein the information to be input includes the original feature information and also includes first difference information of statistical values ​​of the original feature information and candidate feature information for the same preset visual indicator, and the candidate feature information includes feature information obtained after adjustment based on the original feature information; Inputting the information to be input into an image quality evaluation model to obtain an output result of the image quality evaluation model, wherein the image quality evaluation model is obtained by using the training method according to any one of claims 1 to 8; Filtering target feature information from the information to be input according to the output result of the image quality evaluation model; The target feature information is used to perform corresponding processing on the image data to be processed, so that the target image data obtained after the processing satisfies the target feature information.

11. The method according to claim 10, wherein: The constructing the information to be input according to the original feature information includes: The information to be input is constructed according to the importance ranking of each item of information to be input corresponding to the image quality evaluation model, the preset adjustment range corresponding to each item of information to be input, and the original feature information.

12. A training device for an image quality evaluation model, comprising: A feature information acquisition module, used to acquire preset sample feature information corresponding to a training sample pair, wherein the training sample pair includes basic sample image data and control sample image data, the image data includes video data and / or picture data, and the preset sample feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic sample image data and the control sample image data, and first difference information of the statistical values ​​of the basic sample image data and the control sample image data for the same preset visual indicator; A sample label acquisition module, used to acquire a sample label corresponding to the training sample pair, wherein the sample label is used to represent a subjective evaluation score of the image quality of the control sample image data compared with the basic sample image data; A feature information input module, used to input the preset sample feature information into a preset image quality evaluation model to obtain an output result of the preset image quality evaluation model; A model training module, used for training the preset image quality evaluation model according to the output result of the preset image quality evaluation model and the sample label; The control sample image data is obtained by adjusting the values ​​of one or more visual indicators based on the basic sample image data; the control sample image data and the basic sample image data correspond to the same image content.

13. The device according to claim 12, wherein: The preset visual index includes at least one of hue, saturation, brightness, contrast, color and texture.

14. The device according to any one of claims 12 to 13, wherein: The type of the statistical value includes at least one of a maximum value, an average value, and a minimum value.

15. The device according to any one of claims 12-13, wherein: The preset sample feature information also includes at least one of the following: second difference information of different types of statistical values ​​of the basic sample image data for the same preset visual index; third difference information of different types of statistical values ​​of the control sample image data for the same preset visual index; fourth difference information between the second difference information and the third difference information.

16. The device according to any one of claims 12 to 13, wherein: The preset image quality evaluation model includes an extreme gradient boosting tree XGBoost model.

17. The device according to claim 16, wherein: The model setting parameters of the XGBoost model include at least one of the following: The booster uses gbtree; the objective function is reg:gamma; the loss function is the mean absolute error loss; the gamma value is 0.1; the maximum depth of the decision tree is 8; the lambda value is 3; and the seed value is 1000.

18. The device according to any one of claims 12 to 13, wherein: The feature information acquisition module is specifically used to: determine the image type to be trained, and obtain preset sample feature information corresponding to the training sample pair corresponding to the image type; The feature information input module is specifically used to: input the preset sample feature information into a preset image quality evaluation model corresponding to the image type to obtain an output result of the preset image quality evaluation model.

19. The device according to claim 17, wherein: The preset sample characteristic information includes multiple items; the device also includes: A model determination module, configured to determine a target image quality evaluation model according to the output result of the preset image quality evaluation model and the sample label after training the preset image quality evaluation model; The ranking determination module is used to determine the importance ranking of each item of preset sample feature information according to the order from shallow to deep of the tree nodes in the target image quality evaluation model.

20. An image-based processing device, comprising: An information acquisition module, configured to acquire preset feature information corresponding to an image data pair, wherein the image data pair includes basic image data and reference image data, the image data includes video data and / or picture data, and the preset feature information includes statistical values ​​of preset visual indicators respectively corresponding to the basic image data and the reference image data, and first difference information between the statistical values ​​of the basic image data and the reference image data for the same preset visual indicator; An information input module is used to input the preset feature information into an image quality evaluation model to obtain an output result of the image quality evaluation model, wherein the image quality evaluation model is obtained using a training device as described in any one of claims 12-19, and the output result of the image quality evaluation model is used to represent a predicted evaluation score of the image quality of the control image data compared to the basic image data.

21. An image-based processing device, comprising: An original feature information acquisition module, used to acquire original feature information of the image data to be processed, wherein the original feature information includes a statistical value of a preset visual indicator corresponding to the image data to be processed; An information construction module, configured to construct information to be input according to the original feature information, wherein the information to be input includes the original feature information and first difference information of statistical values ​​of the original feature information and candidate feature information for the same preset visual indicator, and the candidate feature information includes feature information obtained after adjustment based on the original feature information; An input module for information to be input, used for inputting the information to be input into an image quality evaluation model to obtain an output result of the image quality evaluation model, wherein the image quality evaluation model is obtained by using the training device according to any one of claims 12 to 19; A target feature information screening module, used for screening target feature information from the information to be input according to the output result of the image quality evaluation model; The image processing module is used to use the target feature information to perform corresponding processing on the image data to be processed so that the target image data obtained after the processing satisfies the target feature information.

22. The device according to claim 21, wherein The information construction module is specifically used for: The information to be input is constructed according to the importance ranking of each item of information to be input corresponding to the image quality evaluation model, the preset adjustment range corresponding to each item of information to be input, and the original feature information.

23. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.

Images