Image processing methods, apparatus and storage media

Abstract

This application relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus, and storage medium. The method includes: processing a acquired raw image according to ISP parameters to obtain a target image; based on the target image, calling a pre-trained image quality assessment model to output a target assessment result, the target assessment result indicating the image quality of the assessed target image; and adjusting the ISP parameters based on the target assessment result. This application embodiment performs image quality assessment on the target image by calling an image quality assessment model, outputting a target assessment result, and automatically adjusting the ISP parameters based on the target assessment result, ensuring the debugging effect of the ISP parameters and improving the image quality after subsequent ISP processing. The method and apparatus provided in this application embodiment can be applied to intelligent vehicles, new energy vehicles, etc.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus and storage medium. Background Technology

[0002] In the field of autonomous driving, the quality of the images output by cameras depends not only on the camera module and manufacturing process itself, but also on the image signal processor (ISP) technology.

[0003] In related technologies, such as Figure 1 As shown, the image imaging process includes: capturing a scene through lens 11, projecting the generated optical image onto the surface of photosensitive sensor 12, then converting the optical image into an analog electrical signal via photoelectric conversion, denoising the analog electrical signal, converting it into a digital image signal via analog-to-digital converter 13, inputting the digital image signal into a digital signal processing chip (DSP) 14, processing it, outputting YUV (or RGB) format data, and then transmitting it to the central processing unit (CPU) via I / O interface for further processing, ultimately converting it into an image displayed on the screen of the electronic device. The DSP 14's structural framework includes an image signal processor (ISP), an image decoder (JPEG encoder), and a USB device controller (USB).

[0004] The Image Sensor (ISP) is used for post-processing of data output from the photosensitive sensor. Its main functions include black level compensation, lens shading correction, bad pixel correction, color interpolation, noise removal, white balance correction, color correction, gamma correction, color space conversion (such as RGB to YUV), color noise removal and edge enhancement in the YUV color space, color and contrast enhancement, and automatic exposure control. The ISP is essential for the camera to reproduce scene details well under different optical conditions, and ISP technology largely determines the image quality of the camera.

[0005] Currently, the image quality of cameras in the field of autonomous driving depends on the ISP parameters. However, the debugging effect of ISP parameters depends on the debugging environment, parameter calibration method, and the experience of the debugging personnel, resulting in poor debugging results. Summary of the Invention

[0006] In view of this, an image processing method, apparatus and storage medium are proposed, which can automatically adjust ISP parameters according to the target evaluation results output by the image quality assessment model, ensuring the debugging effect of ISP parameters and improving the image quality after subsequent ISP processing.

[0007] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0008] The target image is obtained by processing the acquired raw image according to the ISP parameters;

[0009] Based on the target image, a pre-trained image quality assessment model is invoked to output the target assessment result. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the assessed target image.

[0010] The ISP parameters are adjusted based on the target evaluation results.

[0011] In this implementation, the acquired raw image is processed according to the ISP parameters to obtain the target image. Based on the target image, the image quality assessment model is called to output the target assessment result used to indicate the image quality. The ISP parameters are automatically adjusted according to the target assessment result, which ensures the debugging effect of the ISP parameters and improves the image quality after subsequent ISP processing.

[0012] In one possible implementation, the target evaluation result includes objective quality information and subjective quality information of the target image;

[0013] The objective quality information indicates the quality assessment of the target image in terms of objective indicators, while the subjective quality information indicates the quality assessment of the target image in terms of subjective aesthetics.

[0014] In this implementation, the target evaluation result includes objective quality information and subjective quality information of the target image, so that the parameters output by the image quality evaluation model can indicate the quality evaluation status of the target image in terms of objective indicators and / or subjective aesthetics, further ensuring the evaluation effect of the image quality evaluation model.

[0015] In another possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer. The step of obtaining the target assessment result by calling the output of a pre-trained image quality assessment model based on the target image includes:

[0016] The target image is input into the scene classification layer to obtain the scene classification result;

[0017] The target image and the scene classification result are input into the objective scoring layer to obtain the objective quality information, and the target image and the scene classification result are input into the subjective scoring layer to obtain the subjective quality information.

[0018] In this implementation, a scene classification layer is introduced into the image quality assessment model. The electronic device inputs the target image to the scene classification layer to obtain the scene classification result, inputs the target image and the scene classification result to the objective scoring layer to obtain the objective quality information, and inputs the target image and the scene classification result to the subjective scoring layer to obtain the subjective quality information. This provides a multi-task image processing method so that the ISP parameters under different scene types can be automatically optimized to improve the image quality under each scene.

[0019] In another possible implementation, the objective quality information includes objective rating values ​​corresponding to a specified dimension, and the subjective quality information includes subjective rating values ​​corresponding to the specified dimension, wherein the specified dimension includes at least one of brightness dimension, color dimension, contrast dimension, sharpness and noise dimension.

[0020] In this implementation, objective quality information can include objective score values ​​corresponding to specified dimensions, and subjective quality information can include subjective score values ​​corresponding to specified dimensions, which enriches the output parameters of the image quality assessment model and further ensures the assessment effect of the image quality assessment model.

[0021] In another possible implementation, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise, and adjusting the ISP parameters based on the target evaluation result includes:

[0022] For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension.

[0023] Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

[0024] In this implementation, the target score value for the specified dimension is determined based on the objective and subjective score values ​​corresponding to the specified dimension. Based on the target score value and preset parameter threshold, the ISP parameters for the specified dimension are adjusted, thereby automatically optimizing the ISP parameters under different scene types, improving the image quality in various scenes, and automatically adapting to all-weather scenarios of autonomous driving, thus meeting the image quality requirements of autonomous driving perception algorithms.

[0025] In another possible implementation, before obtaining the target evaluation result by calling the output of a pre-trained image quality assessment model based on the target image, the method further includes:

[0026] The image quality assessment model is obtained by training on at least one set of sample data, each set of sample data including: the sample image and the correct assessment result pre-annotated therein.

[0027] In this implementation, an image quality assessment model is trained based on at least one set of sample data. Each set of sample data includes sample images and pre-labeled correct assessment results, so that the target assessment result can be output based on the trained image quality assessment model, thereby automatically adjusting the ISP parameters and providing a prerequisite for ISP parameter debugging.

[0028] In another possible implementation, prior to obtaining the image quality assessment model, the method further includes:

[0029] Obtain a training sample set, wherein the training sample set includes the at least one set of sample data groups;

[0030] For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model;

[0031] The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result;

[0032] The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0033] This implementation provides a possible scheme for training an image quality assessment model based on a training sample set. The image quality assessment model is trained by calculating the loss corresponding to at least one set of sample data, which ensures the evaluation effect of the trained image quality assessment model.

[0034] In another possible implementation, after obtaining the target evaluation result by calling the image quality assessment model based on the target image, the method further includes:

[0035] The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions;

[0036] The training sample set is filtered based on the annotation information to obtain the filtered training sample set;

[0037] The image quality assessment model is trained based on the selected training sample set to obtain an updated image quality assessment model.

[0038] This implementation provides an automatic annotation method for objective indicators, which can automatically annotate the objective annotation values ​​of the target image in a specified dimension. Based on the annotation information, the training sample set is filtered to optimize the sample quality in the training sample set, enhance the stability of the image quality assessment model, and improve the accuracy of subsequent image processing methods.

[0039] Secondly, embodiments of this application provide an image quality assessment method, the method comprising:

[0040] The target image is obtained by processing the acquired raw image according to the ISP parameters;

[0041] Based on the target image, a pre-trained image quality assessment model is invoked to output the target assessment result. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the assessed target image.

[0042] In one possible implementation, the target evaluation result includes objective quality information and subjective quality information of the target image;

[0043] The objective quality information indicates the quality assessment of the target image in terms of objective indicators, while the subjective quality information indicates the quality assessment of the target image in terms of subjective aesthetics.

[0044] In another possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer. The step of obtaining the target assessment result by calling the output of a pre-trained image quality assessment model based on the target image includes:

[0045] The target image is input into the scene classification layer to obtain the scene classification result;

[0046] The target image and the scene classification result are input into the objective scoring layer to obtain the objective quality information, and the target image and the scene classification result are input into the subjective scoring layer to obtain the subjective quality information.

[0047] In another possible implementation, the objective quality information includes objective rating values ​​corresponding to a specified dimension, and the subjective quality information includes subjective rating values ​​corresponding to the specified dimension, wherein the specified dimension includes at least one of brightness dimension, color dimension, contrast dimension, sharpness and noise dimension.

[0048] In another possible implementation, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise, and the method further includes:

[0049] For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension.

[0050] Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

[0051] In another possible implementation, before obtaining the target evaluation result by calling the output of a pre-trained image quality assessment model based on the target image, the method further includes:

[0052] The image quality assessment model is obtained by training it based on at least one set of sample data, each set of sample data including: the sample image and the pre-labeled correct assessment result.

[0053] In another possible implementation, prior to obtaining the image quality assessment model, the method further includes:

[0054] Obtain a training sample set, wherein the training sample set includes the at least one set of sample data groups;

[0055] For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model;

[0056] The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result;

[0057] The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0058] In another possible implementation, after obtaining the target evaluation result by calling the image quality assessment model based on the target image, the method further includes:

[0059] The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions;

[0060] The training sample set is filtered based on the annotation information to obtain the filtered training sample set;

[0061] The image quality assessment model is trained based on the selected training sample set to obtain an updated image quality assessment model.

[0062] Thirdly, embodiments of this application provide an automatic image quality enhancement method, the method comprising:

[0063] The target image is obtained by processing the acquired raw image according to the ISP parameters;

[0064] Based on the target image, a pre-trained image quality assessment model is invoked to output the target assessment result. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the assessed target image.

[0065] The ISP parameters are adjusted based on the target evaluation results;

[0066] The acquired first image is processed according to the adjusted ISP parameters to obtain the second image.

[0067] In one possible implementation, the target evaluation result includes objective quality information and subjective quality information of the target image;

[0068] The objective quality information indicates the quality assessment of the target image in terms of objective indicators, while the subjective quality information indicates the quality assessment of the target image in terms of subjective aesthetics.

[0069] In another possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer. The step of obtaining the target assessment result by calling the output of a pre-trained image quality assessment model based on the target image includes:

[0070] The target image is input into the scene classification layer to obtain the scene classification result;

[0071] The target image and the scene classification result are input into the objective scoring layer to obtain the objective quality information, and the target image and the scene classification result are input into the subjective scoring layer to obtain the subjective quality information.

[0072] In another possible implementation, the objective quality information includes objective rating values ​​corresponding to a specified dimension, and the subjective quality information includes subjective rating values ​​corresponding to the specified dimension, wherein the specified dimension includes at least one of brightness dimension, color dimension, contrast dimension, sharpness and noise dimension.

[0073] In another possible implementation, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise, and adjusting the ISP parameters based on the target evaluation result includes:

[0074] For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension.

[0075] Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

[0076] In another possible implementation, before obtaining the target evaluation result by calling the output of a pre-trained image quality assessment model based on the target image, the method further includes:

[0077] The image quality assessment model is obtained by training it based on at least one set of sample data, each set of sample data including: the sample image and the pre-labeled correct assessment result.

[0078] In another possible implementation, prior to obtaining the image quality assessment model, the method further includes:

[0079] Obtain a training sample set, wherein the training sample set includes the at least one set of sample data groups;

[0080] For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model;

[0081] The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result;

[0082] The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0083] In another possible implementation, after obtaining the target evaluation result by calling the image quality assessment model based on the target image, the method further includes:

[0084] The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions;

[0085] The training sample set is filtered based on the annotation information to obtain the filtered training sample set;

[0086] The image quality assessment model is trained based on the selected training sample set to obtain an updated image quality assessment model.

[0087] Fourthly, embodiments of this application provide an objective indicator labeling method, the method comprising:

[0088] The target image is obtained by processing the acquired raw image according to the ISP parameters;

[0089] The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions;

[0090] The training sample set is filtered based on the annotation information to obtain the filtered training sample set;

[0091] The pre-trained image quality assessment model is trained based on the selected training sample set to obtain an updated image quality assessment model, which is a neural network model used to assess image quality.

[0092] In one possible implementation, before training the pre-trained image quality assessment model based on the selected training sample set to obtain the updated image quality assessment model, the method further includes:

[0093] Based on the target image, the image quality assessment model is invoked to output the target assessment result. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the assessed target image.

[0094] The ISP parameters are adjusted based on the target evaluation results.

[0095] In another possible implementation, the target evaluation result includes objective quality information and subjective quality information of the target image;

[0096] The objective quality information indicates the quality assessment of the target image in terms of objective indicators, while the subjective quality information indicates the quality assessment of the target image in terms of subjective aesthetics.

[0097] In another possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer. The step of calling the image quality assessment model to output the target assessment result based on the target image includes:

[0098] The target image is input into the scene classification layer to obtain the scene classification result;

[0099] The target image and the scene classification result are input into the objective scoring layer to obtain the objective quality information, and the target image and the scene classification result are input into the subjective scoring layer to obtain the subjective quality information.

[0100] In another possible implementation, the objective quality information includes objective rating values ​​corresponding to a specified dimension, and the subjective quality information includes subjective rating values ​​corresponding to the specified dimension, wherein the specified dimension includes at least one of brightness dimension, color dimension, contrast dimension, sharpness and noise dimension.

[0101] In another possible implementation, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise, and adjusting the ISP parameters based on the target evaluation result includes:

[0102] For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension.

[0103] Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

[0104] In another possible implementation, before obtaining the target evaluation result by calling the output of a pre-trained image quality assessment model based on the target image, the method further includes:

[0105] The image quality assessment model is obtained by training it based on at least one set of sample data, each set of sample data including: the sample image and the pre-labeled correct assessment result.

[0106] In another possible implementation, prior to obtaining the image quality assessment model, the method further includes:

[0107] Obtain the training sample set, which includes the at least one set of sample data groups;

[0108] For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model;

[0109] The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result;

[0110] The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0111] Fifthly, embodiments of this application provide a camera, the camera including: an ISP and a processor;

[0112] The processor is configured to execute the method provided by the first aspect or any possible implementation thereof, or to execute the method provided by the second aspect or any possible implementation thereof, or to execute the method provided by the third aspect or any possible implementation thereof, or to execute the method provided by the fourth aspect or any possible implementation thereof.

[0113] Sixthly, embodiments of this application provide a vehicle, the vehicle comprising:

[0114] processor;

[0115] Memory used to store processor-executable instructions;

[0116] Wherein, the processor is configured to implement, when executing the instructions, the method provided by the first aspect or any possible implementation of the first aspect, or the method provided by the second aspect or any possible implementation of the second aspect, or the method provided by the third aspect or any possible implementation of the third aspect, or the method provided by the fourth aspect or any possible implementation of the fourth aspect.

[0117] In a seventh aspect, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0118] The processing unit is used to process the acquired raw image according to the ISP parameters to obtain the target image;

[0119] The output unit is used to call a pre-trained image quality assessment model to output a target assessment result based on the target image. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the target image being assessed.

[0120] An adjustment unit is used to adjust the ISP parameters based on the target evaluation results.

[0121] In one possible implementation, the target evaluation result includes objective quality information and subjective quality information of the target image;

[0122] The objective quality information indicates the quality assessment of the target image in terms of objective indicators, while the subjective quality information indicates the quality assessment of the target image in terms of subjective aesthetics.

[0123] In another possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer, and the output unit is further used for:

[0124] The target image is input into the scene classification layer to obtain the scene classification result;

[0125] The target image and the scene classification result are input into the objective scoring layer to obtain the objective quality information, and the target image and the scene classification result are input into the subjective scoring layer to obtain the subjective quality information.

[0126] In another possible implementation, the objective quality information includes objective rating values ​​corresponding to a specified dimension, and the subjective quality information includes subjective rating values ​​corresponding to the specified dimension, wherein the specified dimension includes at least one of brightness dimension, color dimension, contrast dimension, sharpness and noise dimension.

[0127] In another possible implementation, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise, and the adjustment unit is further configured to:

[0128] For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension.

[0129] Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

[0130] In another possible implementation, the apparatus further includes: an acquisition unit;

[0131] The acquisition unit is used to acquire the image quality assessment model, which is trained based on at least one set of sample data, each set of sample data including: the sample image and the pre-labeled correct assessment result.

[0132] In another possible implementation, the apparatus further includes: a training unit; the training unit is used for:

[0133] Obtain a training sample set, wherein the training sample set includes the at least one set of sample data groups;

[0134] For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model;

[0135] The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result;

[0136] The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0137] In another possible implementation, the apparatus further includes: a retraining unit; the retraining unit is configured to:

[0138] The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions;

[0139] The training sample set is filtered based on the annotation information to obtain the filtered training sample set;

[0140] The image quality assessment model is trained based on the selected training sample set to obtain an updated image quality assessment model.

[0141] Eighthly, embodiments of this application provide an image quality assessment apparatus, the apparatus comprising:

[0142] The processing unit is used to process the acquired raw image according to the ISP parameters to obtain the target image;

[0143] The output unit is used to call a pre-trained image quality assessment model to output a target assessment result based on the target image. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the assessed target image.

[0144] In one possible implementation, the target evaluation result includes objective quality information and subjective quality information of the target image;

[0145] The objective quality information indicates the quality assessment of the target image in terms of objective indicators, while the subjective quality information indicates the quality assessment of the target image in terms of subjective aesthetics.

[0146] In another possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer, and the output unit is further used for:

[0147] The target image is input into the scene classification layer to obtain the scene classification result;

[0148] The target image and the scene classification result are input into the objective scoring layer to obtain the objective quality information, and the target image and the scene classification result are input into the subjective scoring layer to obtain the subjective quality information.

[0149] In another possible implementation, the objective quality information includes objective rating values ​​corresponding to a specified dimension, and the subjective quality information includes subjective rating values ​​corresponding to the specified dimension, wherein the specified dimension includes at least one of brightness dimension, color dimension, contrast dimension, sharpness and noise dimension.

[0150] In another possible implementation, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise, and the device further includes: an adjustment unit; the adjustment unit is configured to:

[0151] For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension.

[0152] Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

[0153] In another possible implementation, the apparatus further includes: an acquisition unit; the acquisition unit is configured to:

[0154] The image quality assessment model is obtained by training it based on at least one set of sample data, each set of sample data including: the sample image and the pre-labeled correct assessment result.

[0155] In another possible implementation, the apparatus further includes: a training unit; the training unit is configured to:

[0156] Obtain a training sample set, wherein the training sample set includes the at least one set of sample data groups;

[0157] For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model;

[0158] The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result;

[0159] The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0160] In another possible implementation, the apparatus further includes a retraining unit, the retraining unit being configured to:

[0161] The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions;

[0162] The training sample set is filtered based on the annotation information to obtain the filtered training sample set;

[0163] The image quality assessment model is trained based on the selected training sample set to obtain an updated image quality assessment model.

[0164] Ninthly, embodiments of this application provide an automatic image quality enhancement device, the device comprising:

[0165] The first processing unit is used to process the acquired raw image according to the ISP parameters to obtain the target image;

[0166] The output unit is used to call a pre-trained image quality assessment model to output a target assessment result based on the target image. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the target image being assessed.

[0167] An adjustment unit is used to adjust the ISP parameters based on the target evaluation results;

[0168] The second processing unit is used to process the acquired first image according to the adjusted ISP parameters to obtain the second image.

[0169] In one possible implementation, the target evaluation result includes objective quality information and subjective quality information of the target image;

[0170] The objective quality information indicates the quality assessment of the target image in terms of objective indicators, while the subjective quality information indicates the quality assessment of the target image in terms of subjective aesthetics.

[0171] In another possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer, and the output unit is further used for:

[0172] The target image is input into the scene classification layer to obtain the scene classification result;

[0173] The target image and the scene classification result are input into the objective scoring layer to obtain the objective quality information, and the target image and the scene classification result are input into the subjective scoring layer to obtain the subjective quality information.

[0174] In another possible implementation, the objective quality information includes objective rating values ​​corresponding to a specified dimension, and the subjective quality information includes subjective rating values ​​corresponding to the specified dimension, wherein the specified dimension includes at least one of brightness dimension, color dimension, contrast dimension, sharpness and noise dimension.

[0175] In another possible implementation, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise, and the adjustment unit is further configured to:

[0176] For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension.

[0177] Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

[0178] In another possible implementation, the apparatus further includes: an acquisition unit; the acquisition unit is configured to:

[0179] The image quality assessment model is obtained by training it based on at least one set of sample data, each set of sample data including: the sample image and the pre-labeled correct assessment result.

[0180] In another possible implementation, the apparatus further includes: a training unit; the training unit is used for:

[0181] Obtain a training sample set, wherein the training sample set includes the at least one set of sample data groups;

[0182] For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model;

[0183] The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result;

[0184] The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0185] In another possible implementation, the apparatus further includes: a retraining unit; the retraining unit is configured to:

[0186] The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions;

[0187] The training sample set is filtered based on the annotation information to obtain the filtered training sample set;

[0188] The image quality assessment model is trained based on the selected training sample set to obtain an updated image quality assessment model.

[0189] Tenthly, embodiments of this application provide an objective indicator labeling device, the device comprising:

[0190] The processing unit is used to process the acquired raw image according to the ISP parameters to obtain the target image;

[0191] The calibration unit is used to automatically calibrate the target image to obtain annotation information, wherein the annotation information includes objective annotation values ​​corresponding to a specified dimension;

[0192] The filtering unit is used to filter the training sample set according to the annotation information to obtain the filtered training sample set;

[0193] The retraining unit is used to train the pre-trained image quality assessment model based on the selected training sample set to obtain an updated image quality assessment model, which is a neural network model used to assess image quality.

[0194] In one possible implementation, the device further includes:

[0195] The output unit is used to call the image quality assessment model to output a target assessment result based on the target image. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the assessed target image.

[0196] An adjustment unit is used to adjust the ISP parameters based on the target evaluation results.

[0197] In another possible implementation, the target evaluation result includes objective quality information and subjective quality information of the target image;

[0198] The objective quality information indicates the quality assessment of the target image in terms of objective indicators, while the subjective quality information indicates the quality assessment of the target image in terms of subjective aesthetics.

[0199] In another possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer, and the output unit is further used for:

[0200] The target image is input into the scene classification layer to obtain the scene classification result;

[0201] The target image and the scene classification result are input into the objective scoring layer to obtain the objective quality information, and the target image and the scene classification result are input into the subjective scoring layer to obtain the subjective quality information.

[0202] In another possible implementation, the objective quality information includes objective rating values ​​corresponding to a specified dimension, and the subjective quality information includes subjective rating values ​​corresponding to the specified dimension, wherein the specified dimension includes at least one of brightness dimension, color dimension, contrast dimension, sharpness and noise dimension.

[0203] In another possible implementation, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise, and the adjustment unit is further configured to:

[0204] For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension.

[0205] Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

[0206] In another possible implementation, the apparatus further includes: an acquisition unit;

[0207] The acquisition unit is used to acquire the image quality assessment model, which is trained based on at least one set of sample data, each set of sample data including: the sample image and the pre-labeled correct assessment result.

[0208] In another possible implementation, the apparatus further includes: a training unit; the training unit is used for:

[0209] Obtain the training sample set, which includes the at least one set of sample data groups;

[0210] For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model;

[0211] The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result;

[0212] The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

[0213] Eleventhly, embodiments of this application provide a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the method provided by the first aspect or any possible implementation of the first aspect, or implement the method provided by the second aspect or any possible implementation of the second aspect, or implement the method provided by the third aspect or any possible implementation of the third aspect, or implement the method provided by the fourth aspect or any possible implementation of the fourth aspect.

[0214] In a twelfth aspect, embodiments of this application provide a computer program product comprising computer-readable code or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code is executed in an electronic device, a processor in the electronic device executes the method provided by the first aspect or any possible implementation thereof, or executes the method provided by the second aspect or any possible implementation thereof, or executes the method provided by the third aspect or any possible implementation thereof, or executes the method provided by the fourth aspect or any possible implementation thereof. Attached Figure Description

[0215] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.

[0216] Figure 1 A schematic diagram of the image imaging process in related technologies is shown.

[0217] Figure 2 A schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this application is shown.

[0218] Figure 3 A flowchart of an image processing method provided in an exemplary embodiment of this application is shown.

[0219] Figure 4 A flowchart of an image processing method provided by another exemplary embodiment of this application is shown.

[0220] Figure 5 A schematic diagram of the structure of an image quality assessment model provided in an exemplary embodiment of this application is shown.

[0221] Figure 6 A flowchart illustrating the training process of an image quality assessment model provided in an exemplary embodiment of this application is shown.

[0222] Figure 7 A flowchart of an image processing method provided by another exemplary embodiment of this application is shown.

[0223] Figure 8 A flowchart of an image processing method provided by another exemplary embodiment of this application is shown.

[0224] Figure 9 A block diagram of an image processing apparatus provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0225] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0226] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0227] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0228] First, the application scenarios involved in this application will be introduced.

[0229] The methods and apparatus provided in this application can be applied to fields such as intelligent vehicles and new energy vehicles. The following description uses an electronic device as an example to illustrate the method provided in this application. Please refer to... Figure 2 This illustration shows a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this application. This electronic device can be applied to embedded platforms for autonomous driving (AD) or advanced driver assistance systems (ADAS).

[0230] The electronic device may include a vehicle 21, which may be a vehicle with wireless communication capabilities. These wireless communication capabilities may be located in the vehicle's onboard terminal, onboard module, onboard unit, chip (system), or other components or parts. In the embodiments of this application, the vehicle 21 may be in an autonomous driving state, meaning that the vehicle 21 is fully autonomously driven, requiring no driver control or only minimal driver control.

[0231] The vehicle 21 is equipped with a camera 22, which is used to optimize the ISP parameters using a pre-trained image quality assessment model before the electronic device leaves the factory. Optionally, the camera 22 includes a lens, an ISP, and a processor. The lens is used to acquire the original image, and the ISP is used to perform post-processing on the original image. Its main functions include black level compensation, lens correction, bad pixel correction, color interpolation, noise removal, white balance correction, color correction, gamma correction, color space conversion (e.g., RGB to YUV), color noise removal and edge enhancement in the YUV color space, color and contrast enhancement, and automatic exposure control. The processor is a newly added processor for image quality enhancement. This processor is used to automatically adjust the ISP parameters according to the target evaluation results output by the image quality assessment model, ensuring the tuning effect of the ISP parameters and thus improving the image quality after subsequent ISP processing.

[0232] In this embodiment, the ISP is used to process the acquired raw image to obtain the target image; the processor is used to acquire the target image, and based on the target image, call the pre-trained image quality assessment model to output the target assessment result. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the assessed target image; the ISP parameters are adjusted based on the target assessment result.

[0233] Vehicle 21 may also be equipped with at least one other sensor 23, such as vehicle-mounted radar (e.g., millimeter-wave radar, lidar, ultrasonic radar, etc.), rain sensor, vehicle attitude sensor (e.g., gyroscope), inertial measurement unit (IMU), global navigation satellite system (GNSS), etc. These sensors 23 may be installed on one vehicle 21 or on multiple vehicles 21.

[0234] The vehicle 21 may also be equipped with an autonomous driving system 24, which can be used to generate an autonomous driving strategy to deal with road conditions based on data collected by sensors, and realize the autonomous driving of the vehicle 21 according to the generated strategy.

[0235] The vehicle 21 may also be equipped with a human machine interface (HMI) 25, which can be used to announce the current road conditions and the strategies adopted by the autonomous driving system 24 for the vehicle 21 through visual icons and voice broadcasts, so as to remind the relevant drivers and passengers.

[0236] In one possible implementation, the electronic device in this embodiment may further include a server 20. The server 20 may be located on the vehicle 21 as an on-board computing unit, or it may be located in the cloud. It may be a physical device or a virtual device such as a virtual machine or container, and it has wireless communication capabilities. For example, the server 20 may be a virtual device provided by pooling resources from multiple locations (spatially decoupled). The wireless communication capability may be set in the chip (system) or other components of the server 20. The server 20 and the vehicle 21 can communicate wirelessly, for example, through mobile communication technologies such as 2G / 3G / 4G / 5G, as well as wireless communication methods such as Wi-Fi, Bluetooth, frequency modulation (FM), data radio, and satellite communication. For example, in a test, the server 20 may be mounted on the vehicle 21 and communicate with the vehicle 21 wirelessly. Through the communication between the vehicle 21 and the server 20, the server 20 can collect data collected by one or more sensors on the vehicle 21 or installed on the road or elsewhere, perform calculations, and send the calculation results back to the corresponding vehicle 21.

[0237] The image processing method provided in the embodiments of this application will now be described using several exemplary embodiments.

[0238] Please refer to Figure 3 It illustrates a flowchart of an image processing method provided in an exemplary embodiment of this application, in which the method is used for... Figure 2 The following example uses an electronic device. The method includes the following steps.

[0239] Step 301: Process the acquired raw image according to the ISP parameters to obtain the target image.

[0240] Optionally, the electronic device acquires the original image, including: acquiring an optical image through a camera, and obtaining a digital image signal, i.e., the original image, through preprocessing by a photosensitive sensor and an analog-to-digital converter.

[0241] Optionally, the electronic device acquires optical images via a camera in real time or at preset time intervals. The preset time interval can be a default setting or a custom setting; this embodiment does not limit the specific settings.

[0242] Optionally, the electronic device processes the acquired raw image through the ISP, including: the electronic device performs at least one of the following processing on the acquired raw image through the ISP: black level compensation, lens correction, bad pixel correction, color interpolation, noise removal, white balance correction, color correction, gamma correction, color space conversion (e.g., RGB to YUV), color noise removal and edge enhancement in the YUV color space, color and contrast enhancement, and automatic exposure control.

[0243] Optionally, the original image is in Bayer format, and the target image is in YUV or RGB format.

[0244] The ISP parameters are image signal processing parameters. Optionally, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise.

[0245] The image quality of the target image is related to the ISP parameters. Optionally, the image quality of the target image indicates its performance in terms of objective metrics and / or subjective aesthetics.

[0246] Optionally, the image quality of the target image indicates the objective quality assessment and / or subjective quality assessment of the target image in a specified dimension. Illustratively, the specified dimension includes at least one of the following: brightness dimension, color dimension, contrast dimension, sharpness dimension, and noise dimension.

[0247] Step 302: Based on the target image, call the pre-trained image quality assessment model to output the target assessment result. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the target image being assessed.

[0248] The electronic device acquires a pre-trained image quality assessment model, inputs the target image into the image quality assessment model, and outputs the target assessment result.

[0249] Electronic devices acquire pre-trained image quality assessment models, including but not limited to the following two possible implementations.

[0250] In one possible implementation, the electronic device pre-trains and stores an image quality assessment model. The electronic device then retrieves the stored image quality assessment model.

[0251] In another possible implementation, the server pre-trains and stores the image quality assessment model, and the electronic device retrieves the image quality assessment model from the server.

[0252] The image quality assessment model is a model obtained by training a neural network using sample images and correct assessment results. That is, the image quality assessment model is determined based on sample images and correct assessment results. The correct assessment results include pre-labeled assessments of the image quality of the sample images. For example, the neural network is a multi-task convolutional neural network (CNN). This application does not limit the type of neural network.

[0253] An image quality assessment model is used to transform an input target image into a target assessment result, which indicates the image quality of the target image being assessed.

[0254] Image quality assessment models are used to represent the correlation between target images and target assessment results.

[0255] The image quality assessment model is a pre-defined mathematical model, which includes model coefficients between the target image and the target assessment result. These model coefficients can be fixed values, dynamically modified over time, or dynamically modified according to the usage scenario.

[0256] One possible implementation is that the image quality assessment model can be a convolutional neural network model capable of assessing the image quality of a target image.

[0257] It should be noted that the training process of the image quality assessment model can be found in the relevant details in the following embodiments, which will not be described here.

[0258] Optionally, the target evaluation results include objective quality information and / or subjective quality information of the target image, wherein the objective quality information indicates the quality evaluation of the target image on objective indicators, and the subjective quality information indicates the quality evaluation of the target image on subjective aesthetics.

[0259] Optionally, the objective quality information includes objective rating values ​​corresponding to the specified dimensions, and the subjective quality information includes subjective rating values ​​corresponding to the specified dimensions. The specified dimensions include at least one of the following: brightness dimension, color dimension, contrast dimension, sharpness dimension, and noise dimension.

[0260] Step 303: Adjust the ISP parameters based on the target evaluation results.

[0261] The electronic device adjusts the ISP parameters based on the target evaluation results, thereby continuously improving the image quality of the output image.

[0262] Optionally, the electronic device processes the acquired first image according to the adjusted ISP parameters to obtain a second image, wherein the image quality of the second image is higher than that of the third image, the second image is the image processed based on the adjusted ISP parameters, and the third image is the image processed based on the ISP parameters before adjustment.

[0263] Optionally, ISP parameters include at least one of brightness, color, contrast, sharpness, and noise.

[0264] Optionally, the electronic device adjusts the ISP parameters based on the target evaluation results, including: for each specified dimension, determining the target score value corresponding to the specified dimension based on the objective score value and subjective score value corresponding to the specified dimension; adjusting the ISP parameters corresponding to the specified dimension based on the target score value corresponding to the specified dimension and a preset parameter threshold, wherein the preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification results.

[0265] It should be noted that, for different ISP parameter types, the way electronic devices adjust ISP parameters based on target evaluation results can be found in the relevant details in the following embodiments, which will not be introduced here.

[0266] In summary, the target image is obtained by processing the acquired raw image according to the ISP parameters. Based on the target image, the image quality assessment model is called to output the target assessment result used to indicate the image quality. The ISP parameters are automatically adjusted according to the target assessment result, which ensures the debugging effect of the ISP parameters and improves the image quality after subsequent ISP processing.

[0267] In one possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer. The scene classification layer classifies the target image into scenes and outputs the scene classification result; the objective scoring layer objectively scores the target image based on the scene classification result to obtain objective quality information; and the subjective scoring layer subjectively scores the target image based on the scene classification result to obtain subjective quality information. In one possible implementation, such as... Figure 4As shown, the image processing method includes, but is not limited to, the following steps: 1. Acquiring the original image; 2. Performing ISP optimization on the original image to obtain the optimized target image; 3. Performing scene classification on the target image, i.e., inputting the target image into the scene classification layer of the image quality assessment model to obtain the scene classification result; 4. Performing image quality assessment based on the target image and the scene classification result, i.e., inputting the target image and the scene classification result into the objective scoring layer to obtain objective quality information, and inputting the target image and the scene classification result into the subjective scoring layer to obtain subjective quality information; 5. Automatically adjusting the ISP parameters based on the objective quality information and the subjective quality information. This method may also include: 6. While the ISP outputs the target image, automatically calibrating the objective indicators of the target image to obtain annotation information, so as to subsequently optimize the training sample set of the image quality assessment model based on the annotation information, thereby enhancing the stability and accuracy of the image quality assessment model.

[0268] Taking the image quality assessment model as an example of a multi-task CNN model, in an illustrative example, such as... Figure 5 As shown, the image quality assessment model includes a CNN encoder 51, a scene classification decoder 52, an objective scoring decoder 53, and a subjective scoring decoder 54. The CNN encoder 51 includes convolutional and pooling layers of a classification network, and uses three convolutional layers for downsampling. The CNN encoder 51 extracts image features from the target image and inputs these features into the scene classification decoder 52. The scene classification decoder 52, also called the scene classification layer, is a fully connected layer with a softmax activation function. It classifies the target image based on the input image features and outputs the scene classification result. The objective scoring decoder 53, also called the objective scoring layer, objectively scores the target image based on the input scene classification result to obtain objective quality information. Internally, the objective scoring decoder 53 uses a scaling layer based on scale invariance to achieve accurate scoring of objective metrics. The subjective scoring decoder 54, also called the subjective scoring layer, subjectively scores the target image based on the input scene classification result to obtain subjective quality information. The subjective scoring decoder 54 upsamples the encoder features using three transposed convolutional layers.

[0269] It should be noted that before an electronic device can acquire an image quality assessment model, it needs to be trained on a training sample set to obtain the image quality assessment model. The training process of the image quality assessment model is described below.

[0270] In one possible implementation, such as Figure 6 As shown, the training process for the image quality assessment model includes the following steps:

[0271] Step 601: Obtain the training sample set, which includes at least one set of sample data.

[0272] The image quality assessment model is trained on at least one set of sample data, each set of sample data including: sample images and pre-annotated correct assessment results.

[0273] The sample images are images from an image database, and the correct evaluation results include pre-annotated evaluation results of the image quality of the sample images.

[0274] Step 602: For each of the at least one set of sample data groups, input the sample image into the original parameter model to obtain the training result. The original parameter model is a multi-task neural network model.

[0275] Optionally, the original parameter model is built based on a neural network model, for example, the original parameter model is built based on a CNN model.

[0276] To illustrate, for each set of sample data, the electronic device creates an input-output pair corresponding to that set of sample data. The input parameter of the input-output pair is the sample image in that set of sample data, and the target parameter is the correct evaluation result in that set of sample data. The electronic device inputs the input parameter into the original parameter model to obtain the training result.

[0277] Optionally, the input-output pair can be represented by feature vectors.

[0278] Step 603: Compare the training results with the correct evaluation results to obtain the calculated loss, which indicates the error between the training results and the correct evaluation results.

[0279] In one possible implementation, the loss is calculated as the sum of the calculated losses for scene classification, subjective metrics, and objective metrics. For each set of sample data, the training results include: scene classification prediction results, subjective quality prediction information, and objective quality prediction information. The correct evaluation results include: correct scene classification results, correct subjective quality information, and correct objective quality information. Correct subjective quality information includes the correct subjective rating value corresponding to the specified dimension, and correct objective quality information includes the correct objective rating value corresponding to the specified dimension.

[0280] Optionally, cross-entropy is used as the computational loss for scene classification, and this computational loss is... class The formula for calculating (p,q) is as follows:

[0281]

[0282] Where p is the scene classification prediction result, q is the correct scene classification result, and c is the set of scene types.

[0283] Optionally, the sum of the cross-entropy of confidence and the loss of the objective indicator is used as the computational loss corresponding to the objective indicator. box The formula for calculating (p,q) is as follows:

[0284]

[0285] Where p is the objective quality prediction information, q is the correct objective quality information, x, y, m, and n represent the brightness dimension, color dimension, contrast dimension, sharpness dimension, and noise dimension, respectively, i indicates a single scene, and the value of i is a positive integer, and I indicates the set of scene types.

[0286] Optionally, empirical mode decomposition-based loss (EMD-based loss) can be used as the computational loss corresponding to the subjective index. This involves predicting the subjective index by generating a rating histogram based on the probability distribution of human evaluations of the sample images. The calculation formula is as follows:

[0287]

[0288] Where p represents subjective quality prediction information. This refers to accurate subjective quality information, where k is the set of specified indicators, r represents the root power, and r is a pre-set fixed value. CDF p (k) represents the cumulative probability of predicting subjective ratings; the higher the predicted subjective rating, the higher the CDF. p The larger the value of (k), the more likely the softmax function will be used to ensure that each independent probability is greater than 0 during the prediction process.

[0289] Step 604: Train an image quality assessment model based on the calculated loss corresponding to at least one set of sample data groups.

[0290] Optionally, the electronic device determines the gradient direction of the image quality assessment model based on the calculated loss using a backpropagation algorithm, and updates the model parameters in the image quality assessment model layer by layer from the output layer of the image quality assessment model forward.

[0291] Based on the image quality assessment model trained in the above embodiments, image quality assessment is performed on the target image. Please refer to... Figure 7 It illustrates a flowchart of an image processing method provided in another exemplary embodiment of this application, which is used in this embodiment for... Figure 1 The following example uses an electronic device. The method includes the following steps.

[0292] Step 701: Process the acquired raw image according to the ISP parameters to obtain the target image.

[0293] It should be noted that the process by which the electronic device processes the acquired raw image according to the ISP parameters to obtain the target image can be referred to the relevant details in the above embodiments, and will not be repeated here.

[0294] Step 702: Obtain the image quality assessment model. The image quality assessment model is trained based on at least one set of sample data. Each set of sample data includes: sample images and pre-annotated correct assessment results.

[0295] It should be noted that the training process of the image quality assessment model can be referred to the relevant details in the above embodiments, and will not be repeated here.

[0296] Step 703: Input the target image into the scene classification layer to obtain the scene classification result.

[0297] The electronic device inputs the target image to the scene classification layer and outputs the scene classification result. The scene classification result of the target image is used to uniquely indicate the scene type corresponding to the target image in the scene type set.

[0298] Optionally, the scene classification layer is used to determine the scene type corresponding to the target image based on the color temperature and photosensibility (ISO) of the target image. The color temperature can be divided into seven values, such as D75, D65, D50, TL84, A, and H, and its fitted curve can match the color temperature of autonomous driving all-weather traffic scenes. ISO sensitivity can be divided into three ranges: low ISO (below 800), medium ISO (800 to 6400), and high ISO (above 6400). Alternatively, ISO can be divided into sixteen ranges: ISO 100, ISO 200, ISO 400, ISO 800, ISO 1600, ISO 3200, ISO 6400, ISO 12800, ISO 25600, ISO 51200, ISO 102400, ISO 204800, ISO 409600, ISO 819200, ISO 1638400, and ISO 3276800. These sixteen ranges essentially cover the all-weather lighting conditions required for autonomous driving in traffic scenarios. Based on color temperature and ISO sensitivity, these ranges essentially cover the all-weather requirements for autonomous driving.

[0299] In one possible implementation, the scene type set includes five scene types: scene type 1 "noon", scene type 2 "morning and afternoon", scene type 3 "early morning and evening", scene type 4 "basement and tunnel", and scene type 5 "night". This application does not limit the method of classifying scene types.

[0300] Step 704: Input the target image and scene classification results into the objective scoring layer to obtain objective quality information, and input the target image and scene classification results into the subjective scoring layer to obtain subjective quality information.

[0301] Optionally, the objective quality information includes objective score values ​​corresponding to specified dimensions, which include at least one of brightness, color, contrast, sharpness, and noise.

[0302] The objective scoring layer is used to objectively score the white balance of the target image based on the scene classification results to obtain an objective score value in the brightness dimension, and / or objectively score the color reproduction of the target image to obtain an objective score value in the color dimension, and / or objectively score the texture details of the target image to obtain an objective score value in the contrast dimension, and / or objectively score the sharpness and signal-to-noise ratio of the target image to obtain objective score values ​​in the sharpness and noise dimensions.

[0303] Optionally, the objective scoring layer includes a deep object detection neural network, which is used to score the image quality of the target image based on the scene classification results to obtain objective quality information, thereby enabling quantitative analysis of the image quality level based on objective indicators describing the image quality dimension of the target image.

[0304] Optionally, subjective quality information includes subjective ratings for specified dimensions, which may include at least one of brightness, color, contrast, sharpness, and noise. The subjective ratings are based on subjective aesthetics. Subjective aesthetics are important factors in the aesthetic quality of tone, contrast, and image de-noising, and can be optimized by using multi-layer Laplacian filters to find aesthetically close settings for their parameters.

[0305] The subjective scoring layer is used to subjectively score the white balance of the target image based on the scene classification results to obtain a subjective score value in the brightness dimension, and / or to subjectively score the color reproduction of the target image to obtain a subjective score value in the color dimension, and / or to subjectively score the texture details of the target image to obtain a subjective score value in the contrast dimension, and / or to subjectively score the sharpness and signal-to-noise ratio of the target image to obtain subjective score values ​​in the sharpness and noise dimensions.

[0306] Optionally, the subjective rating layer includes a deep object detection neural network and a semantic segmentation neural network, which are used to obtain subjective quality information by scoring the image quality of the target image based on the scene classification results, thereby predicting the distribution of human evaluation opinions on the image from the perspective of direct perception (technical perspective) and attractiveness (aesthetic perspective).

[0307] In one possible implementation, the target image is any sample image in the training sample set. When the ISP outputs the target image, it automatically calibrates the objective indicators of the target image to obtain annotation information, so as to optimize the training sample set of the image quality assessment model based on the annotation information.

[0308] Optionally, the target image is automatically labeled to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions. The training sample set is then filtered based on the annotation information to obtain a filtered training sample set. The image quality assessment model is then trained based on the filtered training sample set to obtain an updated image quality assessment model.

[0309] Optionally, the training sample set is filtered according to the annotation information to obtain the filtered training sample set, including: when the objective annotation value corresponding to the specified dimension meets the preset rejection condition, the target image is removed from the training sample set to obtain the filtered training sample set.

[0310] Schematic illustration: The preset rejection criteria include the absolute value of the difference between the objective labeled value corresponding to a specified dimension and the pre-labeled correct objective score value being greater than a first preset threshold. The first preset threshold is either a default setting or a custom setting. This application embodiment does not limit this.

[0311] For illustrative purposes, the preset rejection criteria include an objective label value corresponding to a specified dimension that is greater than a second preset threshold. The second preset threshold can be a default setting or a custom setting. This application does not limit this specific setting.

[0312] Optionally, at least one set of candidate sample data is obtained, each set including: candidate sample images and pre-annotated correct evaluation results; at least one set of candidate sample data is added to the filtered training sample set, and the image quality assessment model is trained based on the added training sample set to obtain an updated image quality assessment model. This method can continuously increase the number of samples in the training sample set, optimize the sample quality, and enhance the stability and accuracy of the image quality assessment model.

[0313] Optionally, the annotation information includes objective annotation values ​​corresponding to specified dimensions, and the specified dimensions include at least one of brightness dimension, color dimension, contrast dimension, sharpness dimension and noise dimension.

[0314] Optionally, the specified dimension includes a brightness dimension, and the objective labeled value corresponding to the brightness dimension includes a brightness deviation value. The brightness deviation value indicates the exposure condition. The mean and variance of the target image on the grayscale image are calculated for different scene types. When there is a brightness anomaly, the mean will deviate from the mean point range, and the variance will also be smaller. The brightness deviation value of the target image is determined by calculating the mean and variance of the grayscale image.

[0315] To illustrate, the mean and variance of the target image in the grayscale are calculated. When there is a brightness anomaly, the mean will deviate from the mean point (which can be assumed to be 128), and the variance will also be smaller. By calculating the mean and variance of the grayscale image, it is determined whether the target image is overexposed or underexposed. For example, with a second preset threshold of 1, a brightness deviation value less than or equal to 1 indicates that the brightness of the target image is normal and does not need to be removed; a brightness deviation value greater than 1 indicates that the brightness of the target image is abnormal, and the target image is removed from the training sample set. As another example, different thresholds can be set for different scene types. A brightness deviation value greater than a third preset threshold indicates that the target image is too bright, and a brightness deviation value less than a fourth preset threshold indicates that the target image is too dark. The third preset threshold is greater than the fourth preset threshold.

[0316] Optionally, the specified dimension includes a color dimension, and the objective labeled value corresponding to the color dimension includes a color cast value. For illustration, the target image is an RGB image, which is transformed to the CIE Lab space, where L represents image luminance, a represents the red / green component, and b represents the yellow / blue component. Images with color casts typically have mean values ​​for components a and b that deviate significantly from the origin, and their variances are also smaller. By calculating the mean and variance of the target image for components a and b, the color cast value of the target image is determined.

[0317] Optionally, the specified dimension includes the contrast dimension, and the objective labeled value corresponding to the contrast dimension includes the gamma value.

[0318] Optionally, the specified dimensions include sharpness and noise, and the objective labeled values ​​corresponding to the sharpness and noise dimensions include Brenner gradient and peak signal-to-noise ratio.

[0319] Optionally, after obtaining the target evaluation result by calling the image quality assessment model output based on the target image, the method further includes: the electronic device adding the target image and the target evaluation result to the training sample set to obtain an updated training sample set; and training the image quality assessment model based on the updated training sample set to obtain an updated image quality assessment model.

[0320] Step 705: Adjust the ISP parameters based on objective quality information and subjective quality information.

[0321] Optionally, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise. The ISP parameters are adjusted according to the target evaluation results, including: for each specified dimension, determining the target score value corresponding to the specified dimension based on the objective score value and subjective score value corresponding to the specified dimension; adjusting the ISP parameters corresponding to the specified dimension based on the target score value corresponding to the specified dimension and a preset parameter threshold, wherein the preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification results.

[0322] Optionally, the target score for the specified dimension is determined based on the objective score, subjective score, first weight value of the objective score, and second weight value of the subjective score. The first weight value of the objective score and the second weight value of the subjective score are either default settings or custom settings, and this embodiment does not limit their specific settings.

[0323] Optionally, the training sample set is updated according to the adjusted ISP parameters, and the image quality assessment model is trained according to the updated training sample set to obtain the updated image quality assessment model.

[0324] In summary, this proposed multi-task image processing method for autonomous driving can automatically optimize ISP parameters under different scene types, improve image quality in various scenarios, and automatically adapt to all-weather scenarios for autonomous driving, thereby meeting the image quality requirements of autonomous driving perception algorithms. It also provides an automatic annotation method for objective indicators, which can automatically annotate the target image with objective values ​​in specified dimensions. Based on the annotation information, the training sample set is filtered to optimize the sample quality within the training sample set. Furthermore, the number of samples can be continuously increased to enhance the stability of the image quality assessment model and improve the accuracy of the image processing method.

[0325] Please refer to Figure 8 It illustrates a flowchart of an automatic image quality enhancement method provided in an exemplary embodiment of this application. This embodiment uses this method for... Figure 1 The following example uses an electronic device. The method includes the following steps.

[0326] Step 801: Process the acquired raw image according to the ISP parameters to obtain the target image.

[0327] Step 802: Based on the target image, call the pre-trained image quality assessment model to output the target assessment result. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the target image being assessed.

[0328] Step 803: Adjust the ISP parameters based on the target evaluation results.

[0329] It should be noted that the process by which the electronic device adjusts the ISP parameters based on the target evaluation results output by the image quality assessment model can be found in the relevant details in the above embodiments, and will not be repeated here.

[0330] Step 804: Process the acquired first image according to the adjusted ISP parameters to obtain the second image.

[0331] Optionally, the electronic device processes the first image according to the adjusted ISP parameters to obtain the second image, wherein the image quality of the second image is higher than that of the third image, the second image is the image processed based on the adjusted ISP parameters, and the third image is the image processed based on the ISP parameters before adjustment.

[0332] It should be noted that the process by which the electronic device processes the first image acquired according to the adjusted ISP parameters to obtain the second image can be compared with the process by which the original image acquired according to the ISP parameters is processed to obtain the target image. The details will not be repeated here.

[0333] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0334] Please refer to Figure 9 This diagram illustrates a block diagram of an image processing apparatus provided in an exemplary embodiment of this application. The apparatus can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The apparatus may include a processing unit 910, an output unit 920, and an adjustment unit 930.

[0335] The processing unit 910 is used to process the acquired raw image according to the ISP parameters to obtain the target image. The image quality of the target image is related to the ISP parameters.

[0336] The output unit 920 is used to call a pre-trained image quality assessment model to output the target assessment result based on the target image. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the target image being assessed.

[0337] Adjustment unit 930 is used to adjust the ISP parameters based on the target evaluation results.

[0338] In one possible implementation, the target evaluation result includes objective quality information and subjective quality information of the target image;

[0339] Among them, objective quality information indicates the quality assessment of the target image in terms of objective indicators, while subjective quality information indicates the quality assessment of the target image in terms of subjective aesthetics.

[0340] In another possible implementation, the image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer. The output unit 920 is also used for:

[0341] The target image is input into the scene classification layer and the scene classification result is output.

[0342] The target image and scene classification results are input into the objective scoring layer to obtain objective quality information, and the target image and scene classification results are input into the subjective scoring layer to obtain subjective quality information.

[0343] In another possible implementation, objective quality information includes objective rating values ​​corresponding to specified dimensions, and subjective quality information includes subjective rating values ​​corresponding to specified dimensions. The specified dimensions include at least one of brightness, color, contrast, sharpness, and noise.

[0344] In another possible implementation, the ISP parameters include at least one of brightness, color, contrast, sharpness, and noise, and the adjustment unit 930 is further used for:

[0345] For each specified dimension, determine the target score for the specified dimension based on the objective score and subjective score corresponding to the specified dimension;

[0346] Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameters corresponding to the specified dimension are adjusted. The preset parameter threshold is the parameter threshold of the specified dimension under the scene type indicated by the scene classification result.

[0347] In another possible implementation, the device further includes: an acquisition unit;

[0348] The acquisition unit is used to acquire the image quality assessment model, which is trained based on at least one set of sample data. Each set of sample data includes: sample images and pre-labeled correct assessment results.

[0349] In another possible implementation, the device further includes: a training unit; the training unit is used for:

[0350] Obtain a training sample set, which includes at least one set of sample data.

[0351] For each of at least one set of sample data, the sample image is input into the original parameter model to obtain the training result. The original parameter model is a multi-task neural network model.

[0352] The loss is calculated by comparing the training results with the correct evaluation results. The calculated loss indicates the error between the training results and the correct evaluation results.

[0353] An image quality assessment model is trained by calculating the loss corresponding to at least one set of sample data.

[0354] In another possible implementation, the device further includes: a retraining unit; the retraining unit is used for:

[0355] The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions;

[0356] The training sample set is filtered based on the annotation information to obtain the filtered training sample set;

[0357] The image quality assessment model is trained using the selected training sample set to obtain the updated image quality assessment model.

[0358] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0359] This application provides a camera, which includes an ISP and a processor; the processor is used to execute the method described above performed by the electronic device.

[0360] An embodiment of this application provides a vehicle, the vehicle including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the method described above performed by an electronic device when executing instructions.

[0361] This application provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the method described above performed by the electronic device.

[0362] This application provides a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the method described above that is executed by an electronic device.

[0363] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital video disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing.

[0364] The computer-readable program instructions or code described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0365] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from computer-readable program instructions. These electronic circuits can execute computer-readable program instructions to implement various aspects of this application.

[0366] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0367] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0368] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0369] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0370] It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented using hardware (such as circuits or ASICs (Application Specific Integrated Circuits)) that performs the corresponding function or action, or using a combination of hardware and software, such as firmware.

[0371] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0372] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized in that, The method includes: The target image is obtained by processing the acquired raw image according to the ISP parameters; Based on the target image, a pre-trained image quality assessment model is invoked to output the target assessment result. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the assessed target image. The ISP parameters are adjusted based on the target evaluation results; The image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer. The step of calling a pre-trained image quality assessment model to obtain a target assessment result based on the target image includes: inputting the target image to the scene classification layer to obtain a scene classification result; inputting the target image and the scene classification result to the objective scoring layer to obtain objective quality information; and inputting the target image and the scene classification result to the subjective scoring layer to obtain subjective quality information. The target evaluation result includes the objective quality information and the subjective quality information. The objective quality information indicates the quality evaluation of the target image on objective indicators, and the subjective quality information indicates the quality evaluation of the target image on subjective aesthetics.

2. The method according to claim 1, characterized in that, The objective quality information includes objective rating values ​​corresponding to specified dimensions, and the subjective quality information includes subjective rating values ​​corresponding to specified dimensions. The specified dimensions include at least one of brightness dimension, color dimension, contrast dimension, sharpness dimension, and noise dimension.

3. The method according to claim 2, characterized in that, The ISP parameters include at least one of brightness, color, contrast, sharpness, and noise. Adjusting the ISP parameters based on the target evaluation result includes: For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension. Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

4. The method according to any one of claims 1 to 3, characterized in that, Before obtaining the target evaluation result by calling the pre-trained image quality assessment model based on the target image, the method further includes: The image quality assessment model is obtained by training it based on at least one set of sample data, each set of sample data including: the sample image and the pre-labeled correct assessment result.

5. The method according to claim 4, characterized in that, Before obtaining the image quality assessment model, the process also includes: Obtain a training sample set, wherein the training sample set includes the at least one set of sample data groups; For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model; The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result; The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

6. The method according to claim 5, characterized in that, After obtaining the target evaluation result by calling the image quality assessment model based on the target image, the method further includes: The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions; The training sample set is filtered based on the annotation information to obtain the filtered training sample set; The image quality assessment model is trained based on the selected training sample set to obtain an updated image quality assessment model.

7. An image processing apparatus, characterized in that, The device includes: The processing unit is used to process the acquired raw image according to the ISP parameters to obtain the target image; The output unit is used to call a pre-trained image quality assessment model to output a target assessment result based on the target image. The image quality assessment model is a model trained on a neural network using sample images. The target assessment result indicates the image quality of the target image being assessed. An adjustment unit is used to adjust the ISP parameters based on the target evaluation results; The image quality assessment model includes a scene classification layer, an objective scoring layer, and a subjective scoring layer. The output unit is further configured to: input the target image to the scene classification layer to obtain a scene classification result; input the target image and the scene classification result to the objective scoring layer to obtain objective quality information; and input the target image and the scene classification result to the subjective scoring layer to obtain subjective quality information. The target evaluation result includes the objective quality information and the subjective quality information. The objective quality information indicates the quality evaluation of the target image on objective indicators, and the subjective quality information indicates the quality evaluation of the target image on subjective aesthetics.

8. The apparatus according to claim 7, characterized in that, The objective quality information includes objective rating values ​​corresponding to specified dimensions, and the subjective quality information includes subjective rating values ​​corresponding to specified dimensions. The specified dimensions include at least one of brightness dimension, color dimension, contrast dimension, sharpness dimension, and noise dimension.

9. The apparatus according to claim 8, characterized in that, The ISP parameters include at least one of brightness, color, contrast, sharpness, and noise. The adjustment unit is further configured to: For each specified dimension, a target score value corresponding to the specified dimension is determined based on the objective score value and the subjective score value corresponding to the specified dimension. Based on the target score value corresponding to the specified dimension and the preset parameter threshold, the ISP parameter corresponding to the specified dimension is adjusted. The preset parameter threshold is a preset parameter threshold for the specified dimension under the scene type indicated by the scene classification result.

10. The apparatus according to any one of claims 7 to 9, characterized in that, The device further includes: an acquisition unit; The acquisition unit is used to acquire the image quality assessment model, which is trained based on at least one set of sample data, each set of sample data including: the sample image and the pre-labeled correct assessment result.

11. The apparatus according to claim 10, characterized in that, The device further includes: a training unit; the training unit is used for: Obtain a training sample set, wherein the training sample set includes the at least one set of sample data groups; For each of the at least one set of sample data groups, the sample image is input into the original parameter model to obtain the training result, wherein the original parameter model is a multi-task neural network model; The training result is compared with the correct evaluation result to obtain a calculated loss, which indicates the error between the training result and the correct evaluation result; The image quality assessment model is trained based on the calculated loss corresponding to each of the at least one set of sample data groups.

12. The apparatus according to claim 11, characterized in that, The device further includes: a retraining unit; the retraining unit is used for: The target image is automatically calibrated to obtain annotation information, which includes objective annotation values ​​corresponding to specified dimensions; The training sample set is filtered based on the annotation information to obtain the filtered training sample set; The image quality assessment model is trained based on the selected training sample set to obtain an updated image quality assessment model.

13. A camera, characterized in that, The camera includes: an ISP and a processor; The processor is used to execute the method according to any one of claims 1-6.

14. A vehicle, characterized in that, The vehicles include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1-6 when executing the instructions.

15. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-6.

16. A computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code, characterized in that, When the computer-readable code is run in an electronic device, the processor in the electronic device performs the method according to any one of claims 1-6.

Images