Image restoration method and device for specific downstream vision task

Abstract

The application provides an image restoration method and device for a specific downstream visual task, wherein the method comprises: inputting image features of a target image into a trained agent model to obtain a parameter adjustment amount; adjusting a plurality of adjustment parameters contained in algorithms in an image restoration toolbox based on the parameter adjustment amount; processing the target image using the adjusted image restoration toolbox to obtain a restored image; obtaining an image quality score corresponding to the restored image based on an image quality evaluation model; taking the restored image as an updated target image and repeating the above steps until a set termination condition is reached; and outputting the restored image corresponding to the maximum value of the obtained plurality of image quality scores as the image restoration result of the target image. Thus, the restored image is directly optimized for the needs of the downstream visual task and can achieve better performance in the downstream visual task.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image restoration method and apparatus for specific downstream visual tasks. Background Technology

[0002] In existing image restoration methods, most of them design the optimization objective of the image restoration model to maximize the peak signal-to-noise ratio (PSNR) or maximize the structural similarity (SSIM).

[0003] However, PSNR and SSIM, as evaluation metrics, cannot reflect true image quality. Therefore, image restoration achieved by using PSNR and SSIM as optimization targets results in poor visual quality and performs poorly with low accuracy in downstream tasks. Summary of the Invention

[0004] This invention provides an image restoration method and apparatus for specific downstream visual tasks, which addresses the shortcomings of existing image restoration methods, such as poor performance and low accuracy in downstream tasks. It optimizes the restored image to meet the needs of downstream visual tasks and achieves better performance in downstream visual tasks.

[0005] This invention provides an image restoration method for a specific downstream visual task, comprising:

[0006] The image features of the target image are input into the trained intelligent agent model to obtain parameter adjustment amounts; and based on the parameter adjustment amounts, multiple adjustment parameters included in the algorithm in the image restoration toolbox are adjusted.

[0007] The target image is processed using the adjusted image restoration toolbox to obtain a restored image, and the image quality score corresponding to the restored image is obtained based on the image quality assessment model.

[0008] The restored image is used as the updated target image, and the steps of obtaining the parameter adjustment amount, adjusting the multiple adjustment parameters contained in the algorithm in the image restoration toolbox, obtaining the restored image, and obtaining the image quality score corresponding to the restored image are repeated until the set termination condition is reached. The restored image corresponding to the maximum value among the multiple obtained image quality scores is output as the image restoration result of the target image.

[0009] The intelligent agent model is trained based on a first sample image set, the image restoration toolkit, and the image quality assessment model; the image quality assessment model is trained based on a second sample image set with image quality score labels; the image quality score labels are used to reflect the performance of sample images in the second sample image set in downstream visual tasks.

[0010] According to the image restoration method for a specific downstream visual task provided by the present invention, the training method of the agent model includes:

[0011] Select target sample images from the first sample image set, perform image restoration processing on the target sample images, and obtain and store the training data corresponding to each iteration in the image restoration process; wherein, the training data corresponding to each iteration includes: the image features of the input image in the current iteration, the parameter adjustment amount output by the intelligent agent model in the current iteration, the image features of the output image in the current iteration, and the difference in image quality scores between the output image and the input image in the current iteration;

[0012] Randomly select one training data point from all currently stored training data, and update the network weights of the agent model based on the reinforcement learning algorithm;

[0013] If the number of training iterations of the agent model reaches a first threshold, a trained agent model is obtained; if the number of training iterations of the agent model does not reach the first threshold, a new target sample image is selected from the first sample image set, and the steps of performing image restoration processing on the target sample image, obtaining and storing the training data corresponding to each iteration in the image restoration processing, and updating the network weights of the agent model are repeated until the number of training iterations of the agent model reaches the first threshold, and a trained agent model is obtained.

[0014] According to the present invention, an image restoration method for a specific downstream visual task is provided, wherein the agent model includes an Actor network and a Critic network;

[0015] The Actor network is used to predict the parameter adjustment amount of multiple adjustment parameters included in the algorithm of the image restoration toolbox, and the Critic network is used to evaluate the parameter adjustment amount output by the Actor network.

[0016] According to the image restoration method for a specific downstream visual task provided by the present invention, the set termination condition includes:

[0017] The number of iterations in the image restoration process reaches the second threshold; and / or,

[0018] The parameter adjustment amount output by the intelligent agent model is less than the third threshold.

[0019] According to the present invention, an image restoration method for a specific downstream visual task is provided, wherein the image quality assessment model is a blind image quality assessment (BIQA) model.

[0020] According to the present invention, an image restoration method for a specific downstream visual task is provided, wherein the algorithm in the image restoration toolbox includes one or more of the following: image denoising algorithm, color adjustment algorithm, brightness adjustment algorithm, contrast adjustment algorithm, and edge enhancement algorithm;

[0021] Each algorithm includes at least one adjustment parameter to characterize the adjustment strength of the algorithm.

[0022] The present invention also provides an image restoration apparatus for a specific downstream visual task, comprising:

[0023] The adjustment unit is used to input the image features of the target image into the trained intelligent agent model to obtain the parameter adjustment amount; and based on the parameter adjustment amount, to adjust multiple adjustment parameters contained in the algorithm in the image restoration toolbox.

[0024] The scoring unit is used to process the target image using the adjusted image restoration toolbox to obtain a restored image, and to obtain the image quality score corresponding to the restored image based on the image quality assessment model.

[0025] The output unit is used to take the restored image as the updated target image and repeat the steps of obtaining the parameter adjustment amount, adjusting the multiple adjustment parameters included in the algorithm in the image restoration toolbox, obtaining the restored image, and obtaining the image quality score corresponding to the restored image, until a set termination condition is reached, and outputs the restored image corresponding to the maximum value among the multiple obtained image quality scores as the image restoration result of the target image.

[0026] The intelligent agent model is trained based on a first sample image set, the image restoration toolkit, and the image quality assessment model; the image quality assessment model is trained based on a second sample image set with image quality score labels; the image quality score labels are used to reflect the performance of sample images in the second sample image set in downstream visual tasks.

[0027] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image restoration method for a specific downstream visual task as described above.

[0028] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image restoration method for a specific downstream visual task as described above.

[0029] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image restoration method for a specific downstream visual task as described above.

[0030] The image restoration method and apparatus provided by this invention, designed for specific downstream visual tasks, adjusts multiple parameters of the algorithm in the image restoration toolbox based on a trained intelligent agent model. The adjusted toolbox is used to process the target image to obtain a restored image. An image quality score is obtained based on an image quality assessment model. The restored image is then used as the updated target image, and the image restoration process is repeated until a set termination condition is met. The restored image corresponding to the maximum value among the obtained image quality scores is output as the image restoration result of the target image. The image quality assessment model is trained based on a sample image set with image quality score labels. These labels reflect the performance of the sample images in the downstream visual task; a higher image quality score indicates better performance in the downstream visual task. Therefore, the restored image is directly optimized to meet the needs of the downstream visual task and can achieve better performance in that task. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0032] Figure 1 A flowchart illustrating the image restoration method for specific downstream visual tasks provided by this invention.

[0033] Figure 2 This is an overall framework diagram of the image restoration processing system provided by the present invention;

[0034] Figure 3 A flowchart of the intelligent agent model training process provided by the present invention;

[0035] Figure 4 A flowchart of the image restoration process provided by the present invention;

[0036] Figure 5This is an overall framework diagram of the image restoration system for human visual perception tasks provided by the present invention.

[0037] Figure 6 This is an overall framework diagram of the image restoration system for saliency detection tasks provided by the present invention;

[0038] Figure 7 A schematic diagram of an image restoration device for a specific downstream visual task provided by the present invention;

[0039] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0041] Most current image restoration methods ignore the application scenario of the obtained restored image and simply design the optimization objective of the image restoration model to maximize PSNR or SSIM. The restored image obtained by this method is not optimal for specific downstream vision tasks.

[0042] In practice, visual tasks can be broadly categorized into two main types: human visual perception tasks and machine vision analysis tasks. Recent research has shown that PSNR and SSIM, as evaluation metrics, cannot accurately reflect true image quality. For example, in human visual perception tasks, images with higher PSNR or SSIM do not necessarily have better visual quality. There exists a scenario where image A has higher SSIM and PSNR than image B, but image B still has better visual quality than image A. Similarly, PSNR and SSIM cannot reflect the performance of the restored image in machine vision tasks.

[0043] Therefore, incorporating the downstream visual task of the restored image into the optimization objective of the image restoration model, allowing the model to be trained directly to meet the needs of the specific downstream visual task, is more practical than simply maximizing PSNR or SSIM in existing methods. To this end, this invention provides an image restoration method and apparatus tailored to a specific downstream visual task.

[0044] Figure 1 This is a flowchart illustrating the image restoration method for a specific downstream visual task provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps:

[0045] Step 100: Input the image features of the target image into the trained agent model to obtain the parameter adjustment amount; and based on the parameter adjustment amount, adjust the multiple adjustment parameters contained in the algorithm in the image restoration toolbox.

[0046] Step 101: Use the adjusted image restoration toolbox to process the target image to obtain the restored image, and obtain the image quality score corresponding to the restored image based on the image quality assessment model.

[0047] Step 102: Use the restored image as the updated target image, and repeat the steps of obtaining parameter adjustment amounts, adjusting multiple adjustment parameters contained in the algorithm in the image restoration toolbox, obtaining the restored image, and obtaining the image quality score corresponding to the restored image, until the set termination condition is reached. Output the restored image corresponding to the maximum value among the multiple obtained image quality scores as the image restoration result of the target image.

[0048] The intelligent agent model is trained based on a first sample image set, an image restoration toolkit, and an image quality assessment model; the image quality assessment model is trained based on a second sample image set with image quality score labels; and the image quality score labels are used to reflect the performance of sample images in the second sample image set in downstream visual tasks.

[0049] Specifically, in this embodiment of the invention, the target image is the image to be restored. The image restoration toolbox may include various image processing algorithms to process the target image input to the toolbox. The algorithms in the image restoration toolbox include multiple adjustment parameters to represent the intensity of the algorithm's adjustment to the image. Optionally, before starting image restoration processing on the target image, the multiple adjustment parameters included in the algorithms in the image restoration toolbox are initialized to 0.

[0050] The intelligent agent model in this embodiment of the invention is trained based on a first sample image set, an image restoration toolbox, and an image quality assessment model. It should be noted that the image restoration toolbox and image quality assessment model used in the process of obtaining the restored image using the trained intelligent agent model should be consistent with the image restoration toolbox and image quality assessment model used in the training process of the intelligent agent model.

[0051] The image quality assessment model is trained on a second set of sample images with image quality score labels. The image quality score labels are used to reflect the performance of the sample images in the second set of sample images in downstream visual tasks. In other words, for different types of downstream visual tasks, different types of image quality assessment models can be trained on second set of sample images with different types of image quality score labels.

[0052] Optionally, the image quality assessment model in this embodiment of the invention is a blind image quality assessment (BIQA) model, which does not require a reference image and can estimate the image quality based on the image's own characteristics.

[0053] In one implementation, the downstream visual task is a human visual perception task, which can be trained to obtain an image quality assessment model oriented towards human visual perception tasks.

[0054] For example, several people score multiple sample images. Then, for each sample image, the average of the scores given by the people is used as the image quality score label for that sample image. These sample images with image quality score labels are used to train the image quality assessment model, thus obtaining an image quality assessment model for human visual perception tasks.

[0055] In one implementation, the downstream vision task is a machine vision analysis task, which can train an image quality assessment model for machine vision analysis tasks.

[0056] Taking saliency detection as an example, a saliency detection dataset can be selected. Different levels and types of distortion factors, such as Gaussian noise, Gaussian blur, color bias, brightness bias, and contrast bias, are randomly added to each image in the dataset to generate damaged images. These damaged images and the original images are then used as the sample image set for training an image quality assessment model. Each image in this sample image set is input into the saliency detector. The F-score is calculated based on the detection result of the input image and the corresponding labeled sample image (ground truth), serving as the image quality score label for that input image. The image quality assessment model is trained using these labeled sample images, thus obtaining an image quality assessment model for the saliency detection task.

[0057] Therefore, it is understandable that by training an image quality assessment model for a specific downstream task, and then training an agent model based on the image quality assessment model, the image restoration toolkit, and the first sample image set, the resulting agent model is also for a specific downstream task and is an agent model used in conjunction with the image quality assessment model.

[0058] Figure 2 This is an overall framework diagram of the image restoration processing system provided by the present invention, as shown below. Figure 2 As shown, when performing image restoration processing on a target image using the method provided by this invention, the image features of the target image are first input into a trained intelligent agent model, and the intelligent agent model can output the parameter adjustment amount of the image restoration toolbox.

[0059] It should be noted that the parameter adjustment amount in the embodiments of the present invention may include the adjustment amount of each adjustment parameter included in the algorithm in the image restoration toolbox.

[0060] Optionally, the algorithms in the image restoration toolbox include one or more of the following: image denoising algorithms, color adjustment algorithms, brightness adjustment algorithms, contrast adjustment algorithms, and edge enhancement algorithms;

[0061] Each algorithm includes at least one adjustment parameter to characterize the adjustment strength of the algorithm.

[0062] Specifically, the algorithms in the image restoration toolbox may include one or more of the following: image denoising algorithm, color adjustment algorithm, brightness adjustment algorithm, contrast adjustment algorithm, and edge enhancement algorithm; each algorithm may include at least one adjustment parameter, which represents the intensity of image denoising, color enhancement, brightness enhancement, contrast enhancement, and edge enhancement, respectively.

[0063] Then, the multiple adjustment parameters included in the algorithm of the image restoration toolbox can be adjusted according to the parameter adjustment amount output by the agent model. For example, if the algorithm in the image restoration toolbox contains 5 adjustment parameters, all adjustable within the range of 0 to 100, and each adjustment parameter is initialized to (0, 0, 0, 0, 0), and the parameter adjustment amount output by the agent model is (10, 20, 30, 40, 50), then the adjusted adjustment parameters included in the algorithm of the image restoration toolbox will be (10, 20, 30, 40, 50).

[0064] After adjusting the multiple adjustment parameters included in the algorithm of the image restoration toolbox based on the parameter adjustment amount output by the intelligent agent model, the adjusted image restoration toolbox can be used to process the target image to obtain the restored image.

[0065] After obtaining the restored image, it can be input into the image quality assessment model to obtain the image quality score corresponding to the restored image.

[0066] After obtaining the image quality score corresponding to the restored image, the restored image and its corresponding image quality score can be stored. The restored image from this restoration process is then used as the target image for the next restoration process, i.e., the updated target image. This process repeats the steps of obtaining parameter adjustments, adjusting multiple adjustment parameters included in the algorithm within the image restoration toolbox, obtaining the restored image, and acquiring the corresponding image quality score, until a set termination condition is met. Once the termination condition is met, the restored image corresponding to the maximum value among the acquired image quality scores is output as the image restoration result for the target image.

[0067] The image restoration method for specific downstream visual tasks provided by this invention adjusts multiple parameters of the algorithm in the image restoration toolbox based on a trained intelligent agent model; the adjusted image restoration toolbox is used to process the target image to obtain a restored image; an image quality score corresponding to the restored image is obtained based on an image quality assessment model; the restored image is used as the updated target image; the image restoration process is repeated until a set termination condition is reached; the restored image corresponding to the maximum value among the obtained multiple image quality scores is output as the image restoration result of the target image. The image quality assessment model is trained based on a sample image set with image quality score labels. The image quality score labels are used to reflect the performance of the sample images in the sample image set in the downstream visual task. The higher the image quality score, the better the restored image performs in the downstream visual task. Thus, the restored image is directly optimized for the needs of the downstream visual task and can achieve better performance in the downstream visual task.

[0068] Optionally, the training methods for the agent model include:

[0069] Select target sample images from the first sample image set, perform image restoration processing on the target sample images, and obtain and store the training data corresponding to each iteration in the image restoration process; wherein, the training data corresponding to each iteration includes: the image features of the input image in the current iteration, the parameter adjustment amount output by the agent model in the current iteration, the image features of the output image in the current iteration, and the difference in image quality scores between the output image and the input image in the current iteration.

[0070] Randomly select one training data point from all currently stored training data, and update the network weights of the agent model based on the reinforcement learning algorithm;

[0071] If the number of training iterations of the agent model reaches the first threshold, a trained agent model is obtained. If the number of training iterations of the agent model does not reach the first threshold, a new target sample image is selected from the first sample image set, and the image restoration processing of the target sample image is repeated. The training data corresponding to each iteration in the image restoration process is obtained and stored, and the network weights of the agent model are updated, until the number of training iterations of the agent model reaches the first threshold, and a trained agent model is obtained.

[0072] Specifically, during the training of the agent model, a sample image can be randomly selected from the first sample image set as the target sample image, and the target sample image can be processed for image restoration. The training data corresponding to each iteration in the image restoration process can be obtained and stored.

[0073] The image restoration process for the target sample image includes: using the target sample image as the input image (i.e., the input image in the first iteration), inputting the image features of the input image into the agent model to obtain parameter adjustment amounts, and adjusting multiple adjustment parameters included in the algorithm in the image restoration toolbox based on the parameter adjustment amounts. The adjusted image restoration toolbox is then used to process the input image to obtain the output image (i.e., the output image in the first iteration). The image quality scores of the input and output images are obtained respectively, and the image quality score of the output image is subtracted from the image quality score of the input image. The training data corresponding to the current iteration (image features of the input image, parameter adjustment amounts output by the agent model, image features of the output image, and the difference between the image quality scores of the output and input images) is stored. The output image from the first iteration is used as the input image for the second iteration, and the above image restoration process is repeated until the set termination condition is met, at which point the image restoration process for the target sample image ends.

[0074] After the image restoration process of a target sample image is completed, all the currently stored training data includes the training data corresponding to each iteration in the image restoration process of the currently selected target sample image and the training data corresponding to each iteration in the image restoration process of each previously selected target sample image.

[0075] For example, the current target sample image is selected for the 3rd time. The training data corresponding to each iteration in the image restoration process of the target sample image selected this time includes C1, C2, and C3. The training data corresponding to each iteration in the image restoration process of the previous two target sample images includes A1, A2, and A3 and B1, B2, and B3, respectively. Then, all the training data currently stored are A1, A2, A3, B1, B2, B3, C1, C2, and C3.

[0076] Randomly select one training data point from all currently stored training data. In each training data point, the image features of the input image represent the current state, the parameter adjustment amount output by the agent model represents the action taken by the agent, the image features of the output image represent the next state predicted by the agent model, and the difference in image quality scores between the output image and the input image represents the reward value after the agent takes that action.

[0077] Then, based on reinforcement learning algorithms, such as Deep Q Network (DQN), State Action Reward State Action (SARSA), and Proximal Policy Optimization (PPO), the network weights of the agent model can be updated according to this training data.

[0078] Optionally, the agent model includes an actor network and a critic network;

[0079] The Actor network is used to predict the parameter adjustments of multiple adjustment parameters included in the algorithm of the image restoration toolbox, while the Critic network is used to evaluate the parameter adjustments output by the Actor network.

[0080] Specifically, the structure of the agent model can include two neural networks: an Actor network and a Critic network. The Actor network is used to take image features as input and predict the parameter adjustments (actions taken by the agent) of multiple adjustment parameters included in the algorithm in the image restoration toolbox. The Critic network is used to evaluate the parameter adjustments output by the Actor network based on the current state, the next state, and the reward value, and then update the network weights of the agent model through the backpropagation algorithm.

[0081] In one implementation, both the Actor network and the Critic network can be designed as neural networks consisting of four fully connected layers. The dimensions of the four fully connected layers of the Actor network are 1024, 512, 128, and n (where n is the number of adjustment parameters included in the algorithm in the image restoration toolbox); the dimensions of the four fully connected layers of the Critic network are 1024, 512, 128, and 1.

[0082] After updating the network weights of the agent model, it is determined whether the number of training iterations of the agent model has reached a first threshold. The first threshold can be a pre-set number of iterations.

[0083] Training ends when the number of training iterations of the agent model reaches the first threshold, resulting in a well-trained agent model.

[0084] If the number of training iterations of the agent model has not reached the first threshold, a new target sample image is selected from the first sample image set, and the image restoration processing of the target sample image is repeated. The training data corresponding to each iteration in the image restoration process is obtained and stored, and the network weights of the agent model are updated, until the number of training iterations of the agent model reaches the first threshold, and the trained agent model is obtained.

[0085] Optionally, the termination conditions may include:

[0086] The number of iterations in the image restoration process reaches the second threshold; and / or,

[0087] The parameter adjustment amount output by the agent model is less than the third threshold.

[0088] Specifically, the termination condition can be that the number of iterations in the entire image restoration process reaches a preset second threshold. For example, when the number of iterations reaches 4, that is, when four restored images and their corresponding quality scores are obtained, the image restoration process will no longer proceed. The termination condition can also be that the parameter adjustment amount output by the agent model is less than a third threshold. For example, if the third threshold is set to 5, in the current image restoration process, after inputting the features of the target image into the agent model, if the parameter adjustment amounts of multiple adjustment parameters output by the agent model are all less than 5, then the current image restoration process will be completed and the image restoration process will no longer proceed. The termination condition can also be that the number of iterations in the entire image restoration process reaches a preset second threshold, and the parameter adjustment amount output by the agent model is less than the third threshold.

[0089] The following examples, through specific application scenarios, further illustrate the image restoration method provided by the present invention for specific downstream visual tasks.

[0090] Example 1:

[0091] Figure 3 The flowchart of the intelligent agent model training process provided by the present invention is as follows: Figure 3 As shown, the training process for the agent model is as follows:

[0092] Step 1: Given a training set D containing several images to be restored, a given image quality assessment model H, an image restoration toolbox T, and an image feature extractor F, set an upper limit L for the number of training iterations of the agent model and a termination condition C for the image restoration iteration, and initialize the buffer. The number of training iterations for the agent model is k = 1.

[0093] Step 2: Initialize the parameters p of each image processing algorithm in the given image restoration toolbox T, randomly sample an image I from the training set D, input the current image I into the given image feature extractor F, and output the image features s.

[0094] Step 3: Input the current image I into the given image quality assessment model H to obtain the image quality score q.

[0095] Step 4: Input the image features s into the intelligent agent model, output the adjustment amount a of the parameters, and update the parameters p' = p + a of each image processing algorithm in the given image restoration toolbox T.

[0096] Step 5: Use the given image restoration toolbox T to process the current image I to obtain the restored image I'.

[0097] Step 6: Input the restored image I' into the given image feature extractor F, and output the features s' of the restored image.

[0098] Step 7: Input the restored image I' into the given image quality assessment model H, and output the image quality score q'.

[0099] Step 8: Calculate the reward value r = q' - q.

[0100] Step 9: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.]<a,s,r,s’> The data is stored in the given buffer B.

[0101] Step 10: Determine whether the given image restoration iteration termination condition C is met. If not, proceed to step 11; if yes, proceed to step 12.

[0102] Step 11: Update the current image I = I', the current parameter p = p', the image feature s = s', and return to step 4.

[0103] Step 12: Sample one data point in the given buffer B.<a,s,r,s’> The network weights of the agent model are updated using reinforcement learning algorithms, such as DQN, SARSA, and PPO.

[0104] Step 13: Determine whether the number of model training iterations k has reached the preset upper limit L. If yes, the agent model training process ends. If no, update the number of agent model training iterations k = k + 1 and return to step 2.

[0105] Example 2:

[0106] Figure 4 The flowchart of the image restoration process provided by the present invention is as follows: Figure 4 As shown, the training process for the agent model is as follows:

[0107] Step 1: Given a trained agent model, an image I to be restored, an image quality assessment model H, an image restoration toolbox T, an image feature extractor F, a termination condition C for the image restoration iteration, and initialize the buffer. Initialize the parameters p of each image processing algorithm in the given image restoration toolbox T.

[0108] Step 2: Input image I into the given image feature extractor F, and output the features s of the image.

[0109] Step 3: Input the current image I into the given image quality assessment model H to obtain the image quality score q.

[0110] Step 4: Input the image state s into the given agent model, output the adjustment amount a of the output parameters, and update the parameters p' = p + a of each image processing algorithm in the given image restoration toolbox T.

[0111] Step 5: Use the given image restoration toolbox T to process image I and obtain the restored image I'.

[0112] Step 6: Input the restored image I' into the given image quality assessment model H, and output the image quality score q'.

[0113] Step 7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full<I’,q’> The data is stored in the given buffer B.

[0114] Step 8: Determine whether the given image restoration iteration termination condition C is met. If not, proceed to step 9; if yes, proceed to step 10.

[0115] Step 9: Update the current input image I = I', the current parameter p = p', and return to step 2.

[0116] Step 10: The restoration process ends. Select the restored image with the highest image quality score from the given buffer B as the final output.

[0117] Example 3: Downstream visual application is photography.

[0118] Due to factors such as internal camera equipment issues, camera shake, ambient lighting, shadows, and weather conditions, photos may suffer from problems such as noise, blur, color deviation, excessive brightness or darkness, and insufficient or excessive contrast. In such cases, photo restoration processing is necessary to achieve a visually pleasing image.

[0119] Figure 5 This is an overall framework diagram of the image restoration system for human visual perception tasks provided by the present invention. Figure 5As shown, this downstream visual task is geared towards human visual perception. A BIQA model trained on an image quality assessment dataset can be selected to evaluate the quality of the restored image. The image quality assessment dataset consists of a series of images with varying degrees of impairment, each with a corresponding image quality score, calculated by averaging scores from several people. Therefore, a BIQA model trained on this dataset can reflect human visual preferences to some extent. The Semantic Part-aware Query (SPAQ) dataset is one of the largest image quality assessment datasets currently available, and the Hyper-Resolution Image Quality Assessment (HyperIQA) model is one of the most advanced BIQA models. A HyperIQA model trained on the SPAQ dataset is chosen as the image quality assessment model H.

[0120] The image restoration toolbox contains five image processing algorithms: image denoising, color adjustment, brightness adjustment, contrast adjustment, and edge enhancement. Each algorithm includes an adjustable parameter representing the intensity of image denoising, color enhancement, brightness enhancement, contrast enhancement, and edge enhancement, respectively.

[0121] The agent model is implemented using a reinforcement learning algorithm. The most popular PPO algorithm is chosen, which consists of two networks: an Actor network and a Critic network. The Actor network predicts the adjustment amounts of parameters for various image processing algorithms in the image restoration toolbox, while the Critic network evaluates the quality of the Actor network's output. Both the Actor and Critic networks are designed as neural networks with four fully connected layers. The dimensions of the four fully connected layers in the Actor network are 1024, 512, 128, and 5; the dimensions of the four fully connected layers in the Critic network are 1024, 512, 128, and 1.

[0122] The steps for training an agent model can be summarized as follows:

[0123] Step 1: Given a training set D, such as the SPAQ dataset, containing 6675 distorted images of varying degrees and types, a HyperIQA model trained on SPAQ as the image quality assessment model H, an image restoration toolkit T containing five image processing algorithms: image denoising, color adjustment, brightness adjustment, contrast adjustment, and edge enhancement, and a ResNet50 pre-trained on ImageNet as the image feature extractor F. The feature map output from the conv5_x layer is used as the image features. The upper limit for the number of training iterations of the agent model is set to L = 50000. The termination condition C for the image restoration iteration is set to reach the maximum number of iterations M = 4. Initialize the buffer. The number of training iterations for the agent model is k = 1.

[0124] Step 2: Initialize the parameters p = [0,0,0,0,0] of each image processing algorithm in the given image restoration toolbox T. Each element in p corresponds to the image denoising intensity, color enhancement intensity, brightness enhancement intensity, contrast enhancement intensity, and edge enhancement intensity, respectively. Randomly sample an image I from the training set D, input image I into the given image feature extractor F, and output image features s.

[0125] Step 3: Input the current image I into the given image quality assessment model H, and output the image quality score q.

[0126] Step 4: Input the image features s into the Actor network of the intelligent agent model, output the adjustment amount a of the parameters, and update the parameters p' = p + a of each image processing algorithm in the given image restoration toolbox T.

[0127] Step 5: Use the given image restoration toolbox T to process the current image I to obtain the restored image I'.

[0128] Step 6: Input the restored image I' into the given feature extractor F, and output the image features s'.

[0129] Step 7: Input the restored image I' into the given image quality assessment model H, and output the image quality score q'.

[0130] Step 8: Calculate the reward value r = q' - q.

[0131] Step 9: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.]<a,s,r,s’> The data is stored in the given buffer B.

[0132] Step 10: Determine whether the given image restoration iteration termination condition C is met. If not, proceed to step 11; if yes, proceed to step 12.

[0133] Step 11: Update the current image I = I', the current parameter p = p', the image feature s = s', and return to step 4.

[0134] Step 12: Sample one data point in the given buffer B.<a,s,r,s’> The PPO reinforcement learning algorithm is used to update the network weights of the agent.

[0135] Step 13: Update the number of training iterations of the agent model k = k + 1, and determine whether the number of training iterations of the agent model k has reached the preset upper limit L. If yes, the agent model training process ends; otherwise, return to step 2.

[0136] The restoration process of an image to be restored using the trained agent model described above can be summarized in the following steps:

[0137] Step 1: Given a trained agent model, an image I to be restored, an image quality assessment model H, and an image restoration toolbox T, set the termination condition C for the image restoration iteration as: reaching the maximum number of iterations M = 4, and initialize the buffer. Initialize the parameters p = [0,0,0,0,0] of each image processing algorithm in the given image restoration toolbox T. Each element in p corresponds to the image denoising intensity, color enhancement intensity, brightness enhancement intensity, contrast enhancement intensity, and edge enhancement intensity, respectively.

[0138] Step 2: Input the current image I into the given image feature extractor F, and output the image features s.

[0139] Step 3: Input the current image I into the given image quality assessment model H to obtain the image quality score q.

[0140] Step 4: Input the image features s into the Actor network of the given agent model, output the parameter adjustment amount a, and update the parameters p' = p + a of each image processing algorithm in the given image restoration toolbox T.

[0141] Step 5: Use the given image restoration toolbox T to process the current image I to obtain the restored image I'.

[0142] Step 6: Input the restored image I' into the given image quality assessment model H, and output the image quality score q'.

[0143] Step 7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full<I’,q’> The data is stored in the given buffer B.

[0144] Step 8: Determine whether the given image restoration iteration termination condition C is met. If not, proceed to step 9; if yes, proceed to step 10.

[0145] Step 9: Update the current input image I = I', the current parameter p = p', and return to step 2.

[0146] Step 10: The restoration process is complete. Select the restored image with the highest image quality score from the given buffer B as the final output.

[0147] Example 4: Downstream visual application is saliency detection.

[0148] In scenarios such as autonomous driving and video surveillance, images captured by cameras may contain issues such as noise, blurriness, color deviation, excessive brightness or darkness, and insufficient or excessive contrast, which will affect subsequent tasks such as saliency detection, object detection, and image segmentation. Therefore, image restoration processing is necessary to improve the accuracy of downstream machine vision analysis tasks.

[0149] Figure 6 This is an overall framework diagram of the image restoration system for saliency detection tasks provided by the present invention. Figure 6 As shown, for the saliency detection task, a trained saliency detector can be used to detect saliency in the restored image. The detected results are then compared with labeled sample images (ground truth) to calculate the F-score, which serves as an evaluation of the quality of the restored image. However, considering that ground truth does not exist in real-world applications, the F-score cannot be obtained. Therefore, we can consider using a neural network to automatically estimate the F-score of an image.

[0150] The PASCAL-S dataset is a commonly used saliency detection dataset, containing 850 high-resolution images, each with a corresponding ground truth saliency map. PoolNet is one of the most advanced saliency detection algorithms available, and a PoolNet model trained on the PASCAL-S dataset was chosen as the saliency detector. Simultaneously, different levels and types of distortion factors, including Gaussian noise, Gaussian blur, color bias, brightness bias, and contrast bias, were randomly added to each image in the PASCAL-S dataset to generate damaged images, which served as the training dataset D for the agent model.

[0151] The image restoration toolbox contains five image processing algorithms: image denoising, color adjustment, brightness adjustment, contrast adjustment, and edge enhancement. Each algorithm includes an adjustable parameter representing the intensity of image denoising, color enhancement, brightness enhancement, contrast enhancement, and edge enhancement, respectively.

[0152] The agent model is implemented using a reinforcement learning algorithm, specifically the widely used PPO algorithm, which comprises two networks: an Actor network and a Critic network. The Actor network predicts the adjustment amounts of various image processing algorithm parameters within the image restoration toolkit, while the Critic network evaluates the quality of the Actor network's output. Both the Actor and Critic networks are designed as neural networks consisting of four fully connected layers. The dimensions of the four fully connected layers in the Actor network are 1024, 512, 128, and 5, respectively; the dimensions of the four fully connected layers in the Critic network are 1024, 512, 128, and 1, respectively.

[0153] The steps for training an agent model can be summarized as follows:

[0154] Step 0: Train a BIQA model for saliency detection: Select a commonly used saliency detection dataset, such as PASCAL-S. Randomly add different levels and types of distortion factors to each image in the dataset, including Gaussian noise, Gaussian blur, color bias, brightness bias, and contrast bias, generating damaged images. Combine these damaged images with the original images in the dataset to form a new dataset. Input each image in this dataset into a pre-trained saliency detector, such as the PoolNet model. Calculate the F-score of each image's detection result with its ground truth, using this as the image quality score. Based on the images and their corresponding quality scores, train a BIQA model using existing methods, such as the HyperNet model.

[0155] Step 1: Given a training set D, a BIQA model trained in Step 0 for saliency detection, and an image restoration toolkit T containing five image processing algorithms: image denoising, color adjustment, brightness adjustment, contrast adjustment, and edge enhancement, a ResNet50 pre-trained on ImageNet is used as the image feature extractor F. The feature map output from the conv5_x layer is used as the image features. The upper limit for the number of model training iterations is set to L = 50000. The termination condition for the image restoration iteration is set to reach the maximum number of iterations M = 4. The buffer is initialized. The number of training iterations for the agent model is k = 1.

[0156] Step 2: Initialize the parameters p = [0,0,0,0,0] of each image processing algorithm in the given image restoration toolbox T. Each element in p corresponds to the image denoising intensity, color enhancement intensity, brightness enhancement intensity, contrast enhancement intensity, and edge enhancement intensity, respectively. Randomly sample an image I and its corresponding ground truth saliency map M from the training set D. Input the image I into the given image feature extractor F and output the image features s.

[0157] Step 3: Input the current image I into the given BIQA model and output the predicted F-score as the image quality score q of image I.

[0158] Step 4: Input the image features s into the Actor network of the intelligent agent model, output the adjustment amount a of the parameters, and update the parameters p' = p + a of each image processing algorithm in the given image restoration toolbox T.

[0159] Step 5: Use the given image restoration toolbox T to process the current image I to obtain the restored image I'.

[0160] Step 6: Input the restored image I' into the given feature extractor F, and output the image features s'.

[0161] Step 7: Input the restored image I' into the given BIQA model and output the predicted F-score as the image quality score q' of the restored image I'.

[0162] Step 8: Calculate the reward value r = q' - q.

[0163] Step 9: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.]<a,s,r,s’> The data is stored in the given buffer B.

[0164] Step 10: Determine whether the given image restoration iteration termination condition C is met. If not, proceed to step 11; if yes, proceed to step 12.

[0165] Step 11: Update the current image I = I', the current parameter p = p', the image feature s = s', and return to step 4.

[0166] Step 12: Sample one data point in the given buffer B.<a,s,r,s’> The PPO reinforcement learning algorithm is used to update the network weights of the agent.

[0167] Step 13: Update the number of training iterations of the agent model k = k + 1, and determine whether the number of training iterations of the agent model k has reached the preset upper limit L. If yes, the agent model training process ends; otherwise, return to step 2.

[0168] The restoration process of an image to be restored using the trained agent model described above can be summarized in the following steps:

[0169] Step 1: Given a trained agent model, an image I to be restored, its corresponding ground truth saliency map M, a saliency detector, and an image restoration toolbox T, set the termination condition C for the image restoration iteration to: reaching the maximum number of iterations M = 4, and initialize the buffer. Initialize the parameters p = [0,0,0,0,0] of each image processing algorithm in the given image restoration toolbox T. Each element in p corresponds to the image denoising intensity, color enhancement intensity, brightness enhancement intensity, contrast enhancement intensity, and edge enhancement intensity, respectively.

[0170] Step 2: Input the current image I into the given image feature extractor F, and output the image features s.

[0171] Step 3: Input the current image I into the given BIQA model and output the predicted F-score as the image quality score q of image I.

[0172] Step 4: Input the image features s into the Actor network of the given agent model, output the parameter adjustment amount a, and update the parameters p' = p + a of each image processing algorithm in the given image restoration toolbox T.

[0173] Step 5: Use the given image restoration toolbox T to process the current image I to obtain the restored image I'.

[0174] Step 6: Input the restored image I' into the given BIQA model and output the predicted F-score as the image quality score q' of the restored image I'.

[0175] Step 7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full<I’,q’> The data is stored in the given buffer B.

[0176] Step 8: Determine whether the given image restoration iteration termination condition C is met. If not, proceed to step 9; if yes, proceed to step 10.

[0177] Step 9: Update the current input image I = I', the current parameter p = p', and return to step 2.

[0178] Step 10: The restoration process ends. Select the restored image with the highest image quality score from the given buffer B as the final output.

[0179] The image restoration apparatus for a specific downstream visual task provided by the present invention is described below. The image restoration apparatus for a specific downstream visual task described below can be referred to in correspondence with the image restoration method for a specific downstream visual task described above.

[0180] Figure 7 This is a schematic diagram of the image restoration device for a specific downstream visual task provided by the present invention, as shown below. Figure 7 As shown, the device includes:

[0181] The adjustment unit 700 is used to input the image features of the target image into the trained intelligent agent model to obtain the parameter adjustment amount; and based on the parameter adjustment amount, to adjust multiple adjustment parameters contained in the algorithm in the image restoration toolbox.

[0182] The scoring unit 710 is used to process the target image using the adjusted image restoration toolbox to obtain the restored image, and to obtain the image quality score corresponding to the restored image based on the image quality assessment model.

[0183] The output unit 720 is used to take the restored image as the updated target image and repeat the steps of obtaining parameter adjustment amount, adjusting multiple adjustment parameters contained in the algorithm in the image restoration toolbox, obtaining the restored image and obtaining the image quality score corresponding to the restored image, until the set termination condition is reached, and output the restored image corresponding to the maximum value among the multiple obtained image quality scores as the image restoration result of the target image.

[0184] The intelligent agent model is trained based on a first sample image set, an image restoration toolkit, and an image quality assessment model; the image quality assessment model is trained based on a second sample image set with image quality score labels; and the image quality score labels are used to reflect the performance of sample images in the second sample image set in downstream visual tasks.

[0185] Optionally, the training methods for the agent model include:

[0186] Select target sample images from the first sample image set, perform image restoration processing on the target sample images, and obtain and store the training data corresponding to each iteration in the image restoration process; wherein, the training data corresponding to each iteration includes: the image features of the input image in the current iteration, the parameter adjustment amount output by the agent model in the current iteration, the image features of the output image in the current iteration, and the difference in image quality scores between the output image and the input image in the current iteration.

[0187] Randomly select one training data point from all currently stored training data, and update the network weights of the agent model based on the reinforcement learning algorithm;

[0188] If the number of training iterations of the agent model reaches the first threshold, a trained agent model is obtained. If the number of training iterations of the agent model does not reach the first threshold, a new target sample image is selected from the first sample image set, and the image restoration processing of the target sample image is repeated. The training data corresponding to each iteration in the image restoration process is obtained and stored, and the network weights of the agent model are updated, until the number of training iterations of the agent model reaches the first threshold, and a trained agent model is obtained.

[0189] Optionally, the agent model includes an Actor network and a Critic network;

[0190] The Actor network is used to predict the parameter adjustments of multiple adjustment parameters included in the algorithm of the image restoration toolbox, while the Critic network is used to evaluate the parameter adjustments output by the Actor network.

[0191] Optionally, the termination conditions may include:

[0192] The number of iterations in the image restoration process reaches the second threshold; and / or,

[0193] The parameter adjustment amount output by the agent model is less than the third threshold.

[0194] Optionally, the image quality assessment model is a blind image quality assessment (BIQA) model.

[0195] Optionally, the algorithms in the image restoration toolbox include one or more of the following: image denoising algorithms, color adjustment algorithms, brightness adjustment algorithms, contrast adjustment algorithms, and edge enhancement algorithms;

[0196] Each algorithm includes at least one adjustment parameter to characterize the adjustment strength of the algorithm.

[0197] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute an image restoration method for a specific downstream visual task, the method including:

[0198] The image features of the target image are input into the trained intelligent agent model to obtain parameter adjustment amounts; and based on the parameter adjustment amounts, multiple adjustment parameters contained in the algorithm in the image restoration toolbox are adjusted.

[0199] The target image is processed using the adjusted image restoration toolbox to obtain the restored image, and the image quality score corresponding to the restored image is obtained based on the image quality assessment model.

[0200] The restored image is used as the updated target image, and the steps of obtaining parameter adjustment amounts, adjusting multiple adjustment parameters contained in the algorithm in the image restoration toolbox, obtaining the restored image, and obtaining the image quality score corresponding to the restored image are repeated until the set termination condition is reached. The restored image corresponding to the maximum value among the multiple obtained image quality scores is output as the image restoration result of the target image.

[0201] The intelligent agent model is trained based on a first sample image set, an image restoration toolkit, and an image quality assessment model; the image quality assessment model is trained based on a second sample image set with image quality score labels; and the image quality score labels are used to reflect the performance of sample images in the second sample image set in downstream visual tasks.

[0202] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0203] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the image restoration method for a specific downstream visual task provided by the above methods, the method comprising:

[0204] The image features of the target image are input into the trained intelligent agent model to obtain parameter adjustment amounts; and based on the parameter adjustment amounts, multiple adjustment parameters contained in the algorithm in the image restoration toolbox are adjusted.

[0205] The target image is processed using the adjusted image restoration toolbox to obtain the restored image, and the image quality score corresponding to the restored image is obtained based on the image quality assessment model.

[0206] The restored image is used as the updated target image, and the steps of obtaining parameter adjustment amounts, adjusting multiple adjustment parameters contained in the algorithm in the image restoration toolbox, obtaining the restored image, and obtaining the image quality score corresponding to the restored image are repeated until the set termination condition is reached. The restored image corresponding to the maximum value among the multiple obtained image quality scores is output as the image restoration result of the target image.

[0207] The intelligent agent model is trained based on a first sample image set, an image restoration toolkit, and an image quality assessment model; the image quality assessment model is trained based on a second sample image set with image quality score labels; and the image quality score labels are used to reflect the performance of sample images in the second sample image set in downstream visual tasks.

[0208] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an image restoration method for a specific downstream visual task provided by the methods described above, the method comprising:

[0209] The image features of the target image are input into the trained intelligent agent model to obtain parameter adjustment amounts; and based on the parameter adjustment amounts, multiple adjustment parameters contained in the algorithm in the image restoration toolbox are adjusted.

[0210] The target image is processed using the adjusted image restoration toolbox to obtain the restored image, and the image quality score corresponding to the restored image is obtained based on the image quality assessment model.

[0211] The restored image is used as the updated target image, and the steps of obtaining parameter adjustment amounts, adjusting multiple adjustment parameters contained in the algorithm in the image restoration toolbox, obtaining the restored image, and obtaining the image quality score corresponding to the restored image are repeated until the set termination condition is reached. The restored image corresponding to the maximum value among the multiple obtained image quality scores is output as the image restoration result of the target image.

[0212] The intelligent agent model is trained based on a first sample image set, an image restoration toolkit, and an image quality assessment model; the image quality assessment model is trained based on a second sample image set with image quality score labels; and the image quality score labels are used to reflect the performance of sample images in the second sample image set in downstream visual tasks.

[0213] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0214] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0215] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image restoration method for a specific downstream visual task, characterized in that, include: The image features of the target image are input into the trained agent model to obtain the parameter adjustment amount; Based on the parameter adjustment amount, the multiple adjustment parameters included in the algorithm in the image restoration toolbox are adjusted; The target image is processed using the adjusted image restoration toolbox to obtain a restored image, and the image quality score corresponding to the restored image is obtained based on the image quality assessment model. The restored image is used as the updated target image, and the steps of obtaining the parameter adjustment amount, adjusting the multiple adjustment parameters contained in the algorithm in the image restoration toolbox, obtaining the restored image, and obtaining the image quality score corresponding to the restored image are repeated until the set termination condition is reached. The restored image corresponding to the maximum value among the multiple obtained image quality scores is output as the image restoration result of the target image. The intelligent agent model is trained based on a first sample image set, the image restoration toolkit, and the image quality assessment model; the image quality assessment model is trained based on a second sample image set with image quality score labels; the image quality score labels are used to reflect the performance of sample images in the second sample image set in downstream visual tasks. The training method for the agent model includes: Select target sample images from the first sample image set, perform image restoration processing on the target sample images, and obtain and store the training data corresponding to each iteration in the image restoration process; wherein, the training data corresponding to each iteration includes: the image features of the input image in the current iteration, the parameter adjustment amount output by the intelligent agent model in the current iteration, the image features of the output image in the current iteration, and the difference in image quality scores between the output image and the input image in the current iteration; Randomly select one training data point from all currently stored training data, and update the network weights of the agent model based on the reinforcement learning algorithm; If the number of training iterations of the agent model reaches a first threshold, a trained agent model is obtained; if the number of training iterations of the agent model does not reach the first threshold, a new target sample image is selected from the first sample image set, and the steps of performing image restoration processing on the target sample image, obtaining and storing the training data corresponding to each iteration in the image restoration processing, and updating the network weights of the agent model are repeated until the number of training iterations of the agent model reaches the first threshold, and a trained agent model is obtained.

2. The image restoration method for a specific downstream visual task according to claim 1, characterized in that, The agent model includes an Actor network and a Critic network; The Actor network is used to predict the parameter adjustment amount of multiple adjustment parameters included in the algorithm of the image restoration toolbox, and the Critic network is used to evaluate the parameter adjustment amount output by the Actor network.

3. The image restoration method for a specific downstream visual task according to claim 1, characterized in that, The specified termination conditions include: The number of iterations in the image restoration process reaches the second threshold; and / or, The parameter adjustment amount output by the intelligent agent model is less than the third threshold.

4. The image restoration method for a specific downstream visual task according to any one of claims 1 to 3, characterized in that, The image quality assessment model is the Blind Image Quality Assessment (BIQA) model.

5. The image restoration method for a specific downstream visual task according to any one of claims 1 to 3, characterized in that, The algorithms in the image restoration toolbox include one or more of the following: image denoising algorithm, color adjustment algorithm, brightness adjustment algorithm, contrast adjustment algorithm, and edge enhancement algorithm. Each algorithm includes at least one adjustment parameter to characterize the adjustment strength of the algorithm.

6. An image restoration device for a specific downstream visual task, characterized in that, include: The adjustment unit is used to input the image features of the target image into the trained agent model to obtain the parameter adjustment amount; Based on the parameter adjustment amount, the multiple adjustment parameters included in the algorithm in the image restoration toolbox are adjusted; The scoring unit is used to process the target image using the adjusted image restoration toolbox to obtain a restored image, and to obtain the image quality score corresponding to the restored image based on the image quality assessment model. The output unit is used to take the restored image as the updated target image and repeat the steps of obtaining the parameter adjustment amount, adjusting the multiple adjustment parameters included in the algorithm in the image restoration toolbox, obtaining the restored image, and obtaining the image quality score corresponding to the restored image, until a set termination condition is reached, and outputs the restored image corresponding to the maximum value among the multiple obtained image quality scores as the image restoration result of the target image. The intelligent agent model is trained based on a first sample image set, the image restoration toolkit, and the image quality assessment model; the image quality assessment model is trained based on a second sample image set with image quality score labels; the image quality score labels are used to reflect the performance of sample images in the second sample image set in downstream visual tasks. The training method for the agent model includes: Select target sample images from the first sample image set, perform image restoration processing on the target sample images, and obtain and store the training data corresponding to each iteration in the image restoration process; wherein, the training data corresponding to each iteration includes: the image features of the input image in the current iteration, the parameter adjustment amount output by the intelligent agent model in the current iteration, the image features of the output image in the current iteration, and the difference in image quality scores between the output image and the input image in the current iteration; Randomly select one training data point from all currently stored training data, and update the network weights of the agent model based on the reinforcement learning algorithm; If the number of training iterations of the agent model reaches a first threshold, a trained agent model is obtained; if the number of training iterations of the agent model does not reach the first threshold, a new target sample image is selected from the first sample image set, and the steps of performing image restoration processing on the target sample image, obtaining and storing the training data corresponding to each iteration in the image restoration processing, and updating the network weights of the agent model are repeated until the number of training iterations of the agent model reaches the first threshold, and a trained agent model is obtained.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the image restoration method for a specific downstream visual task as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image restoration method for a specific downstream visual task as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image restoration method for a specific downstream visual task as described in any one of claims 1 to 5.

Images