Image self-optimization method and device, electronic equipment and storage medium

Abstract

The application provides an image self-optimization method and device, electronic equipment and storage medium. For an in-vehicle identification camera, the in-vehicle image of the camera is collected in real time, the internal parameters of the camera are automatically calibrated based on a deep reinforcement learning algorithm DDPG, the imaging quality of the camera is optimized, the generated image is more conducive to a target identification algorithm, and thus the self-optimization of the in-vehicle image is completed.

Description

TECHNICAL FIELD

[0001] The present application relates to the intelligent cockpit vehicle vision technical field, more particularly, to an image self-optimization method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the popularization and wide application of the intelligent cockpit concept, more and more vehicles complete the detection and recognition of in-vehicle personnel and objects by mounting in-vehicle recognition cameras, thereby providing users with more intelligent and personalized services, such as driver fatigue detection, face recognition, emotion recognition, dangerous driving behavior detection, and rear seat left object detection.

[0003] Currently, the implementation of these functions mainly involves real-time acquisition of in-vehicle images by in-vehicle recognition cameras, and then the detection of objects by the recognition algorithm of the back-end. Obviously, the imaging quality of the camera has a great influence on the recognition accuracy of the back-end algorithm. Due to the influence of many factors such as the type of the camera, the arrangement position, the environment in the vehicle cabin, and the lighting conditions, after the camera is installed and fixed, manual adjustment of the camera parameters is usually required, that is, the image quality is subjectively evaluated and adjusted by direct visual observation by the human eye. However, this approach has the following problems: on the one hand, it is difficult to ensure that the camera can achieve good imaging quality, and on the other hand, it is limited by the experience of the technical personnel, and the applicability is poor. SUMMARY

[0004] Therefore, in order to solve the above problems, the present application provides an image self-optimization method, device, electronic equipment and storage medium, and the technical solutions are as follows:

[0005] In one aspect, the present application provides an image self-optimization method, which comprises:

[0006] acquiring a target in-vehicle image collected by an in-vehicle recognition camera;

[0007] calling an optimization model trained in advance based on a deep deterministic policy gradient (DDPG) algorithm, wherein the optimization model is trained based on the current network of the action, and the state is the in-vehicle image, the action is the camera parameter offset, and the reward is the confidence returned by the target recognition algorithm;

[0008] inputting the target in-vehicle image into the optimization model to obtain a target camera parameter offset output by the optimization model;

[0009] adjusting the internal parameters of the in-vehicle recognition camera according to the target camera parameter offset to optimize the imaging quality of the in-vehicle recognition camera.

[0010] Optionally, the process of training the optimization model based on the deep deterministic policy gradient (DDPG) algorithm comprises:

[0011] building an action network and a critic network, the action network comprising an action current network and an action target network, the critic network comprising a critic current network and a critic target network;

[0012] emptying an experience replay pool, and randomly initializing network parameters of the action current network and the critic current network, network parameters of the action target network being obtained by soft updating network parameters of the action current network, network parameters of the critic target network being obtained by soft updating network parameters of the critic current network;

[0013] storing, based on the network parameters of the action current network, a data set comprising a first in-vehicle image, a first camera parameter offset and a confidence level in a same state, and a second in-vehicle image in a next state into the experience replay pool;

[0014] randomly batch sampling the experience replay pool to extract a target data set as a sample;

[0015] inputting the second in-vehicle image in the target data set into the action target network and the critic target network respectively, and inputting the first in-vehicle image, the first camera parameter offset and the confidence level in the target data set into the critic current network;

[0016] the action target network processes the inputted second in-vehicle image based on its network parameters to obtain a second camera parameter offset and outputs to the critic target network; the critic target network processes the second camera parameter offset and the inputted second in-vehicle image based on its network parameters to obtain a first critic value and outputs to the critic current network; the critic current network processes the inputted first in-vehicle image and the first camera parameter offset according to its network parameters to obtain a second critic value and outputs to the action current network; the critic current network calculates a loss function thereof using the first critic value, the second critic value and the inputted confidence level, and adjusts its network parameters according to a calculation result; the action current network calculates a loss function thereof using the second critic value, and adjusts its network parameters according to a calculation result;

[0017] in a case where a preset training end condition is met, taking the action current network with adjusted network parameters as the optimized model.

[0018] Optionally, the storing, based on the network parameters of the action current network, a data set into the experience replay pool comprises:

[0019] obtaining a first in-vehicle image in a first state collected by the in-vehicle recognition camera;

[0020] inputting the first in-vehicle image in the first state into the target recognition algorithm to obtain a confidence in the first state; and inputting the first in-vehicle image in the first state into the action current network, so that the action current network outputs a first camera parameter offset in the first state based on network parameters thereof;

[0021] adjusting internal parameters of the in-vehicle recognition camera according to the first camera parameter offset in the first state to obtain a second in-vehicle image in a second state collected by the in-vehicle recognition camera;

[0022] storing the first in-vehicle image in the first state, the first camera parameter offset and the confidence in the first state, and the second in-vehicle image in the second state as a set of data collection into the experience replay pool.

[0023] Optionally, the network structure of the action current network comprises, in sequence from input to output:

[0024] a first input layer, a first hidden layer, a second hidden layer, a third hidden layer, a fourth hidden layer, a fifth hidden layer and a first output layer, the first hidden layer, the second hidden layer, the third hidden layer, the fourth hidden layer and the fifth hidden layer each being composed of a convolution layer and a max-pooling layer, and the first output layer being set as a global average pooling layer;

[0025] the network structure of the action target network is the same as that of the action current network.

[0026] Optionally, the network structure of the critic current network comprises, in sequence from input to output:

[0027] a second input layer, a sixth hidden layer, a seventh hidden layer, an eighth hidden layer, a ninth hidden layer, a tenth hidden layer and a second output layer, the sixth hidden layer, the seventh hidden layer and the eighth hidden layer each being composed of a convolution layer and a max-pooling layer, the ninth hidden layer being composed of a global average pooling layer and a concatenation layer, the tenth hidden layer being set as a fully connected layer, and the second output layer being set as a single neuron;

[0028] the network structure of the critic target network is the same as that of the critic current network.

[0029] Another aspect of the present application provides an image self-optimization device, which comprises:

[0030] a vehicle-mounted camera parameter calibration module configured to acquire a target in-vehicle image collected by an in-vehicle recognition camera;

[0031] The vehicle-mounted camera image quality self-optimization network module is configured to call an optimization model trained in advance based on a deep deterministic policy gradient (DDPG) algorithm. The optimization model is trained based on a current network of actions, an in-vehicle image as a state, a camera parameter offset as an action, and a confidence returned by a target recognition algorithm as a reward. The target in-vehicle image is input into the optimization model to obtain a target camera parameter offset output by the optimization model.

[0032] The vehicle-mounted camera parameter calibration module is further configured to adjust internal parameters of the in-vehicle recognition camera according to the target camera parameter offset to optimize imaging quality of the in-vehicle recognition camera.

[0033] Optionally, the vehicle-mounted camera image quality self-optimization network module trains the optimization model based on a deep deterministic policy gradient (DDPG) algorithm, and the process includes:

[0034] building an action network and a critic network, the action network comprising an action current network and an action target network, the critic network comprising a critic current network and a critic target network; emptying an experience replay pool and randomly initializing network parameters of the action current network and the critic current network, network parameters of the action target network being obtained by soft updating network parameters of the action current network, network parameters of the critic target network being obtained by soft updating network parameters of the critic current network; storing a data set in the experience replay pool based on the network parameters of the action current network, the data set comprising a first in-vehicle image, a first camera parameter offset and a confidence level in a same state, and a second in-vehicle image in a next state; randomly batch sampling the experience replay pool to extract a target data set as a sample; inputting the second in-vehicle image in the target data set into the action target network and the critic target network respectively, and inputting the first in-vehicle image, the first camera parameter offset and the confidence level in the target data set into the critic current network; the action target network processing the input second in-vehicle image based on its network parameters to obtain a second camera parameter offset and output to the critic target network; the critic target network processing the second camera parameter offset and the input second in-vehicle image based on its network parameters to obtain a first critic value and output to the critic current network; the critic current network processing the input first in-vehicle image and first camera parameter offset according to its network parameters to obtain a second critic value and output to the action current network; the critic current network calculating a loss function thereof using the first critic value, the second critic value and the input confidence level, and adjusting its network parameters according to a calculation result; the action current network calculating a loss function thereof using the second critic value, and adjusting its network parameters according to a calculation result; in a case where a preset training end condition is met, taking the action current network with adjusted network parameters as the optimization model.

[0035] Optionally, the vehicle-mounted camera parameter calibration module is further configured to:

[0036] obtain a first in-vehicle image in a first state collected by the in-vehicle recognition camera; and after obtaining the first camera parameter offset in the first state output by the vehicle-mounted camera image quality self-optimization network module, adjust internal parameters of the in-vehicle recognition camera according to the first camera parameter offset in the first state to obtain a second in-vehicle image in a second state collected by the in-vehicle recognition camera;

[0037] Correspondingly, the process of storing a data set in the experience replay pool by the vehicle-mounted camera image quality self-optimization network module based on network parameters of the action current network comprises:

[0038] inputting the first in-vehicle image in the first state into the target recognition algorithm to obtain a confidence in the first state; and inputting the first in-vehicle image in the first state into the action current network, so that the action current network outputs a first camera parameter offset in the first state based on network parameters thereof; and storing the first in-vehicle image in the first state, the first camera parameter offset and the confidence, and the second in-vehicle image in the second state as a set of data collection into the experience replay pool.

[0039] In another aspect, the present application provides an electronic device, comprising at least one memory and at least one processor; the memory stores a program, and the processor invokes the program stored in the memory, and the program is used to implement each step of the image self-optimization method.

[0040] In another aspect, the present application provides a storage medium, wherein the storage medium stores computer executable instructions, and the computer executable instructions are used to execute each step of the image self-optimization method.

[0041] Compared with the prior art, the present application has the following beneficial effects:

[0042] The present application provides an image self-optimization method, device, electronic device and storage medium, which is used for an in-vehicle recognition camera, collects in-vehicle images in real time, automatically calibrates internal parameters of the camera based on a deep reinforcement learning algorithm DDPG, optimizes imaging quality of the camera, and makes the generated images more conducive to a target recognition algorithm, so as to complete self-optimization of the in-vehicle images. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0044] Figure 1 The method flowchart of the image self-optimization method provided by the embodiments of the present application is shown in the following.

[0045] Figure 2 The part of the method flowchart of the image self-optimization method provided by the embodiments of the present application is shown in the following.

[0046] Figure 3 The structural schematic diagram of the action current network provided by the embodiments of the present application is shown in the following.

[0047] Figure 4This is a schematic diagram of the current network structure of commentators provided in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of the structure of the DDPG network model provided in an embodiment of the present invention;

[0049] Figure 6 This is a schematic diagram of the image self-optimization device provided in an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Reinforcement learning involves interacting with the environment through a series of action strategies to generate new data. This new data is then used to modify the agent's action strategies. After several iterations, the agent learns the action strategies needed to complete the task. Deep reinforcement learning combines the high-dimensional input capabilities of deep learning with reinforcement learning. Furthermore, current deep learning-based object recognition technologies have achieved high accuracy, and their output confidence scores can be used as a measure of image quality.

[0053] This invention employs DDPG (Deep Deterministic Policy Gradient), a policy learning method for continuous behavior, to describe the process of optimizing the image quality of an in-vehicle recognition camera as a reinforcement learning problem. By adjusting the camera parameters, the current in-vehicle image state is obtained, and the policy is modified by reward feedback through the recognition algorithm, ultimately optimizing the imaging quality of the camera.

[0054] See Figure 1 The flowchart shown illustrates an image self-optimization method provided in this embodiment of the invention, which includes the following steps:

[0055] S10: Acquire the target vehicle interior image captured by the in-vehicle recognition camera.

[0056] In the embodiment of the present application, the target vehicle interior image is the vehicle interior image to be processed collected by the vehicle interior recognition camera in real time. The target vehicle interior image can also be pre-processed, such as cropping and rotating, before being input into the subsequent optimization model, and the embodiment of the present application does not limit this.

[0057] S20, calling an optimization model trained in advance based on a deep deterministic policy gradient (DDPG) algorithm, the optimization model being trained based on the current network of the action with the vehicle interior image as the state, the camera parameter offset as the action, and the confidence returned by the target recognition algorithm as the reward.

[0058] In the embodiment of the present application, the vehicle interior recognition camera image quality optimization process is described as a Markov decision process in a reinforcement learning problem, mainly including state s design, action a design, and reward function R design.

[0059] 1.1 State s design

[0060] In the camera image quality optimization task, the state space s is represented as an image space, which includes the input original image and all intermediate image results generated in the automatic optimization process. The s data format is [W, H, C], where W is the height of the image, H is the width of the image, and C is the depth of the image. For example, the s data format is [416, 416, 3], the height of the image is 416, the width of the image is 416, and the depth of the image is 3.

[0061] 1.2 Action a design

[0062] The internal parameters of the vehicle interior recognition camera are represented as , where N represents the number of internal parameters that can be adjusted by the camera to affect the image quality, including exposure value e, sharpness value sh, saturation value sa, white balance value wb, contrast value c, etc. In order to accelerate the network convergence process, the camera parameter adjustment amount is selected as the action a, and the action can be represented as a vector , where , the action a represents the camera parameter offset predicted each time, that is, after the action a is adjusted, the internal parameters will be adjusted to ; where f is the magnification factor corresponding to each parameter offset, which is represented as:

[0063] .

[0064] For example, the internal parameters of the vehicle interior recognition camera are represented as . In order to accelerate the network convergence process, the camera parameter adjustment amount is selected as the action a, and the action can be represented as a vector , where each component takes a value in the range [-1, 1], and after the action a is adjusted, the internal parameters will be adjusted to where f is defined as:

[0065] .

[0066] 1.3 Reward function r design

[0067] The goal of reinforcement learning problem is to maximize its total reward, in which the reward function is used to measure the benefit of the current decision. In the in-vehicle camera image quality optimization task, the target recognition algorithm can be used to measure the quality of the camera image, and the recognition confidence returned by the target recognition algorithm is used as the reward value. The reward function r is designed to input the image s in the current state to the target recognition algorithm, and return the value of the recognition confidence obtained, that is, the reward function can be represented as: , r e [0, 1] where the target recognition algorithm is represented as a function, and the returned confidence value is in the range [0, 1].

[0068] Therefore, the DDPG network takes the confidence returned by the target recognition algorithm as the reward and updates the DDPG network parameters.

[0069] In the specific implementation process, the process of training and optimizing the model based on the deep deterministic policy gradient DDPG algorithm includes the following steps, and the method flow chart is as shown in Figure 2 .

[0070] S201, build an action network and a critic network, the action network includes an action current network and an action target network, and the critic network includes a critic current network and a critic target network.

[0071] According to the "action-critic" structure in the DDPG algorithm and for the in-vehicle camera image quality optimization task, the embodiment of the application designs an action network to output an adjustment command, and designs a critic network to guide the update process of the action network. Among them, the action network includes an action current network and an action target network, and the critic network includes a critic current network and a critic target network. The specific design is as follows:

[0072] Action current network : responsible for iterative update of policy network parameters , responsible for selecting a current action according to the current state , used for interacting with the environment to generate a next state , a reward . The action current network includes 1 input layer, 5 hidden layers, and 1 convolutional output layer, wherein:

[0073] 1) The first input layer: the input is camera image data s, stored in the form of a matrix, with a data format of [W, H, C], where W is the height of the image, H is the width of the image, and C is the depth of the image.

[0074] 2) The first hidden layer: contains a convolutional layer and a max pooling layer, the convolutional layer has a kernel size of 3x3, a depth of 16, a convolutional step stride of 1, a padding method of zero padding, and a ReLU activation function; the max pooling layer has a filter size of 2x2 and a pooling step of 2.

[0075] 3) The second hidden layer: contains a convolutional layer and a max pooling layer, the convolutional layer has a kernel size of 3x3, a depth of 32, a convolutional step stride of 1, a padding method of zero padding, and a ReLU activation function; the max pooling layer has a filter size of 2x2 and a pooling step of 2.

[0076] 4) The third hidden layer: contains a convolutional layer and a max pooling layer, the convolutional layer has a kernel size of 3x3, a depth of 64, a convolutional step stride of 1, a padding method of zero padding, and a ReLU activation function; the max pooling layer has a filter size of 2x2 and a pooling step of 2.

[0077] 5) The fourth hidden layer: contains a convolutional layer and a max pooling layer, the convolutional layer has a kernel size of 3x3, a depth of 128, a convolutional step stride of 1, a padding method of zero padding, and a ReLU activation function; the max pooling layer has a filter size of 2x2 and a pooling step of 2.

[0078] 6) The fifth hidden layer: contains a convolutional layer and a max pooling layer, the convolutional layer has a kernel size of 3x3, a depth of 256, a convolutional step stride of 1, a padding method of zero padding, and a ReLU activation function; the max pooling layer has a filter size of 2x2 and a pooling step of 2.

[0079] 7) The first output layer: set as a global average pooling layer (GAP, Global Average Pooling), with a depth of N, and the output layer has a dimension of [1, 1, N], where N output values represent the adjustment amount of the internal parameters of the in-vehicle recognition camera, i.e., the action .

[0080] Referring to Figure 3 the structural diagram of the action target network. The image s input by the first input layer has a data format of [416, 416, 3], and the dimension of the first output layer is [1, 1, 5], with 5 output values representing the camera parameter adjustment amount, i.e., the action .

[0081] Action target network : its network structure and action current network is responsible for updating the next state sampled from the experience replay pool selects the optimal next action : its network parameters are updated from .

[0082] critic current network : responsible for iterative updating of the value network parameters , responsible for calculating the current target . The critic current network contains 1 input layer, 4 hidden layers, and 1 output layer, wherein:

[0083] 1) The second input layer: the input is the camera image data s, stored in the form of a matrix, with a data format of [W, H, C], where W is the height of the image, H is the width of the image, and C is the depth of the image.

[0084] 2) The sixth hidden layer (i.e., the first hidden layer of the critic current network): contains a convolutional layer and a max pooling layer, the convolutional layer has a kernel size of 5x5, a depth of 32, a convolutional step size stride of 1, a padding method of zero padding, and a ReLU activation function; the max pooling layer has a filter size of 2x2 and a pooling step size of 2.

[0085] 3) The seventh hidden layer (i.e., the second hidden layer of the critic current network): contains a convolutional layer and a max pooling layer, the convolutional layer has a kernel size of 5x5, a depth of 64, a convolutional step size stride of 1, a padding method of zero padding, and a ReLU activation function; the max pooling layer has a filter size of 2x2 and a pooling step size of 2.

[0086] 4) The eighth hidden layer (i.e., the third hidden layer of the critic current network): contains a convolutional layer and a max pooling layer, the convolutional layer has a kernel size of 5x5, a depth of 128, a convolutional step size stride of 1, a padding method of zero padding, and a ReLU activation function; the max pooling layer has a filter size of 2x2 and a pooling step size of 2.

[0087] 5) The ninth hidden layer (i.e., the fourth hidden layer of the critic current network): contains a global average pooling layer and a concatenation layer. The global average pooling layer has a depth of N, where N is the dimension size of the action a; the concatenation layer concatenates the output of the global average pooling layer with the action a, and the final output dimension is [1, 1, 2N].

[0088] 6) the tenth hidden layer (i.e. the fifth hidden layer of the critic current network): set as a full connection layer, including 30 neurons; the fifth hidden layer is a dropout layer, used to prevent model overfitting, and automatically discarding a certain proportion of neurons in the full connection layer.

[0089] 7) the second output layer: outputting a single neuron, whose value represents the current value.

[0090] Referring to Figure 4 the structural diagram of the critic current network shown in FIG. 1. The image s input by the second input layer has a data format of [416, 416, 3].

[0091] Critic target network : its network structure is consistent with that of the critic current network , and is responsible for calculating the value, whose network parameters are updated from .

[0092] In addition, for the action current network and the critic current network, the embodiment of the present application designs a loss function for each of them respectively:

[0093] 1) for the critic current network, its loss function adopts mean square error, that is:

[0094] (1)

[0095] wherein, , is a reward discount factor, and m is the sample set size of a random batch.

[0096] 2) for the action current network, its loss function is:

[0097] (2)

[0098] It should be noted that before building the action network and the critic network, the embodiment of the present application will first build a training environment:

[0099] First, install and fix the in-vehicle recognition camera on the vehicle and place the calibration samples. Then, in order to accelerate the network convergence speed, the imaging quality is subjectively viewed, and the camera parameters are manually adjusted so that the image quality is in a subjectively acceptable state; wherein the calibration samples include but are not limited to passengers and articles, and the manual adjustment mode is to manually change the built-in parameters of the camera through a camera parameter calibration tool.

[0100] S202, clear the experience replay pool and randomly initialize the network parameters of the current action network and the current commentator network. The network parameters of the action target network are obtained by softly updating the network parameters of the current action network, and the network parameters of the commentator target network are obtained by softly updating the network parameters of the current commentator network.

[0101] In this embodiment of the invention, an experience playback pool is set up to store... Where M is the maximum amount of data that the experience replay pool can store. The experience replay pool is first cleared during parameter initialization. For example, M can be 3000.

[0102] In addition, network parameters are initialized randomly. and , In addition, the initial update step number is 0.

[0103] S203, based on the network parameters of the current network, store a data set in the experience playback pool. The data set includes the first in-vehicle image in the same state, the first camera parameter offset and confidence level, and the second in-vehicle image in the next state.

[0104] As described above, in this embodiment of the invention, the experience replay pool can store multiple data sets. For each dataset, the following steps can be taken to obtain the data:

[0105] Acquire the first in-vehicle image in the first state captured by the in-vehicle recognition camera;

[0106] The first in-vehicle image in the first state is input into the target recognition algorithm to obtain the confidence level in the first state; and the first in-vehicle image in the first state is input into the current action network so that the current action network outputs the first camera parameter offset in the first state based on its network parameters.

[0107] Adjust the internal parameters of the in-vehicle recognition camera according to the parameter offset of the first camera in the first state, so as to obtain the second in-vehicle image captured by the in-vehicle recognition camera in the second state.

[0108] The first in-vehicle image in the first state, the first camera parameter offset and confidence level, and the second in-vehicle image in the second state are stored as a set of data in the experience playback pool.

[0109] In this embodiment of the invention, the image in the first state The input is fed into the current action network, which then selects the appropriate action based on its network parameters. ; Perform actions This involves adjusting the internal parameters of the camera, based on which new images can be obtained. , and The input to the target identification algorithm can obtain a single-step reward . Thus, the is stored in the experience replay pool as a data set.

[0110] S204, a random batch of experience replay pool is sampled to extract the target data set as a sample.

[0111] In the embodiment of the application, a random batch of m data sets is sampled from the experience replay pool, and the target data set is collected as a sample, and the target data set , wherein, The value range is 1 to m. For example, m can be 32.

[0112] S205, the second in-vehicle image in the target data set is input into the action target network and the critic target network, respectively, and the first in-vehicle image, the first camera parameter offset and the confidence in the target data set are input into the critic current network, so as to realize:

[0113] The action target network processes the input second in-vehicle image based on its network parameters to obtain the second camera parameter offset and outputs it to the critic target network; the critic target network processes the second camera parameter offset and the input second in-vehicle image based on its network parameters to obtain the first comment value and outputs it to the critic current network; the critic current network processes the input first in-vehicle image and first camera parameter offset according to its network parameters to obtain the second comment value and outputs it to the action current network; the critic current network calculates its loss function using the first comment value, the second comment value and the input confidence, and adjusts its network parameters according to the calculation result; the action current network calculates its loss function using the second comment value, and adjusts its network parameters according to the calculation result.

[0114] Referring to Figure 5 The structure diagram of the DDPG network model is shown. After obtaining the target data set from the experience replay pool, the state in the target data set is input into the action target network and the critic target network, respectively, the state , action and reward in the target data set are input into the critic current network.

[0115] The action target network processes the state based on its network parameters to obtain the optimal next action , which is input into the critic target network; the critic target network processes the action and the state based on its network parameters to obtain the first comment value It is then input into the critic's current network; the critic's current network processes the state based on its network parameters. and actions Get the second comment value It is then input into the action current network; the critic current network uses the first comment value. Second comment value and rewards The loss function is calculated according to formula (1), and the network parameters are updated using the gradient descent algorithm. Specifically, the network parameters can be implemented by the Adam optimizer. The update; the current network then uses the second comment value. The loss function is calculated according to formula (2), and the network parameters are updated using the gradient descent algorithm. Specifically, the network parameters can be implemented by the Adam optimizer. Update.

[0116] It should be noted that the network parameters of the action target network and the commentator target network are updated using a soft update method, as follows:

[0117] (3)

[0118] (4)

[0119] in, For example, the target network update factor. It can take the value 0.1.

[0120] S206, if the current training termination condition is met, use the current network with adjusted network parameters as the optimized model.

[0121] In this embodiment of the invention, if the update step number step is greater than the maximum update step number MAX_STSPS, then the training ends; otherwise, the update step number step is incremented by 1, and S203 continues. For example, the maximum update step number MAX_STSPS can be 40000.

[0122] Therefore, the trained DDPG network model (action current network) can also output camera parameter offsets according to the actual environmental conditions to perform online camera calibration and complete the self-optimization process of camera image imaging quality.

[0123] S30: Input the target vehicle interior image into the optimization model to obtain the target camera parameter offset output by the optimization model.

[0124] S40 adjusts the internal parameters of the in-vehicle recognition camera based on the target camera parameter offset to optimize the imaging quality of the in-vehicle recognition camera.

[0125] In this embodiment of the invention, during the calibration of camera parameters, an in-vehicle image of the current state can be acquired. The camera parameters are then predicted using a DDPG network to determine adjustment actions. The adjusted parameters are then burned into the camera hardware, allowing for the acquisition of in-vehicle images of the next state. Through continuous interactive updates, the camera's imaging quality is self-optimized.

[0126] In addition, the internal parameters of the in-vehicle recognition camera can be adjusted to optimize image quality, and the internal parameters can be recalibrated by burning in the parameters.

[0127] The advantages of this invention over the prior art are as follows:

[0128] (1) To address the issue of image quality optimization for in-vehicle recognition cameras, this invention proposes an image quality self-optimization scheme based on the deep reinforcement learning algorithm DDPG, which replaces the original manual calibration process. This scheme optimizes camera image quality through continuous interaction with the environment, ultimately improving the accuracy of the detection and recognition algorithm. This invention avoids the limitations of experience and subjectivity of technical personnel and is adaptable to different in-vehicle environments and application scenarios, thus having a wide range of applicability.

[0129] (2) The DDPG network structure designed in this invention can adapt to different camera types and input resolution specifications, and has strong scalability.

[0130] Based on the image self-optimization method provided in the above embodiments, this invention provides an apparatus for performing the image self-optimization method, the structural schematic diagram of which is shown below. Figure 6 As shown, it includes:

[0131] The vehicle-mounted camera parameter calibration module 10 is used to acquire the target vehicle interior image collected by the in-vehicle recognition camera.

[0132] The in-vehicle camera image quality self-optimization network module 20 is used to retrieve the optimization model pre-trained based on the Deep Deterministic Policy Gradient (DDPG) algorithm. The optimization model is trained on the current network with the in-vehicle image as the state, the camera parameter offset as the action, and the confidence score returned by the target recognition algorithm as the reward. The target in-vehicle image is input into the optimization model to obtain the target camera parameter offset output by the optimization model.

[0133] The vehicle-mounted camera parameter calibration module 10 is also used to adjust the internal parameters of the in-vehicle recognition camera according to the target camera parameter offset, so as to optimize the imaging quality of the in-vehicle recognition camera.

[0134] Optionally, the process of training the optimization model of the vehicle-mounted camera image quality self-optimization network module 20 based on the deep deterministic policy gradient (DDPG) algorithm includes:

[0135] An action network and a commentator network are constructed. The action network includes a current action network and an action target network, while the commentator network includes a current commentator network and a commentator target network. The experience replay pool is cleared, and the network parameters of the current action network and the current commentator network are randomly initialized. The network parameters of the action target network are obtained by softly updating the network parameters of the current action network, and the network parameters of the commentator target network are obtained by softly updating the network parameters of the current commentator network. Based on the network parameters of the current action network, a data set is stored in the experience replay pool. The data set includes the first in-vehicle image in the same state, the first camera parameter offset and confidence level, and the second in-vehicle image in the next state. The experience replay pool is randomly batch-sampled to extract the target data set as samples. The second in-vehicle image in the target data set is input into the action target network and the commentator target network respectively. At the same time, the first in-vehicle image and the first camera parameter offset in the target data set are input into the action target network and the commentator target network respectively. The offset and confidence score are input to the critic's current network to achieve the following: The action target network processes the input second in-vehicle image based on its network parameters to obtain the second camera parameter offset, and outputs it to the critic target network; The critic target network processes the second camera parameter offset and the input second in-vehicle image based on its network parameters to obtain the first review value, and outputs it to the critic current network; The critic current network processes the input first in-vehicle image and the first camera parameter offset based on its network parameters to obtain the second review value, and outputs it to the action current network; The critic current network calculates its loss function using the first review value, the second review value, and the input confidence score, and adjusts its network parameters according to the calculation result; The action current network calculates its loss function using the second review value, and adjusts its network parameters according to the calculation result; If the preset training termination condition is met, the action current network with adjusted network parameters is used as the optimized model.

[0136] Optionally, the vehicle-mounted camera parameter calibration module 10 is also used for:

[0137] Acquire a first in-vehicle image in a first state captured by an in-vehicle recognition camera; and, after obtaining the first camera parameter offset in the first state output by the in-vehicle camera image quality self-optimization network module, adjust the internal parameters of the in-vehicle recognition camera according to the first camera parameter offset in the first state to obtain a second in-vehicle image in a second state captured by the in-vehicle recognition camera.

[0138] Correspondingly, the process by which the vehicle-mounted camera image quality self-optimization network module 20 stores the data set in the experience playback pool based on the network parameters of the current network for the action includes:

[0139] The first in-vehicle image in the first state is input into the target recognition algorithm to obtain the confidence level in the first state; and the first in-vehicle image in the first state is input into the action current network so that the action current network outputs the first camera parameter offset in the first state based on its network parameters; and the first in-vehicle image in the first state, the first camera parameter offset and the confidence level, and the second in-vehicle image in the second state are stored as a set of data in the experience playback pool.

[0140] Based on the image self-optimization method provided in the above embodiments, this invention also provides an electronic device, which includes: at least one memory and at least one processor; the memory stores a program, and the processor calls the program stored in the memory, the program being used to implement the image self-optimization method.

[0141] Based on the image self-optimization method provided in the above embodiments, this embodiment of the invention also provides a storage medium storing computer-executable instructions for executing the image self-optimization method.

[0142] The above provides a detailed description of the image self-optimization method, apparatus, electronic device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0143] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0144] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that elements inherent to a process, method, article, or apparatus that comprises a list of elements, or elements inherent to such processes, methods, articles, or apparatus, are also included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image self-optimization method, characterized by, The method comprises: acquiring a target in-vehicle image collected by an in-vehicle recognition camera; calling an optimization model trained in advance based on a deep deterministic policy gradient (DDPG) algorithm, wherein the optimization model is trained based on an in-vehicle image as a state, a camera parameter offset as an action, and a confidence returned by a target recognition algorithm as a reward; inputting the target in-vehicle image into the optimization model to obtain a target camera parameter offset output by the optimization model; adjusting internal parameters of the in-vehicle recognition camera according to the target camera parameter offset to optimize imaging quality of the in-vehicle recognition camera; wherein the process of training the optimization model based on the deep deterministic policy gradient (DDPG) algorithm comprises: building an action network and a critic network, wherein the action network comprises an action current network and an action target network, and the critic network comprises a critic current network and a critic target network; emptying an experience replay pool and randomly initializing network parameters of the action current network and the critic current network; storing a data set in the experience replay pool based on the network parameters of the action current network, wherein the data set comprises a first in-vehicle image, a first camera parameter offset and a confidence under the same state, and a second in-vehicle image under a next state; randomly sampling the experience replay pool in batches to extract a target data set as a sample; inputting the second in-vehicle image in the target data set into the action target network and the critic target network respectively, and inputting the first in-vehicle image, the first camera parameter offset and the confidence in the target data set into the critic current network: the action target network processes the inputted second in-vehicle image based on its network parameters to obtain a second camera parameter offset and outputs it to the critic target network; the critic target network processes the second camera parameter offset and the inputted second in-vehicle image based on its network parameters to obtain a first critic value and outputs it to the critic current network; the critic current network processes the inputted first in-vehicle image and the first camera parameter offset based on its network parameters to obtain a second critic value and outputs it to the action current network; the critic current network calculates its loss function using the first critic value, the second critic value and the inputted confidence, and adjusts its network parameters according to the calculation result; the action current network calculates its loss function using the second critic value, and adjusts its network parameters according to the calculation result; in the case where the current training meets a preset end condition, the action current network with adjusted network parameters is taken as the optimization model.

2. The method of claim 1, wherein, The network parameters of the action target network are obtained by soft updating the network parameters of the action current network, and the network parameters of the critic target network are obtained by soft updating the network parameters of the critic current network.

3. The method of claim 2, wherein, The process of storing the data set in the experience replay pool based on the network parameters of the action current network comprises: acquiring a first in-vehicle image under a first state collected by the in-vehicle recognition camera; inputting the first in-vehicle image in the first state into the target recognition algorithm to obtain a confidence in the first state; and inputting the first in-vehicle image in the first state into the action current network, so that the action current network outputs a first camera parameter offset in the first state based on network parameters thereof; adjusting internal parameters of the in-vehicle recognition camera according to the first camera parameter offset in the first state to obtain a second in-vehicle image in a second state collected by the in-vehicle recognition camera; storing the first in-vehicle image in the first state, the first camera parameter offset and the confidence in the first state, and the second in-vehicle image in the second state as a set of data collections into the experience replay pool.

4. The method of claim 2, wherein, The network structure of the action current network sequentially comprises, from input to output: a first input layer, a first hidden layer, a second hidden layer, a third hidden layer, a fourth hidden layer, a fifth hidden layer, and a first output layer, the first hidden layer, the second hidden layer, the third hidden layer, the fourth hidden layer, and the fifth hidden layer each being composed of a convolution layer and a max-pooling layer, and the first output layer being set as a global average pooling layer; The network structure of the action target network is the same as that of the action current network.

5. The method of claim 2, wherein, The network structure of the critic current network sequentially comprises, from input to output: a second input layer, a sixth hidden layer, a seventh hidden layer, an eighth hidden layer, a ninth hidden layer, a tenth hidden layer, and a second output layer, the sixth hidden layer, the seventh hidden layer, and the eighth hidden layer each being composed of a convolution layer and a max-pooling layer, the ninth hidden layer being composed of a global average pooling layer and a concatenation layer, the tenth hidden layer being set as a fully connected layer, and the second output layer being set as a single neuron; The network structure of the critic target network is the same as that of the critic current network.

6. An image self-optimizing apparatus, characterized by, The device comprises: a vehicle-mounted camera parameter calibration module configured to acquire a target in-vehicle image collected by an in-vehicle recognition camera; a vehicle-mounted camera image quality self-optimization network module configured to call an optimization model pre-trained based on a deep deterministic policy gradient (DDPG) algorithm, the optimization model being trained based on an in-vehicle image as a state, a camera parameter offset as an action, and a confidence returned by a target recognition algorithm as a reward for an action current network; and inputting the target in-vehicle image into the optimization model to obtain a target camera parameter offset output by the optimization model; The vehicle-mounted camera parameter calibration module is further configured to adjust internal parameters of the in-vehicle recognition camera according to the target camera parameter offset to optimize imaging quality of the in-vehicle recognition camera. The process of training the optimization model by the vehicle-mounted camera image quality self-optimization network module based on the deep deterministic policy gradient (DDPG) algorithm comprises: The action network includes the action current network and an action target network, and the critic network includes a critic current network and a critic target network; an experience replay pool is emptied, and network parameters of the action current network and the critic current network are randomly initialized; a data set including a first in-vehicle image, a first camera parameter offset and a confidence level in a same state, and a second in-vehicle image in a next state is stored in the experience replay pool based on the network parameters of the action current network; the experience replay pool is randomly batch sampled to extract a target data set as a sample; the second in-vehicle image in the target data set is input into the action target network and the critic target network respectively, and the first in-vehicle image, the first camera parameter offset and the confidence level in the target data set are input into the critic current network; the action target network processes the input second in-vehicle image based on the network parameters thereof to obtain a second camera parameter offset and outputs the second camera parameter offset to the critic target network; the critic target network processes the second camera parameter offset and the input second in-vehicle image based on the network parameters thereof to obtain a first critic value and outputs the first critic value to the critic current network; the critic current network processes the input first in-vehicle image and the first camera parameter offset based on the network parameters thereof to obtain a second critic value and outputs the second critic value to the action current network; the critic current network calculates a loss function thereof by using the first critic value, the second critic value and the input confidence level, and adjusts the network parameters thereof according to a calculation result; the action current network calculates a loss function thereof by using the second critic value, and adjusts the network parameters thereof according to a calculation result; and in a case where a preset training end condition is met, the action current network with the adjusted network parameters is taken as the optimization model.

7. The apparatus of claim 6, wherein, The network parameters of the action target network are obtained by soft updating the network parameters of the action current network, and the network parameters of the critic target network are obtained by soft updating the network parameters of the critic current network.

8. The apparatus of claim 7, wherein, The vehicle-mounted camera parameter calibration module is further configured to: obtain a first in-vehicle image in a first state collected by the in-vehicle recognition camera; and after obtaining the first camera parameter offset in the first state output by the vehicle-mounted camera image quality self-optimization network module, adjust internal parameters of the in-vehicle recognition camera according to the first camera parameter offset in the first state to obtain a second in-vehicle image in a second state collected by the in-vehicle recognition camera; Correspondingly, the process of storing a data set in the experience replay pool by the vehicle-mounted camera image quality self-optimization network module based on the network parameters of the action current network includes: inputting the first in-vehicle image in the first state into the target recognition algorithm to obtain a confidence level in the first state; and inputting the first in-vehicle image in the first state into the action current network to enable the action current network to output a first camera parameter offset in the first state based on network parameters thereof; and storing the first in-vehicle image in the first state, the first camera parameter offset and the confidence level, and the second in-vehicle image in the second state as a set of data collection into the experience replay pool.

9. An electronic device, comprising: The electronic device comprises at least one memory and at least one processor; the memory stores a program, and the processor invokes the program stored in the memory, and the program is used to implement the image self-optimization method in any one of claims 1-5.

10. A storage medium, characterized by The storage medium stores computer executable instructions, and the computer executable instructions are used to execute the image self-optimization method in any one of claims 1-5.

Images