Image enhancement method, vehicle snapshot method, device and medium

Abstract

The application relates to an image enhancement method, a vehicle snapshot method, equipment and a medium, and belongs to the technical field of image processing. The method comprises the following steps: performing preprocessing on original image data collected by an image collection component; inputting the preprocessed image data into an image enhancement model to obtain image data after brightness enhancement; the calculation efficiency of the image enhancement network can be improved by performing down-sampling processing on the image input into the student network. The image input into the teacher network is not subjected to down-sampling processing, the performance of the teacher network in image enhancement can be ensured, and the calculation speed of the student network can also be ensured, and the network complexity of the student network is lower than that of the teacher network. Then, the teacher network distills the learned knowledge to the student network, so that the student network can ensure the image enhancement performance while ensuring the calculation speed. Sharing the intermediate feature map output by the teacher network to the student network can also improve the training efficiency of the student network.

Description

Technical Field

[0001] This application relates to an image enhancement method, a vehicle capture method, a device, and a medium, belonging to the field of image processing technology. Background Technology

[0002] Deep learning-based object detection has been applied in many real-world scenarios, such as pedestrian recognition, autonomous driving, and image segmentation. However, object detection models can only detect objects in images captured when the brightness of the scene meets preset requirements. In such cases, images captured in low-light scenes require brightness enhancement.

[0003] Traditional brightness enhancement methods involve using a low-light image enhancement network based on a deep neural network to enhance the brightness of low-light images, resulting in an enhanced illuminated image.

[0004] However, in order to improve the brightness enhancement performance of low-light image enhancement networks, the model complexity of these networks is usually high, which leads to low computational efficiency and consequently, low computational efficiency and high latency in image enhancement. Summary of the Invention

[0005] This application provides an image enhancement method, a vehicle capture method, an apparatus, and a medium. On one hand, downsampling the image input to the student network improves the computational efficiency of the student network. On the other hand, the image input to the teacher network is not downsampled; only the downsampled image is input to the student network. This ensures both the performance of the teacher network in image enhancement and the computational speed of the student network. Furthermore, the network complexity of the student network is lower than that of the teacher network. The teacher network then distills the learned knowledge into the student network, thus maintaining both computational speed and image enhancement performance. This improves both the efficiency and effectiveness of image enhancement. Simultaneously, during knowledge distillation, intermediate feature maps output by the teacher network are shared with the corresponding network layers of the student network, enabling the student network to accelerate convergence and improve training efficiency. This application provides the following technical solutions:

[0006] Firstly, an image enhancement method is provided, the method comprising:

[0007] Obtain the raw image data acquired by the image acquisition component;

[0008] The original image data is preprocessed to obtain preprocessed image data, which is adapted to the input layer of the image enhancement model.

[0009] input the preprocessed image data into the image enhancement model to obtain image data with enhanced brightness, the image enhancement model being obtained by distilling knowledge of a teacher network to a student network, the image input into the student network during the knowledge distillation being obtained by down-sampling an image input into the teacher network, and an intermediate feature map output by the teacher network during the knowledge distillation being shared to a corresponding network layer of the student network; the student network being trained using the intermediate feature map and the down-sampled sample image, the network complexity of the student network being lower than that of the teacher network.

[0010] Optionally, the training process of the image enhancement model comprises:

[0011] obtaining training data, the training data comprising the sample image and a label image corresponding to the sample image, the brightness of the label image being greater than that of the sample image;

[0012] preprocessing the sample image so that the preprocessed sample image meets the input requirement of the teacher network;

[0013] inputting the preprocessed sample image into the teacher network to obtain a soft label image and an intermediate feature map;

[0014] sharing the intermediate feature map to a corresponding network layer of the student network to obtain a soft label prediction image and a first hard label prediction image output by the student network;

[0015] training the teacher network based on the difference between the soft label image and the soft label prediction image;

[0016] down-sampling the preprocessed sample image to obtain a down-sampled sample image;

[0017] inputting the down-sampled sample image into the student network to obtain a second hard label prediction image;

[0018] training the student network based on the difference between the first hard label prediction image and the label image, the difference between the second hard label prediction image and the label image, and the difference between the soft label image and the soft label prediction image, to obtain the image enhancement model.

[0019] Optionally, the teacher network comprises at least two layers of first feature extraction layers and an output layer connected in sequence, and the student network comprises at least two layers of second feature extraction layers and an output layer connected in sequence; the number of the second feature extraction layers is equal to the number of the first feature extraction layers, the model complexity of the second feature extraction layers is lower than the model complexity of the first feature extraction layers, and the dimension of the feature map output by each layer of the first feature extraction layers is the same as the dimension of the feature map output by the corresponding layer of the second feature extraction layers; the inputting of the preprocessed sample image into the teacher network to obtain a soft label image and an intermediate feature map comprises:

[0020] the inputting of the preprocessed sample image into the teacher network to obtain an intermediate feature map output by an i-th layer of first feature extraction layers of the teacher network and a soft label image output by an output layer of the teacher network; wherein the i is a positive integer;

[0021] Correspondingly, the sharing of the intermediate feature map to the corresponding network layer of the student network to obtain a soft label prediction image and a first hard label prediction image output by the student network comprises:

[0022] the intermediate feature map is taken as an output result of an i-th layer of second feature extraction layers in the student network to input an i+1-th layer of second feature extraction layers to obtain the soft label prediction image and the first hard label prediction image.

[0023] Optionally, the training of the student network based on the difference between the first hard label prediction image and the label image, the difference between the second hard label prediction image and the label image, and the difference between the soft label image and the soft label prediction image to obtain the image enhancement model comprises:

[0024] the inputting of the soft label image and the soft label prediction image into a first loss function to obtain a teacher loss value;

[0025] the inputting of the label image and the first hard label prediction image into a second loss function to obtain a first loss value;

[0026] the inputting of the label image and the second hard label prediction image into the second loss function to obtain a second loss value;

[0027] determining a weighted sum of the first loss value and the second loss value to obtain a student loss value;

[0028] updating network parameters in the student network based on the teacher loss value and the student loss value to train the student network.

[0029] Optionally, the sample image is in a Bayer format, and the preprocessing of the sample image comprises:

[0030] extracting a region of interest in the sample image data;

[0031] channel separating the region of interest to obtain a separated sample image;

[0032] adjusting brightness of the separated sample image based on a pre-set brightness adjustment parameter to obtain the pre-processed sample image.

[0033] Optionally, the pre-processing of the original image data comprises:

[0034] extracting a region of interest in the original image data;

[0035] down-sampling the region of interest;

[0036] channel separating the down-sampled image data;

[0037] adjusting brightness of the channel separated image data based on a pre-set brightness adjustment parameter to obtain the pre-processed image data.

[0038] Optionally, the original image data comprises a plurality of groups of Raw data stored in a Bayer format, and the down-sampling of the region of interest comprises:

[0039] extracting a group of Raw data every n rows and n columns to obtain a plurality of groups of Raw data; the n is a positive integer.

[0040] In a second aspect, a vehicle snapshot method is provided, and the method comprises:

[0041] obtaining original image data collected by an image collection component on a vehicle;

[0042] obtaining brightness-enhanced image data based on the original image data and a pre-trained image enhancement model; the image enhancement model is obtained by distilling knowledge of a teacher network to a student network, an image input to the student network in the knowledge distillation process is obtained by down-sampling an image input to the teacher network, and an intermediate feature map output by the teacher network in the knowledge distillation process is also shared to a corresponding network layer of the student network; the student network is trained using the intermediate feature map and a down-sampled sample image, and a network complexity of the student network is lower than a network complexity of the teacher network;

[0043] inputting the brightness-enhanced image data into a pre-trained vehicle detection network to obtain a vehicle detection result;

[0044] In a case where the vehicle detection result indicates that a vehicle is detected, the light-emitting component is controlled to emit light, and the image capturing component is controlled to capture an image again to obtain a snapshot image of the vehicle.

[0045] In a third aspect, an electronic device is provided, which includes a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement the image enhancement or vehicle snapshot method provided in the above aspects.

[0046] In a fourth aspect, a computer-readable storage medium is provided, which stores a program, which, when executed by a processor, is used to implement the image enhancement or vehicle snapshot method provided in the above aspects.

[0047] The beneficial effects of the present application at least include: obtaining original image data captured by an image capturing component; pre-processing the original image data to obtain pre-processed image data; inputting the pre-processed image data into an image enhancement model to obtain image data after brightness enhancement; since the larger the size of the image input into the student network, the longer the student network takes to extract features, on the one hand, the image input into the student network is subjected to down-sampling processing, which can improve the computing efficiency of the student network. On the other hand, in the traditional knowledge distillation method, the size of the image input into the student network and the teacher network is the same. If the down-sampled sample image is input into the teacher network, it will affect the performance of the teacher network in image enhancement. In the present application, the image input into the teacher network during the training of the image enhancement model is not subjected to down-sampling processing, and only the down-sampled image is input into the student network, which can ensure the performance of the teacher network in image enhancement and the computing speed of the student network, and the network complexity of the student network is lower than that of the teacher network. Then, the teacher network distills the learned knowledge to the student network, so that the student network can ensure the computing speed and the image enhancement performance at the same time. In this way, the efficiency of image enhancement can be improved and the effect of image enhancement can be ensured at the same time in the image enhancement process. At the same time, since the size of the image of the teacher network and the image input into the student network is inconsistent, it will cause the problem that the student network is difficult to converge in the knowledge distillation process. Based on this, in the present application, the intermediate feature map output by the teacher network is also shared to the corresponding network layer of the student network in the knowledge distillation process, so as to accelerate the convergence of the student network using the intermediate feature map and improve the training efficiency of the student network (image enhancement network).

[0048] In addition, by training the teacher network and the student network at the same time, it is not necessary to separately pre-train the teacher network, which can improve the model training efficiency.

[0049] In addition, the student loss value is determined by a weighted sum of the first loss value and the second loss value, and the network parameters of the student network are updated based on the student loss value and the teacher loss value. Compared with updating the network parameters of the student network based on the first loss value and the teacher loss value and the second loss value and the teacher loss value respectively, the network parameters of the student network only need to be updated once for each preprocessed sample image, the number of times of updating the network parameters of the student network can be reduced, and since the student loss value is obtained by combining the first loss value and the second loss value, the accuracy of updating the network parameters using the student loss value and the teacher loss value will not be reduced.

[0050] In addition, by extracting the region of interest in the original image data, and performing down-sampling processing on the region of interest, on the one hand, the calculation amount of the image enhancement model can be reduced, and on the other hand, the image data of the region of interest can be ensured not to lose too much, thereby ensuring the image enhancement effect.

[0051] In addition, by adjusting the brightness of the image based on the pre-set brightness adjustment parameter before the image is input into the image enhancement model, it can be ensured that the brightness of the image input into the image enhancement model is basically consistent, so that the brightness of the image output by the image enhancement model is also basically consistent, thereby ensuring the stability of the model output result.

[0052] In addition, by flashing the light-emitting component once when it is detected that the image data after brightness enhancement has a vehicle, on the one hand, the energy consumption of the light-emitting component can be reduced, the service life of the light-emitting component can be greatly prolonged, and the light pollution caused by the constant light of the light-emitting component in dark light conditions can be reduced; on the other hand, the snapshot rate of the vehicle in the dark light environment can also be ensured.

[0053] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, and can be implemented according to the content of the specification, the following will be described in detail with the preferred embodiments of the present application and with the help of the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 is a flowchart of an image enhancement method provided by an embodiment of the present application;

[0055] Figure 2 is a schematic diagram of down-sampling and channel separation of original image data provided by an embodiment of the present application;

[0056] Figure 3 is a flowchart of a training method of an image enhancement model provided by an embodiment of the present application;

[0057] Figure 4 is a schematic diagram of down-sampling of a preprocessed sample image provided by an embodiment of the present application;

[0058] Figure 5 is a schematic diagram of a knowledge distillation process provided by an embodiment of the present application;

[0059] Figure 6 is a flowchart of a vehicle snapshot method provided by an embodiment of the present application;

[0060] Figure 7 is a flowchart of a vehicle snapshot method provided by another embodiment of the present application;

[0061] Figure 8 is a schematic diagram of a vehicle snapshot process provided by an embodiment of the present application;

[0062] Figure 9 is a block diagram of an image enhancement device provided by an embodiment of the present application;

[0063] Figure 10 is a block diagram of a vehicle snapshot device provided by an embodiment of the present application;

[0064] Figure 11 is a block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0065] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings and embodiments. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0066] First, several terms related to the present application are introduced.

[0067] Knowledge distillation: is a commonly used method of model compression, unlike pruning and quantization in model compression, knowledge distillation is to train a small model by constructing a lightweight model, using the supervision information of a better large model, in order to achieve better performance and accuracy. Among them, the large model is generally called teacher (teacher network), and the small model is generally called Student (student network). The supervision information from the teacher network output is called knowledge (knowledge), and the process of the student network learning to transfer the supervision information from the teacher network is called distillation (distillation).

[0068] Generally, when distillation is used, a student network with a smaller parameter amount is often found, and the lightweight student network cannot learn the potential relationship between the data sets well compared to the teacher network. The teacher network performs logistic regression (softmax) on the output vector logits, which more smoothly processes the labels, such as outputting the number 1 as 0.6 (prediction of 1) and 0.4 (prediction of 0), and then inputting the smoothed label into the student network. Compared with 1, the smoothed label contains more information. The goal of distillation is to let the student network learn the generalization ability of the teacher network, and theoretically, the result obtained will be better than the student network that simply fits the training data.

[0069] In the traditional knowledge distillation process, the images input to the teacher network and the student network are consistent. In the image enhancement scene, the teacher network pays more attention to the image enhancement performance, and the student network pays more attention to the real-time performance of the calculation. Based on this, in order to ensure the image enhancement performance of the teacher network, the image size input to the teacher network cannot be too small. At this time, if the image input to the teacher network is input to the student network, even if the complexity of the student network is low, due to the large size of the image, the real-time performance of the student network calculation will still be reduced.

[0070] Based on the above technical problems, in the present application, the image input to the student network is obtained by down-sampling the image input to the teacher network. That is, the size of the image input to the teacher network and the size of the image input to the student network are inconsistent.

[0071] However, due to the inconsistency of the size of the image input to the teacher network and the size of the image input to the student network, it will cause the problem that the student network is difficult to converge in the knowledge distillation process. Based on this, in the present application, in the knowledge distillation process, the intermediate feature map output by the teacher network will also be shared to the corresponding network layer of the student network, so that the student network uses the intermediate feature map to accelerate the convergence. For specific content, please refer to the following embodiments.

[0072] Soft label: refers to the output of the softmax layer obtained by the input data through the teacher network. Compared with the hard label (Ground Truth), the soft label has higher entropy and smaller gradient change.

[0073]

[0074] The above formula is a Softmax function, when the temperature parameter T is 1, it is a standard Softmax formula, and the greater the value of T is, the smoother the distribution of the result is. More information is obtained. The hard label prediction image (hard predictions) of the student network when the network is trained by the present application is obtained by T = 1; the soft label image (soft lables) of the teacher network and the soft label prediction image (soft predictions) of the student network are obtained by T = t (t > 1).

[0075] Bayer format: through the analysis of the perception of human eyes to colors, it is found that human eyes are more sensitive to green, so the number of green pixels in a general bayer format image is the sum of the number of red and blue pixels. A general bayer array is composed of 1 / 2 green, 1 / 4 red and 1 / 4 blue.

[0076] Optionally, the present application takes the electronic device provided by each embodiment as an example for description, the electronic device is a terminal or a server, the terminal can be a mobile phone, a computer, a tablet computer, a scanner, an electronic eye, a monitoring camera, etc., and the type of the electronic device is not limited in the embodiment.

[0077] Figure 1 is a flowchart of an image enhancement method provided by an embodiment of the present application, and the method comprises at least the following steps:

[0078] In step 101, raw image data collected by an image collection component is obtained.

[0079] Optionally, the working environment of the image collection component is a low-brightness environment, or an environment with light intensity lower than a brightness threshold value, and the image collected in the environment generally cannot be recognized by a target detection model. At this time, the raw image data is a low-brightness image.

[0080] The raw image data refers to Raw data stored in a bayer format. Since the bayer format is different from image formats such as jpg, the Raw data stored in the bayer format has not been converted, and therefore the probability of image loss is low. Using the raw image data for image enhancement can ensure the image enhancement effect.

[0081] In step 102, the raw image data is preprocessed to obtain preprocessed image data, and the preprocessed image data is adapted to the input layer of a pre-trained image enhancement model.

[0082] The preprocessed image data is adapted to the input layer of the image enhancement model, including that the image size of the preprocessed image data is the same as the image size allowed by the input layer of the image enhancement model. For example, the image size allowed by the input layer of the image enhancement model is 750*250, and the image size of the preprocessed image data should also be 750*250.

[0083] In one example, the electronic device only performs downsampling processing on the original image data according to the image size allowed by the input layer, to obtain the preprocessed image data.

[0084] In another example, the electronic device extracts a region of interest (ROI) in the original image data; performs downsampling processing on the region of interest; performs channel separation on the down-sampled image data; adjusts the brightness of the channel-separated image data based on a pre-set brightness adjustment parameter, to obtain the preprocessed image data.

[0085] Since the post-processing process after image enhancement may only be interested in the target, in this example, by first extracting the region of interest and then performing downsampling processing on the region of interest, on the one hand, the computational load of the image enhancement model can be reduced, and on the other hand, the image data of the region of interest can be ensured not to lose too much, thereby ensuring the effect of image enhancement.

[0086] For example, an original image data with a resolution of 4096*2176 can usually obtain an effective region of interest ROI region with a size of about 3000*1000 in a lens or police scene. In order to further reduce the computational load, the 3000*1000 ROI region needs to be down-sampled.

[0087] The way of extracting the region of interest in the original image data includes extracting a region with a preset size at a target position. The target position can be the middle position of the original image region, or other positions of the original image region. The target position and the preset size can be set by the user or fixedly set in the electronic device, and the implementation of the target position and the preset size is not limited in this embodiment.

[0088] The original image data includes multiple groups of Raw data stored in Bayer format, and the downsampling processing on the region of interest includes extracting a group of Raw data every n rows and n columns, to obtain multiple groups of Raw data.

[0089] Due to the limitation of the Bayer format, the value of n is a positive odd number, and the value of n is pre-stored in the electronic device. As the value of n increases, the brightness enhancement effect will also deteriorate. Based on this, in this embodiment, the value of n is taken as 1 for example.

[0090] Channel separation is performed on the downsampled image data, including separating the downsampled image data into channels according to pixel type. Taking the extraction and channel separation process when n is 1 as an example, refer to... Figure 2 As shown, assuming the image data of the region of interest is grouped into four data points (RGGB), such as... Figure 2 The data enclosed in dashed boxes contains a total of 3000 * 1000 / 4 = 750,000 data sets. For example... Figure 2 As shown, taking only the data from odd-numbered rows and columns yields a total of 750000 / 4 = 187500 data sets. Each set's four pixel values ​​(RGGB) are then split into four channels according to their pixel type (RGGB), resulting in... Figure 2 The last image shows the format. At this point, a data set of size 750*250*4 is obtained, with the RGGB data separated into individual channels.

[0091] Since the number of bits per data item stored by the image acquisition component is unreadable by the computer, after channel separation of the downsampled image data, the number of bits per pixel data can be converted to a preset number of bits, which is a number of bits readable by the computer. For example, the number of bits per data item stored by the image acquisition component is 12 bits. The preset number of bits readable by the computer is 8 bits or 16 bits. To reduce the amount of data processed by the model, this embodiment takes an 8-bit preset number of bits as an example. In this case, the pixel data can be shifted right by 4 bits, thereby converting the number of bits per pixel data to the preset number of bits.

[0092] In other embodiments, the number of computer-readable bits can also be 16 bits. In this case, it is only necessary to pad the pixel data to 16 bits. This embodiment does not limit the method of converting the number of bits of pixel data.

[0093] The brightness of the image data after channel separation is adjusted based on the pre-set brightness adjustment parameters to obtain pre-processed image data, including: obtaining the brightness adjustment parameters and adjusting the brightness of the image data after channel separation to the brightness level indicated by the brightness adjustment parameters.

[0094] The process of adjusting the brightness of the image data after channel separation can be set by the following function: B = clip(A*scale, 0, 255). A represents the pixel value of the image data after channel separation, scale is the brightness adjustment parameter, 0 represents the minimum value of the pixel value after brightness adjustment, 255 represents the maximum value of the pixel value after brightness adjustment, and B represents the pixel value obtained after brightness enhancement according to scale. The pixel value after brightness adjustment needs to be limited in the value range formed by the minimum value and the maximum value. Scale is a value set by the user, and the value range of scale is [1.0, 5.0] illustratively. In other embodiments, the value range of scale can also be other numerical values, and the present embodiment does not limit the implementation manner of the value range.

[0095] Step 103: input the preprocessed image data into the image enhancement model to obtain image data after brightness enhancement.

[0096] In the present embodiment, the image enhancement model is obtained after distilling the knowledge of the teacher network to the student network. In the knowledge distillation process, the image input to the student network is obtained by down-sampling the image input to the teacher network, and the intermediate feature map output by the teacher network is also shared to the corresponding network layer of the student network during the knowledge distillation process; the student network is trained using the intermediate feature map and the down-sampled sample image.

[0097] Specifically, Figure 3 is a flowchart of a training method of an image enhancement model provided by an embodiment of the present application, and the method includes at least the following steps:

[0098] Step 31: obtain training data, which includes a sample image and a label image corresponding to the sample image.

[0099] The brightness of the label image is greater than the brightness of the sample image.

[0100] In one example, the label image and the sample image are obtained by photographing the same target under different illuminations; or the sample image and the label image are obtained by photographing the same target under the same illumination using different exposure times, and the exposure time of the label image is greater than the exposure time of the sample image. The target can be a vehicle, a person, or other objects that need to be detected and recognized subsequently, and the type of the target is not limited in the present embodiment.

[0101] Step 32: pre-process the sample image to meet the input requirements of the teacher network.

[0102] In one example, the sample image is in a Bayer format, and the pre-processing of the sample image includes: extracting a region of interest in sample image data; performing channel separation on the region of interest to obtain a separated sample image; and adjusting the brightness of the separated sample image based on a pre-set brightness adjustment parameter to obtain a pre-processed sample image.

[0103] The region of interest extraction, the channel separation, and the brightness adjustment process are described in the related description of step 102, and will not be repeated here.

[0104] Step 33: inputting the pre-processed sample image into the teacher network to obtain a soft label image and an intermediate feature map.

[0105] In this embodiment, the teacher network includes at least two layers of first feature extraction layers and an output layer connected in sequence, and the student network includes at least two layers of second feature extraction layers and an output layer connected in sequence; the number of the second feature extraction layers is equal to the number of the first feature extraction layers, the model complexity of the second feature extraction layers is lower than the model complexity of the first feature extraction layers, and the dimension of the feature map output by each layer of the first feature extraction layers is the same as the dimension of the feature map output by the corresponding layer of the second feature extraction layers.

[0106] Correspondingly, inputting the pre-processed sample image into the teacher network to obtain a soft label image and an intermediate feature map includes: inputting the pre-processed sample image into the teacher network to obtain an intermediate feature map output by the i-th layer of the first feature extraction layers of the teacher network and a soft label image output by the output layer of the teacher network; where i is a positive integer.

[0107] When constructing the teacher network, instead of paying too much attention to the size of the network model, attention is paid to the image enhancement performance of the network model. It is assumed that the teacher network includes three layers of first feature extraction layers Ta, Tb, and Tc. The deep learning network structure used in the first feature extraction layers can be a multi-layer convolution, deconvolution, and / or a hollow convolution, etc., and the network structures of different first feature extraction layers are the same or different, and the implementation of the first feature extraction layers is not limited in this embodiment.

[0108] The value of i is pre-set in the electronic device, the i-th layer can be one layer or at least two layers, and the value of i is less than the number of the first feature extraction layers. For example, in the case where the first feature extraction layers include three layers, i can be 1, or 2, or 1 and 2, and the value of i is not limited in this embodiment.

[0109] When constructing the student network, more attention is paid to the time consumption of the network model. Based on this, each second feature extraction layer in the student network uses a simple convolution layer, such as using a common 3x3, 5x5 convolution, and the number of convolution layers in each second feature extraction layer is not more than a preset number of layers (such as 5 layers, or 4 layers, etc., and the value of the preset number of layers is not limited in the embodiment) when constructing. The number of second feature extraction layers in the student network is equal to the number of first feature extraction layers in the teacher network. Taking a teacher network including three first feature extraction layers Ta, Tb and Tc as an example, the student network also includes three second feature extraction layers Sa, Sb and Sc, and it is necessary to ensure that the output dimensions of Ta and Sa are consistent, the output dimensions of Tb and Sb are consistent, and the output dimensions of Tc and Sc are consistent.

[0110] The preset number of layers is less than or equal to the number of network layers in each first feature extraction layer in the teacher network, in other words, the preset number of layers is less than or equal to the number of network layers of the feature extraction layer in the neural network established based on the image enhancement performance.

[0111] Step 34, sharing the intermediate feature map to the corresponding network layer of the student network to obtain the soft label prediction image and the first hard label prediction image output by the student network.

[0112] Specifically, the intermediate feature map is shared to the corresponding network layer of the student network to obtain the soft label prediction image and the first hard label prediction image output by the student network, including: taking the intermediate feature map as the output result of the i-th second feature extraction layer in the student network, inputting the i+1-th second feature extraction layer to obtain the soft label prediction image and the first hard label prediction image.

[0113] In the present application, the image input size of the teacher network and the student network is inconsistent, which will cause the problem that it is difficult to converge when directly training the student network. Based on this, in the present embodiment, part of the intermediate feature maps extracted by the teacher network are used to accelerate the convergence of the student network when training the student network.

[0114] Step 35, training the teacher network based on the difference between the soft label image and the soft label prediction image.

[0115] Specifically, the soft label image and the soft label prediction image are input into a first loss function to obtain a teacher loss value; the network parameters of the teacher network are updated based on the teacher loss value to train the teacher network.

[0116] The first loss function can be cross-entropy or extended softmax, and the implementation of the first loss function is not limited in the embodiment.

[0117] In other embodiments, the teacher network can also be a pre-trained teacher network, in which case step 36 is performed directly after step 34 without further training of the teacher network.

[0118] At step 36, the pre-processed sample image is down-sampled to obtain a down-sampled sample image.

[0119] Since the larger the size of the image input to the student network, the longer the student network takes to perform feature extraction. Therefore, in order to improve the computational efficiency of the student network, the pre-processed sample image needs to be down-sampled. In the traditional knowledge distillation method, the size of the image input to the student network and the teacher network is the same, and if the down-sampled sample image is input to the teacher network, it will affect the performance of the teacher network in image enhancement. Based on this, in the present embodiment, it is creatively proposed that the image size of the pre-processed sample image is not changed and is directly input to the teacher network, and the down-sampled sample image is input to the student network. By inputting images of different sizes to the teacher network and the student network during knowledge distillation, the performance of the teacher network in image enhancement can be guaranteed, and the computational speed of the student network can also be guaranteed. Then the teacher network distills the learned knowledge to the student network, so that the student network can guarantee the image enhancement performance while guaranteeing the computational speed.

[0120] The down-sampling of the pre-processed sample image can be reducing the image size of the pre-processed sample image to 1 / 4, 1 / 16, etc. of the original size, and the present embodiment does not limit the down-sampling method. For example, the pre-processed sample image DataT is 1500*500*4, i.e. the image size is 1500*500 of 4-channel data; the down-sampled sample image DataS obtained by down-sampling the pre-processed sample image DataT is 750*250*4, i.e. the image size is 750*250 of 4-channel data. As can be seen, the data amount of DataS is only 1 / 4 of that of DataT.

[0121] In one example, the down-sampling of the pre-processed sample image includes: extracting one pixel data every preset row and preset column of the pixel data of each pixel channel in the pre-processed sample image to obtain the down-sampled sample image. For example, referring to Figure 4 In the pre-processed sample image DataT, one pixel data is extracted from each pixel channel of the odd row and the odd column, respectively, to obtain the down-sampled pixel data DataS corresponding to each pixel channel.

[0122] Due to the limitations of the Bayer format, the preset row and preset column values ​​are equal and must be positive odd numbers. Furthermore, as the values ​​of the preset row and preset column increase, the brightness enhancement effect deteriorates. Therefore, in this embodiment, the example is given with a preset row and preset column value of 1.

[0123] Step 37: Input the downsampled sample image into the student network to obtain the second hard-labeled prediction image.

[0124] As can be seen from steps 34 and 37, the student network performs two calculations for each preprocessed sample image.

[0125] For example: Reference Figure 5 The model training process shown is as follows: Figure 5 Taking the example where the first feature extraction layer includes Ta, Tb, and Tc, and the second feature extraction layer includes Sa, Sb, and Sc, with i = 1. Figure 5 It can be seen that after the preprocessed sample image DataT is input into the teacher network for one calculation, the value of F_ta output by Ta is directly assigned to F_sa of Sa. Then, a calculation is performed through Tb and Tc of the student network. After that, the downsampled sample image DataS is input into the student network, and the student network performs another calculation. That is, the teacher network calculates once, and the student network calculates twice.

[0126] Figure 5 In this example, the F_ta output by Ta is shared as the output of Sa. In other embodiments, the F_ta output by Ta can also be shared as the output of Sa, and the F_tb output by Tb can be shared as the output of Sb, so that Tb of the student network uses F_ta and Tc of the student network uses F_tb for one calculation; or, only the F_tb output by Tb can be shared as the output of Sb, so that Tc of the student network uses F_tb for one calculation. This embodiment does not limit the sharing method of intermediate feature maps.

[0127] Optionally, step 37 can be executed after step 34 or before step 34. In this case, the intermediate feature map needs to be cached for a period of time. This embodiment does not limit the execution order between step 34 and step 37.

[0128] Step 38: Based on the differences between the first hard-label predicted image and the label image, the second hard-label predicted image and the label image, and the soft-label image and the soft-label predicted image, train the student network to obtain the image augmentation model.

[0129] In one example, according to the above steps, for the same pre-processed sample image, the student network needs to perform two calculations, and in this example, the losses of the two calculations can be weighted to determine the student loss value corresponding to the student network. Specifically, based on the difference between the first hard label predicted image and the label image, the difference between the second hard label predicted image and the label image, and the difference between the soft label image and the soft label predicted image, the student network is trained to obtain an image enhancement model, including: inputting the soft label image and the soft label predicted image into a first loss function to obtain a teacher loss value; inputting the label image and the first hard label predicted image into a second loss function to obtain a first loss value; inputting the label image and the second hard label predicted image into the second loss function to obtain a second loss value; determining the weighted sum of the first loss value and the second loss value to obtain a student loss value; and updating the network parameters in the student network based on the teacher loss value and the student loss value to train the student network.

[0130] The second loss function can be L2 norm, L1 norm, or cross-entropy, and the present embodiment does not limit the implementation of the second loss function.

[0131] The weights of the first loss value and the second loss value are pre-stored in the electronic device, for example, the weight of the first loss value is 0.5, and the weight of the second loss value is 0.5, and the determination of the weighted sum of the first loss value and the second loss value to obtain the student loss value can be represented by the following formula:

[0132] Loss_FnS = Loss_s1 * 0.5 + Loss_s2 * 0.5;

[0133] Wherein, Loss_FnS represents the student loss value, Loss_s1 represents the first loss value, and Loss_s2 represents the second loss value.

[0134] In actual implementation, the weights of the first loss value and the second loss value can also be other values, and the present embodiment does not limit the values of the weights.

[0135] Optionally, based on the teacher loss value and the student loss value, the network parameters in the student network are updated, including: determining the weighted sum of the teacher loss value and the student loss value to obtain a total loss value; and performing gradient backpropagation on the student network according to the total loss value to update the student network parameters.

[0136] At this time, the total loss value can be represented by the following formula:

[0137] Loss = (1-a) * Loss_FnT + a * Loss_FnS;

[0138] Wherein, Loss represents a total loss value; Loss FnT represents a teacher loss value; Loss FnS represents a student loss value, and a represents a weight of the loss.

[0139] In another example, the student network is trained based on the loss value obtained each time. Specifically, the student network is trained based on the difference between the first hard label predicted image and the label image, the difference between the second hard label predicted image and the label image, and the difference between the soft label image and the soft label predicted image to obtain the image enhancement model, including: inputting the soft label image and the soft label predicted image into a first loss function to obtain a teacher loss value; inputting the label image and the first hard label predicted image into a second loss function to obtain a first loss value; updating network parameters in the student network based on the teacher loss value and the first loss value to train the student network; inputting the label image and the second hard label predicted image into the second loss function to obtain a second loss value; and updating the network parameters in the student network based on the teacher loss value and the second loss value to train the student network.

[0140] Wherein, the related description of updating the network parameters in the student network based on the teacher loss value and the first loss value, and updating the network parameters in the student network based on the teacher loss value and the second loss value is described in detail in the description of updating the network parameters in the student network based on the teacher loss value and the student loss value in the foregoing, and the difference is that the student loss value is replaced by the first loss value or the second loss value, which will not be described herein again.

[0141] In summary, the image enhancement method provided in the embodiment, by acquiring original image data collected by an image collection component; preprocessing the original image data to obtain preprocessed image data; inputting the preprocessed image data into an image enhancement model to obtain image data after brightness enhancement; since the larger the image size input into the student network is, the longer the student network takes to extract features, on the one hand, the image input into the student network is subjected to down-sampling processing, which can improve the calculation efficiency of the student network. On the other hand, in the traditional knowledge distillation method, the image size input into the student network and the teacher network is the same, if the down-sampled sample image is input into the teacher network, the performance of the teacher network in image enhancement will be affected. In the present application, the image input into the teacher network is not subjected to down-sampling processing, and only the down-sampled image is input into the student network, which can ensure the performance of the teacher network in image enhancement and the calculation speed of the student network, and the network complexity of the student network is lower than that of the teacher network. Then the teacher network distills the learned knowledge to the student network, so that the student network can ensure the image enhancement performance while ensuring the calculation speed. In this way, the efficiency of image enhancement can be improved and the effect of image enhancement can be ensured in the image enhancement process. At the same time, since the image size of the teacher network and the image input into the student network is inconsistent, it will cause the problem that the student network is difficult to converge in the knowledge distillation process. Based on this, in the present application, the intermediate feature map output by the teacher network is also shared to the corresponding network layer of the student network in the knowledge distillation process, so that the student network uses the intermediate feature map to accelerate the convergence and improve the training efficiency of the student network.

[0142] In addition, by training the teacher network and the student network at the same time, it is not necessary to separately pre-train the teacher network, which can improve the model training efficiency.

[0143] In addition, the student loss value is determined by the weighted sum of the first loss value and the second loss value, and the network parameters of the student network are updated based on the student loss value and the teacher loss value. Compared with updating the network parameters of the student network based on the first loss value and the teacher loss value, and the second loss value and the teacher loss value respectively, the network parameters of the student network only need to be updated once for each preprocessed sample image, which can reduce the number of times of updating the network parameters of the student network. Since the student loss value is obtained by combining the first loss value and the second loss value, the accuracy of updating the network parameters using the student loss value and the teacher loss value will not be reduced.

[0144] In addition, by extracting the region of interest in the original image data; the region of interest is subjected to down-sampling processing; on the one hand, the calculation amount of the image enhancement model can be reduced, and on the other hand, the image data of the region of interest can be ensured not to lose too much, thereby ensuring the effect of image enhancement.

[0145] In addition, by adjusting the brightness of the image based on a preset brightness adjustment parameter before the image is input to the image enhancement model, the brightness of the image input to the image enhancement model can be ensured to be substantially consistent, so that the brightness of the image output by the image enhancement model is also substantially consistent, and the stability of the model output result can be ensured.

[0146] Optionally, the image enhancement method can be used in a vehicle snapshot scene, or can also be used in other target detection scenes. Hereinafter, the image enhancement method is taken as an example for use in a vehicle snapshot scene. In this embodiment, the vehicle snapshot scene includes an image acquisition component and a light-emitting component, and the image acquisition component and the light-emitting component are both in communication with an electronic device. The electronic device can be implemented in the same device as the image component and / or the light-emitting component, or can be implemented as a mutually independent device. Figure 6 is a flowchart of a vehicle snapshot method provided by an embodiment of the present application. The method includes at least the following steps:

[0147] Step 601: Obtain original image data collected by an image acquisition component on a vehicle.

[0148] For related descriptions of this step, refer to step 101. The difference is that the working environment of step 101 is a vehicle snapshot scene, such as a toll gate or a parking lot. This embodiment will not be described here again.

[0149] Step 602: Obtain brightness-enhanced image data based on the original image data and a pre-trained image enhancement model.

[0150] In one example, the electronic device obtains the brightness-enhanced image data based on the manner in steps 102 and 103.

[0151] For example, the image acquisition component can output a Raw data of the current environment every 40 ms, i.e., 25 frames per second. After the electronic device obtains the latest Raw data, the electronic device processes the Raw data according to the manner in steps 102 and 103 to obtain the brightness-enhanced image data. Figure 1 The embodiment shown in the figure pre-processes the data, inputs the pre-processed image data into the image enhancement model, and obtains the brightness-enhanced image data.

[0152] In another example, if the original image data is compatible with the input layer of the image enhancement model, the electronic device directly inputs the original image data into the pre-trained image enhancement model to obtain the brightness-enhanced image data.

[0153] Step 603: Input the brightness-enhanced image data into a pre-trained vehicle detection network to obtain a vehicle detection result.

[0154] The vehicle detection network is established based on a target detection model and is trained using a vehicle image and a vehicle label corresponding to the vehicle image, and the vehicle detection network is suitable for detecting a vehicle in an image.

[0155] The vehicle detection result is used to indicate whether the enhanced image data includes a vehicle. When a vehicle is included, the vehicle detection result is also used to indicate the vehicle's position in the enhanced image data.

[0156] Step 604: When the vehicle detection result indicates that a vehicle has been detected, control the light-emitting component to emit light and control the image acquisition component to acquire images again to obtain a captured image of the vehicle.

[0157] Optionally, the electronic device can also perform post-processing on the captured image; post-processing refers to the process of image processing on the captured image, including but not limited to: license plate recognition, and / or image enhancement, etc. This embodiment does not limit the post-processing method.

[0158] Unlike traditional supplemental lighting solutions, in this embodiment, the light-emitting component does not emit light (or is not activated) when no vehicle is detected. It only emits light when the vehicle detection result indicates that a vehicle has been detected, and it turns off after the image acquisition component acquires the image. Overall, the light-emitting component flashes once when the vehicle detection result indicates that a vehicle has been detected. On the one hand, this can reduce the energy consumption of the light-emitting component, significantly extend its service life, and reduce light pollution caused by the light-emitting component being constantly on in low-light conditions. On the other hand, it can also ensure the vehicle capture rate in low-light environments.

[0159] To better understand the vehicle capture method proposed in this application, the following will use... Figure 5 Taking the image enhancement method shown as an example, let's illustrate this vehicle capture method with a case study. Figure 7 The method includes at least the following steps:

[0160] Step 71: Obtain the Raw data output by the image acquisition component;

[0161] Step 72: Preprocess the Raw data to obtain preprocessed image data;

[0162] Step 73: Input the preprocessed image data into the image enhancement model to obtain the brightness-enhanced image data;

[0163] Step 74: Perform vehicle detection on the enhanced image data;

[0164] Step 75: Determine whether a vehicle exists based on the vehicle detection results. If a vehicle exists, proceed to step 76; otherwise, proceed to step 72.

[0165] Step 76: Control the light-emitting component to flash once;

[0166] Step 77, acquiring the snapshot image collected in the case that the light-emitting assembly emits light;

[0167] Step 78, post-processing the snapshot image;

[0168] Optionally, the post-processing includes but is not limited to license plate recognition, image enhancement, etc. The embodiments are not limited to the manner of post-processing.

[0169] Step 79, outputting the snapshot image and the post-processing result.

[0170] For example, in the case of no light, the original image data collected by image acquisition is as shown in FIG. A, the screen is extremely dark and the vehicle information cannot be normally seen. After processing by the image enhancement model, the body brightness normal image as shown in FIG. B can be obtained. At this time, the image quality requirement of the vehicle detection algorithm can be met. After the vehicle is detected by the detection algorithm, the light-emitting assembly flashes once the low-illumination fill light, and the snapshot image as shown in FIG. C can be obtained. After post-processing of the snapshot image, the image as shown in FIG. D can be obtained. As can be seen from D, the figure screen is clear and the license plate is clearly visible, which can improve the accuracy of post-processing. Figure 8 Figure 8 Figure 8 Figure 8

[0171] In summary, the vehicle snapshot method provided in the embodiments can acquire the original image data collected by the image acquisition assembly on the vehicle; based on the original image data and the pre-trained image enhancement model, the image data with enhanced brightness is obtained; the image data with enhanced brightness is input into the pre-trained vehicle detection network to obtain the vehicle detection result; in the case that the vehicle detection result indicates that a vehicle is detected, the light-emitting assembly is controlled to emit light and the image acquisition assembly is controlled to collect images again to obtain the snapshot image of the vehicle; the problem that the calculation efficiency of the image enhancement network is low when the image enhancement network with high complexity is trained by using training data can be solved; the student network after training is used as the image enhancement network, which can improve the calculation efficiency of the image enhancement network without affecting the image enhancement performance. At the same time, the light-emitting assembly flashes once in the case that the image data with enhanced brightness indicates that a vehicle exists, which can reduce the energy consumption of the light-emitting assembly, greatly prolong the service life of the light-emitting assembly, and avoid light pollution caused by the constant light of the light-emitting assembly; on the other hand, the snapshot rate of the vehicle in the dark environment can also be ensured.

[0172] Figure 9 FIG. 1 is a block diagram of an image enhancement device provided in an embodiment of the present application. The device includes at least the following modules: a data acquisition module 910, a preprocessing module 920, and an image enhancement module 930.

[0173] ​​​​The data acquisition module 910 is configured to acquire original image data collected by the image collection component.

[0174] The preprocessing module 920 is configured to pre-process the original image data to obtain pre-processed image data, which is adapted to an input layer of a pre-trained image enhancement model.

[0175] The image enhancement module 930 is configured to input the pre-processed image data into the image enhancement model to obtain brightness-enhanced image data. The image enhancement model is obtained by distilling knowledge of a teacher network to a student network. In the knowledge distillation process, the image input into the student network is obtained by down-sampling an image input into the teacher network, and an intermediate feature map output by the teacher network is shared to a corresponding network layer of the student network. The student network is trained using the intermediate feature map and the down-sampled sample image. The network complexity of the student network is lower than that of the teacher network.

[0176] For related details, refer to the above method embodiments.

[0177] It should be noted that the image enhancement device based on the image enhancement model provided in the above embodiments is only used as an example for the division of the above functional modules when performing image enhancement based on the image enhancement model. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the image enhancement device based on the image enhancement model is divided into different functional modules to complete all or part of the functions described above. In addition, the image enhancement device based on the image enhancement model provided in the above embodiments and the image enhancement method based on the image enhancement model embodiment belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0178] Figure 10 is a block diagram of a vehicle snapshot device provided in an embodiment of the present application. The device includes at least the following modules: a data acquisition module 1010, an image enhancement module 1020, a vehicle detection module 1030, and a vehicle snapshot module 1040.

[0179] The data acquisition module 1010 is configured to acquire original image data collected by the image collection component on the vehicle.

[0180] The image enhancement module 1020 is configured to obtain image data after brightness enhancement based on the original image data and a pre-trained image enhancement model. The image enhancement model is obtained by distilling knowledge of a teacher network to a student network. In the knowledge distillation process, an image input to the student network is obtained by down-sampling an image input to the teacher network, and an intermediate feature map output by the teacher network is shared to a corresponding network layer of the student network. The student network is trained using the intermediate feature map and the down-sampled sample image. The network complexity of the student network is lower than that of the teacher network.

[0181] The vehicle detection module 1030 is configured to input the image data after brightness enhancement into a pre-trained vehicle detection network to obtain a vehicle detection result.

[0182] The vehicle snapshot module 1040 is configured to, in a case where the vehicle detection result indicates that a vehicle is detected, control a light-emitting component to emit light and control the image acquisition component to perform image acquisition again to obtain a snapshot image of the vehicle.

[0183] For related details, refer to the above method embodiments.

[0184] It should be noted that, in the vehicle snapshot device provided in the above embodiments, only the division of the above functional modules is used for example description, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the vehicle snapshot device is divided into different functional modules to complete all or part of the above-described functions. In addition, the vehicle snapshot device and the vehicle snapshot method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0185] Figure 11 FIG. 1 is a block diagram of an electronic device according to an embodiment of the present application. The device includes at least a processor 1101 and a memory 1102.

[0186] The processor 1101 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1101 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 1101 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also known as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 1101 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 1101 can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0187] The memory 1102 can include one or more computer-readable storage media that can be non-transitory. The memory 1102 can also include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one instruction for being executed by the processor 1101 to implement the image enhancement or vehicle snapshot method provided by the method embodiments in the present application.

[0188] In some embodiments, the electronic device can also optionally include a peripheral device interface and at least one peripheral device. The processor 1101, the memory 1102, and the peripheral device interface can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface through a bus, a signal line, or a circuit board. Illustratively, the peripheral devices include, but are not limited to, radio frequency circuitry, a touch display screen, audio circuitry, and a power supply, etc.

[0189] Of course, the electronic device can also include fewer or more components, which are not limited in the present embodiment.

[0190] Optionally, the present application also provides a computer-readable storage medium, the computer-readable storage medium storing a program, the program being loaded and executed by a processor to implement the image enhancement or vehicle snapshot method of the above-mentioned method embodiments.

[0191] Optionally, the present application also provides a computer product, which comprises a computer readable storage medium, and the computer readable storage medium stores a program, and the program is loaded and executed by a processor to implement the image enhancement or vehicle snapshot method of the above-mentioned method embodiment.

[0192] The technical features of the above-mentioned embodiments can be combined in any manner. In order to make the description simple, all possible combinations of the technical features in the above-mentioned embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

[0193] The above-mentioned embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An image enhancement method characterized by, The method comprises: acquiring original image data collected by an image collection component; preprocessing the original image data to obtain preprocessed image data, which is adapted to an input layer of an image enhancement model; inputting the preprocessed image data into the image enhancement model to obtain brightness-enhanced image data, the image enhancement model being obtained by distilling knowledge of a teacher network to a student network, in a knowledge distillation process, an image input into the student network being obtained by down-sampling an image input into the teacher network, and an intermediate feature map output by the teacher network being shared to a corresponding network layer of the student network; the student network being trained using the intermediate feature map and a down-sampled sample image, the network complexity of the student network being lower than that of the teacher network; the training process of the image enhancement model comprising: acquiring training data, the training data comprising the sample image and a label image corresponding to the sample image, the brightness of the label image being greater than that of the sample image; preprocessing the sample image to make the preprocessed sample image meet the input requirement of the teacher network; inputting the preprocessed sample image into the teacher network to obtain a soft label image and an intermediate feature map; sharing the intermediate feature map to the corresponding network layer of the student network to obtain a soft label prediction image and a first hard label prediction image output by the student network; training the teacher network based on the difference between the soft label image and the soft label prediction image; down-sampling the preprocessed sample image to obtain a down-sampled sample image; inputting the down-sampled sample image into the student network to obtain a second hard label prediction image; training the student network based on the difference between the first hard label prediction image and the label image, the difference between the second hard label prediction image and the label image, and the difference between the soft label image and the soft label prediction image, to obtain the image enhancement model.

2. The method of claim 1, wherein, The teacher network comprises at least two layers of first feature extraction layers and an output layer connected in sequence, and the student network comprises at least two layers of second feature extraction layers and an output layer connected in sequence; the number of the second feature extraction layers is equal to that of the first feature extraction layers, the model complexity of the second feature extraction layers is lower than that of the first feature extraction layers, and the dimension of a feature map output by each layer of the first feature extraction layers is the same as that of a feature map output by a corresponding layer of the second feature extraction layers; the inputting of the preprocessed sample image into the teacher network to obtain the soft label image and the intermediate feature map comprises: inputting the preprocessed sample image into the teacher network to obtain an intermediate feature map output by an i-th layer of the first feature extraction layers of the teacher network and a soft label image output by the output layer of the teacher network; wherein, the i is a positive integer; Correspondingly, the sharing of the intermediate feature map to the corresponding network layer of the student network to obtain the soft label prediction image and the first hard label prediction image output by the student network comprises: The intermediate feature map is taken as the output result of the i-th layer second feature extraction layer in the student network to input the (i+1)-th layer second feature extraction layer to obtain the soft label prediction image and the first hard label prediction image.

3. The method of claim 1, wherein, The training of the student network based on the difference between the first hard label prediction image and the label image, the difference between the second hard label prediction image and the label image, and the difference between the soft label image and the soft label prediction image to obtain the image enhancement model comprises: The soft label image and the soft label prediction image are input into a first loss function to obtain a teacher loss value; The label image and the first hard label prediction image are input into a second loss function to obtain a first loss value; The label image and the second hard label prediction image are input into the second loss function to obtain a second loss value; The weighted sum of the first loss value and the second loss value is determined to obtain a student loss value; Based on the teacher loss value and the student loss value, the network parameters in the student network are updated to train the student network.

4. The method of claim 1, wherein, The sample image is in Bayer format, and the preprocessing of the sample image comprises: Extracting a region of interest in the sample image data; Channel separation is performed on the region of interest to obtain a separated sample image; Based on a pre-set brightness adjustment parameter, the brightness of the separated sample image is adjusted to obtain the pre-processed sample image.

5. The method of claim 1, wherein, The preprocessing of the original image data comprises: Extracting a region of interest in the original image data; Down-sampling processing is performed on the region of interest; Channel separation is performed on the down-sampled image data; Based on a pre-set brightness adjustment parameter, the brightness of the channel-separated image data is adjusted to obtain the pre-processed image data.

6. The method of claim 5, wherein, The original image data comprises a plurality of groups of Raw data stored in Bayer format, and the down-sampling processing of the region of interest comprises: Every n rows and n columns of Raw data are extracted to obtain a plurality of groups of Raw data; the n is a positive integer.

7. A vehicle snapshot method, characterized by, The method comprises: Obtaining original image data collected by an image collection component on a vehicle; Based on the original image data and a pre-trained image enhancement model, obtaining brightness-enhanced image data; the image enhancement model is obtained by distilling the knowledge of a teacher network to a student network, the image input into the student network in the knowledge distillation process is obtained by down-sampling an image input into the teacher network, and in the knowledge distillation process, an intermediate feature map output by the teacher network is also shared to the corresponding network layer of the student network; the student network is trained using the intermediate feature map and the down-sampled sample image, and the network complexity of the student network is lower than that of the teacher network; The training process of the image enhancement model comprises: obtaining training data, the training data comprising the sample image and a label image corresponding to the sample image, the label image having a brightness greater than that of the sample image; pre-processing the sample image to make the pre-processed sample image meet the input requirement of a teacher network; inputting the pre-processed sample image into the teacher network to obtain a soft label image and an intermediate feature map; sharing the intermediate feature map to a network layer corresponding to a student network to obtain a soft label prediction image and a first hard label prediction image output by the student network; training the teacher network based on the difference between the soft label image and the soft label prediction image; down-sampling the pre-processed sample image to obtain a down-sampled sample image; inputting the down-sampled sample image into the student network to obtain a second hard label prediction image; training the student network based on the difference between the first hard label prediction image and the label image, the difference between the second hard label prediction image and the label image, and the difference between the soft label image and the soft label prediction image, to obtain the image enhancement model; inputting the brightness-enhanced image data into a pre-trained vehicle detection network to obtain a vehicle detection result; in a case where the vehicle detection result indicates that a vehicle is detected, controlling a light-emitting component to emit light and controlling the image acquisition component to acquire an image again to obtain a snapshot image of the vehicle.

8. An electronic device, comprising: The device comprises a processor and a memory; the memory stores a program, the program is loaded and executed by the processor to implement the image enhancement method according to any one of claims 1 to 6, or implement the vehicle snapshot method according to claim 7.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program, the program is executed by the processor to implement the image enhancement method according to any one of claims 1 to 6, or implement the vehicle snapshot method according to claim 7.

Images