Vehicle panoramic view illumination enhancement method based on generative adversarial network

Abstract

The invention belongs to the technical field of vehicle image processing, and relates to a vehicle panoramic view illumination enhancement method based on a generative adversarial network. The method comprises the following steps of: inputting a vehicle panoramic surround view with a poor illumination condition into an illumination enhancement generative adversarial network trained by taking a vehicle panoramic surround view with good illumination and a low-illumination vehicle panoramic surround view with randomly reduced brightness as a training data set, and generating an illumination enhancement image; wherein the illumination enhancement generative adversarial network comprises a generator adopting an attention-guided U-Net network structure and a discriminator adopting a global-local dual-scale discrimination structure, and the loss function is composed of adversarial loss, perception loss and color loss; according to the invention, the problems of poor definition, low contrast and the like of the panoramic view of the vehicle at night or in other low-light scenes are solved, so that an intelligent driving system can be supported to realize accurate target identification and detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle panoramic surround image processing, and particularly relates to a method for enhancing the illumination of a vehicle panoramic surround view based on a generative adversarial network. Background Art

[0002] With the rapid development of vehicle surround view systems and intelligent driving technologies, panoramic surround views play an increasingly important role in vehicle images. Through the panoramic surround view, the environment around the vehicle and the positional relationship with other objects can be clearly seen, facilitating the driver or intelligent driving system to make judgments and decisions.

[0003] The application of panoramic surround views in vehicles has been around for some time, and the generation technology of vehicle panoramic surround views has become relatively mature. The currently commonly used method for generating vehicle panoramic surround views is to collect panoramic images covering 360° around the vehicle through fisheye cameras deployed around the vehicle, perform distortion correction and perspective transformation on the images to form a bird's-eye view of the local vehicle, and then perform image fusion on the overlapping parts in the local bird's-eye view, and then complete image stitching to present a complete vehicle panoramic surround view.

[0004] However, in the dark at night or in an underground parking lot with poor lighting, the vehicle panoramic surround view often presents images with low clarity and low contrast due to low light intensity, making it difficult for the driver to clearly see important information such as surrounding markings, or even completely unable to recognize them. There is inevitably strong distortion of objects during the synthesis process of the panoramic surround view, which also exacerbates the recognition difficulty of the panoramic surround view under low light to a certain extent, interfering with the driver's observation and judgment. With the gradual arrival of the intelligent driving era, the intelligent driving system needs to identify targets such as markings in the environment through target detection methods, which also puts forward higher requirements for the presentation quality of the vehicle panoramic surround view under low light. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method for enhancing the illumination of a vehicle panoramic surround view based on a generative adversarial network.

[0006] In order to achieve the purpose of the present invention, the present invention is implemented by adopting the following technical solutions.

[0007] A method for enhancing the illumination of a vehicle panoramic surround view based on a generative adversarial network is to input a vehicle panoramic surround view with poor illumination conditions into an illumination enhancement generative adversarial network to generate an illumination-enhanced vehicle panoramic surround view; the illumination enhancement generative adversarial network is formed by multi-round adversarial training using well-illuminated vehicle panoramic surround views and their randomly dimmed low-illumination vehicle panoramic surround views as training data sets; the illumination enhancement generative adversarial network consists of a generator and a discriminator, and the loss function is composed of an adversarial loss, a perceptual loss, and a color loss; where:

[0008] The generator has an attention-guided U-Net network structure. By learning the statistical characteristics of the real data provided by the training data set, it can generate samples similar to the real data from the input random noise vector to deceive the discriminator;

[0009] The discriminator has a global-local dual-scale discrimination structure, which is used to discriminate the input samples and output a probability value representing the likelihood that the input samples are real samples to distinguish real samples from the samples generated by the generator;

[0010] The loss function is: LOSS = L GAN + L per + L color ; where, L GAN is the adversarial loss, including the global and local losses of the generator and the global and local losses of the discriminator; L per is the perceptual loss, which is used to represent the distance between the features of the illumination-enhanced image and the features of the well-illuminated image; L color is the color loss, which is used to represent the difference between the illumination-enhanced image and the well-illuminated image on each channel.

[0011] As a preferred solution of the present invention, the U-Net network structure is composed of a downsampling part, an attention map, an efficient channel attention (ECA) module, and an upsampling part. The downsampling part is composed of N 3×3 convolutional layers and N - 1 max pooling layers, and a max pooling layer is arranged between every two adjacent 3×3 convolutional layers for feature extraction. After being processed by the efficient channel attention (ECA) module, the feature representation ability is further improved and then multiplied by the attention map adjusted to an appropriate size; the upsampling part is composed of N groups of combined layers replacing the transposed convolutional layers, and each group of combined layers is composed of a bilinear upsampling layer and a convolutional layer arranged in sequence.

[0012] As a preferred solution of the present invention, the activation function used in each layer of the U-Net network is the LeakyReLu function.

[0013] As a preferred embodiment of the present invention, the attention map normalizes the brightness of each pixel on the input image, that is, represents its brightness as a value in the range of 0 to 1, and uses 1 minus the value obtained after normalization to obtain a grayscale image of the original image.

[0014] As a preferred embodiment of the present invention, the efficient channel attention (ECA) module has steps such as global average pooling (GAP), one-dimensional convolution, and Sigmoid activation function. It adaptively captures the dependencies between channels through a simple and efficient one-dimensional convolution, avoiding the complex multi-layer perceptron (MLP) structure in traditional attention mechanisms, reducing network complexity and computational burden, and further enhancing the network's feature representation ability.

[0015] As a preferred embodiment of the present invention, the adversarial loss L GAN and the perceptual loss L per and the color loss L color are respectively expressed as:

[0016]

[0017] where are respectively the global and local losses of the generator and the global and local losses of the discriminator; x and y are respectively the well-lit image and the low-light image, and G(y) is the image with enhanced illumination; φ i,j represents the value of the pixel at (i, j) on the image; W and H are respectively the width and height of the image; C is the number of channels, C = 3; c is the c-th channel of the image; φ i,j,c represents the value of the pixel at (i, j) on the image in the c-th channel.

[0018] As a preferred embodiment of the present invention, the global-local dual-scale discriminant structure makes local true / false judgments on M corresponding image patches randomly cropped from the image with enhanced illumination and its corresponding well-lit image, and uses the true / false judgments in the local area and the true / false judgment of the whole image as the basis for the final true / false judgment of the image simultaneously, so as to avoid the situation of local overexposure or underexposure in the optimized image.

[0019] As a preferred embodiment of the present invention, the well-lit vehicle panoramic surround view is obtained by fish-eye cameras deployed around the vehicle collecting 360° panoramic images covering the vehicle's surroundings under sufficient light and clear ground markings. The images are subjected to distortion correction and perspective transformation to form a bird's-eye view of the vehicle's local area, and the overlapping parts in the local bird's-eye view are subjected to image fusion, and then image stitching is completed to form the vehicle panoramic surround view.

[0020] As a preferred embodiment of the present invention, the low-light vehicle panoramic surround view is obtained by reducing the brightness of the vehicle panoramic surround view with good lighting conditions, so as to obtain a low-light vehicle panoramic surround view corresponding to the vehicle panoramic surround view with good lighting conditions; wherein, the parameters of the brightness reduction are randomly selected. Figure 1 A corresponding low-light vehicle panoramic surround view; wherein, the parameters of the brightness reduction are randomly selected.

[0021] As a preferred embodiment of the present invention, the specific process of the adversarial training includes the following steps:

[0022] S101: Use any set of corresponding images from the vehicle panoramic surround view dataset with good lighting conditions and the low-light vehicle panoramic surround view dataset that correspond one by one as the current training set;

[0023] S102: Input the low-light panoramic surround view in the current training set into the generator to obtain a vehicle panoramic surround view with enhanced lighting;

[0024] S103: Input the vehicle panoramic surround view with enhanced lighting and the vehicle panoramic surround view with good lighting in the current training set into the discriminator for discrimination, so that the generator and the discriminator perform adversarial training.

[0025] As a preferred embodiment of the present invention, the end condition of the adversarial training is until the balance point is reached when the generator can generate realistic samples and the correct discrimination probability of the discriminator is 0.5, then the training ends.

[0026] Beneficial effects

[0027] The present invention provides a method for enhancing the lighting of a vehicle panoramic surround view based on a generative adversarial network. By making a vehicle panoramic surround view with good lighting and randomly reducing its brightness, a corresponding training dataset is formed and used as a training group in pairs for generative adversarial training, which has the advantages of easy acquisition of paired training datasets and guaranteed training effects; by performing supervised learning for lighting enhancement on a specific image of a vehicle panoramic surround view, more accurate learning can be carried out according to the image features of the vehicle panoramic surround view. Compared with other general unsupervised methods, the effect of generating a vehicle panoramic surround view with enhanced lighting is better and the image effect is more natural; using the trained lighting enhancement generative adversarial network can process a vehicle panoramic surround view with poor lighting conditions into an optimized image with the presentation effect of a vehicle panoramic surround view with good lighting, thereby solving the problems of poor clarity and low contrast of the vehicle panoramic surround view at night or in other low-light scenarios, and further supporting the intelligent driving system to achieve accurate target recognition and detection. Description of the drawings

[0028] Figure 1 It is a flowchart of the steps for generating a vehicle panoramic surround view in the present invention;

[0029] Figure 2 Schematic diagram of the layout of the fisheye camera and its field of view coverage in the present invention;

[0030] Figure 3 Example diagram of the image collected by the fisheye camera in the present invention;

[0031] Figure 4 Example diagram of the distortion correction of the fisheye image in the present invention;

[0032] Figure 5 Example diagram of the original bird's-eye view of the vehicle after perspective transformation and image stitching in the present invention;

[0033] Figure 6 Schematic diagram of the structural principle of the efficient channel attention (ECA) module in the present invention;

[0034] Figure 7 Network structure of the generator of the light enhancement generative adversarial network in the present invention;

[0035] Figure 8 Principle and structure of the discriminator of the light enhancement generative adversarial network in the present invention;

[0036] Figure 9 Example diagram of the effect of light enhancement on low-light images in the present invention;

[0037] Figure 10 Research flow chart of the light enhancement method in the present invention. Detailed implementation manners

[0038] The present invention will be further described in conjunction with the embodiments and the accompanying drawings.

[0039] It should be noted in this embodiment that the current technology for generating a panoramic view of a vehicle is quite mature. As Figure 1 shown, after the steps of collecting images by a fisheye camera, distortion correction of the fisheye image, perspective transformation of the corrected image, and image fusion of the overlapping parts, a panoramic view of the vehicle in the prior art is obtained.

[0040] The fisheye camera has an ultra-wide-angle lens with a short focal length and a large viewing angle, and its viewing angle range can reach 180°. Deploying the fisheye camera in the front, rear, left, and right of the vehicle, as Figure 2 shown, an environmental image of 360° around the vehicle can be obtained. As Figure 3 shown, while the image taken by the fisheye camera presents a super-large viewing angle, a large radial distortion also appears in the picture, causing serious deformation of the objects in the picture.

[0041] The conversion relationship between the world coordinate system and the camera coordinate system in the fisheye lens is: Among them, the world coordinate system is (X W , Y W , Z W ), the camera coordinate system is (X C , Y C , Z C ), the pixel coordinate system is (u, v), is the internal parameter matrix, where the units of u0 and v0 are both pixels. By calibrating the fish-eye lens, the internal parameter matrix can be obtained. Using the camera parameters to solve the distortion parameters, the fish-eye image can be corrected for distortion. As Figure 4 shown, the barrel distortion in the radial direction has been basically eliminated in the corrected image.

[0042] Perform a perspective transformation on the corrected image. Perspective transformation is a process in which the original image forms a new image after operations such as translation, rotation, scaling, and shearing, so as to simulate the visual effect of a human eye or a lens viewing a three-dimensional space object. Through this process, the corrected image can be transformed into a bird's-eye view. The transformation relationship of perspective transformation can be expressed as: [x'y'z'] = [uv 1 where u and v are the horizontal and vertical coordinates in the corrected image, is the perspective matrix. The horizontal and vertical coordinates x and y of the transformed bird's-eye view can be calculated by the following formula: To solve the perspective matrix, the horizontal and vertical coordinates of at least four points are required. Therefore, in the embodiment of the present invention, with the assistance of the calibration cloth checkerboard, four points are selected in the corrected image, and the perspective matrix of the perspective transformation is calculated through the pixel coordinates of these points in the corrected image and the pixel coordinates in the bird's-eye view, so as to transform the corrected image into a bird's-eye view of the vehicle local area.

[0043] Stitch the bird's-eye views transformed from the four fish-eye images to obtain the vehicle panoramic surround view. It should be noted that the overlapping part between adjacent two images needs to be processed by the method of weighted mapping to obtain a smooth overlapping part image. There are various methods of weighted mapping. In the embodiment of the present invention, the pixels are weighted and mapped according to the distance, that is, for the pixels in the overlapping part, the closer to one side image, the greater the weight of this side image on this pixel. The calculation formula of the weight is: where d A , d B refer to the distances from the pixel to the two boundaries (A, B) of the overlapping part. As Figure 5 shown, the completed vehicle panoramic surround view can be obtained.

[0044] Make a vehicle panoramic surround view dataset with good lighting conditions according to the above method. Good lighting conditions mean that the image content is clearly visible, with high contrast and easy to distinguish. The dataset used in the embodiments of the present invention contains 4,000 vehicle panoramic surround views with good lighting conditions. The image content covers various road markings such as lane lines, parking space lines, and zebra crossings, and includes outdoor natural lighting scenes and underground parking lot scenes, ensuring the richness of training samples.

[0045] Perform random brightness reduction on the above vehicle panoramic surround view dataset to make a vehicle panoramic surround view dataset with reduced brightness. The contrast of the image with reduced brightness will decrease, and it will be more difficult to identify image information and distinguish road markings, which can well simulate the image performance of the same scene under low light levels. In the embodiments of the present invention, the following brightness adjustment formula is used for random brightness reduction: I out = k × (I in ) α , where I in is the brightness of the input vehicle panoramic surround view with good lighting conditions, and I out is the brightness of the vehicle panoramic surround view after brightness reduction; k is the channel coefficient, randomly taking values in the range of 0.95 to 1.05; α is the brightness adjustment index, randomly taking values in the range of 0.6 to 0.8. This method not only effectively reduces the image brightness and well simulates the image performance of the same scene under low light levels, but also makes the reduced brightness random, ensuring the generalization ability of the training model to handle different degrees of low light scenes.

[0046] It should be noted in this embodiment that the generative adversarial network used in the present invention is a deep learning model. Its core idea is to perform adversarial training through two neural networks, a generator and a discriminator, so as to realize the generation and simulation of complex data. The goal of the generator is to generate as realistic samples as possible to deceive the discriminator. It generates samples similar to the real data by learning the statistical characteristics of the real data from the input random noise vector; while the goal of the discriminator is to distinguish between real samples and samples generated by the generator as accurately as possible. It discriminates the input samples and outputs a probability value to represent the possibility that this sample is a real sample. After multiple rounds of training, the performance of the generator and the discriminator continues to improve until the balance point where the generator can generate highly realistic samples and the correct discrimination probability of the discriminator is 0.5, then the training ends.

[0047] Construct the generator neural network of the light enhancement generative adversarial network, use U-Net as the backbone network of the generator, and add an attention mechanism to the generator to improve the enhancement effect. The reason for adding the attention mechanism is that light enhancement is not simply to globally and uniformly increase the brightness of the image. The increased brightness should be negatively correlated with the brightness of each region of the image itself, that is, the brighter the region in the original image, the lower the degree of brightness increase; the darker the region in the original image, the higher the degree of brightness increase. Therefore, the generator needs to pay more attention to the darker regions of the image. In the embodiment of the present invention, the brightness of each pixel in the image is normalized, that is, its brightness is represented as a value within the range of 0 to 1, and subtracting the obtained value after normalization from 1 can obtain a grayscale image of the original image, which is called the attention map. In the embodiment of the present invention, the efficient channel attention mechanism (ECA) is used to further improve the feature representation ability of the network. As Figure 6 shown, first perform global average pooling (GAP) on the feature map with parameters C×H×W, compress the spatial dimension (H×W) of each channel into an average value, and the formula is as follows:

[0048]

[0049] In the formula, c i,j is the pixel value of (i, j) on channel c. Then, perform 1D convolution on its C×1×1 output, where the convolution kernel size is adaptively selected according to the number of input channels, and the calculation method is as follows:

[0050]

[0051] In the formula, |·| odd refers to taking the closest odd number of the result. Then, add the Sigmoid activation function to the result of the convolution to limit each value between 0 and 1 to obtain the channel attention weight. Finally, multiply the channel attention weight by the elements of the original input feature map to obtain the weighted feature map. The ECA module adaptively captures the dependencies between channels through a simple and efficient one-dimensional convolution, avoids the complex multi-layer perceptron (MLP) structure in the traditional attention mechanism, reduces the network complexity and computational burden, and can further improve the feature representation ability of the network.

[0052] As Figure 7As shown in the figure, the U-Net network structure mainly consists of a convolutional layer, a max-pooling layer, an upsampling layer, and an attention module. Among them, the downsampling part uses multiple 3×3 convolutional layers and max-pooling layers for feature extraction. After that, it is processed by an Efficient Channel Attention (ECA) module and multiplied element-wise with an attention map adjusted to an appropriate size. The upsampling part uses a combination of a bilinear upsampling layer and a convolutional layer to replace the transposed convolutional layer, which can achieve better performance. In the embodiments of the present invention, the activation function used in each layer of the network is the LeakyReLu function. The processed data is multiplied element-wise with the attention map and added pixel-wise to the original low-light image to obtain the optimized light-enhanced image.

[0053] Construct the discriminator neural network of the light-enhanced generative adversarial network, as Figure 8 shown, a global-local dual-scale discriminant structure is used for the discrimination work of the generative adversarial network. The reason for using this structure is that when there are a few bright regions in the overall low-light image, it is difficult for the global discriminator to distinguish the true and false of these regions, which will cause the optimized image to look not very real in some regions, resulting in local overexposure or underexposure, thus affecting the overall optimization effect of the image. By introducing a local discriminator, several image patches randomly cropped from the image will be discriminated. The global discriminator and the local discriminator use a Markov discriminator to distinguish the true and false. After multiple rounds of adversarial training, it can be ensured that the generated light-enhanced image is quite realistic both globally and locally. In the embodiments of the present invention, the optimized light-enhanced image and the original well-lit image are discriminated globally and locally, and the discrimination results are fed back to the generator. Then, the generator and the discriminator are optimized according to the loss function.

[0054] Construct the loss function of the light-enhanced generative adversarial network. In the present invention, the loss function of the light-enhanced generative adversarial network is: LOSS = L GAN + L per + L color . Among them, L GAN is the adversarial loss, including the global and local losses of the generator and the global and local losses of the discriminator; L per is the perceptual loss, which can be used to represent the distance between the features of the light-enhanced image and the features of the well-lit image; L color is the color loss, which can be used to represent the difference between the light-enhanced image and the well-lit image on each channel. The following is the specific representation of the loss function in the embodiments of the present invention.

[0055] Adversarial loss:

[0056]

[0057] Among them, They are the global and local losses of the generator and the global and local losses of the discriminator respectively.

[0058] Perceptual loss:

[0059]

[0060] Among them, x and y are the well-lit image and the low-light image respectively, G(y) is the light-enhanced image; φ i,j represents the value of the pixel at (i, j) on the image; W and H are the width and height of the image respectively.

[0061] Color loss:

[0062]

[0063] Among them, C is the number of channels, C = 3; c is the c-th channel of the image; φ i,j,c represents the value of the pixel at (i, j) on the c-th channel of the image.

[0064] Use the trained light-enhanced generative adversarial network to enhance the panoramic surround view of the vehicle with poor lighting conditions, and obtain the light-enhanced panoramic surround view of the vehicle.

[0065] In summary, the embodiment of the present invention provides a method for enhancing the lighting of the panoramic surround view of a vehicle based on a generative adversarial network. First, according to the sequence process of collecting images by a fisheye camera, correcting the distortion of the fisheye image, performing perspective transformation on the corrected image, and stitching the overlapping part of the images, a panoramic surround view of the vehicle is made; a dataset of panoramic surround views of the vehicle with good lighting conditions is made, and it is randomly brightness-reduced according to the adopted method, thus completing the preparation of the dataset required for training.

[0066] Secondly, construct the generator and discriminator networks of the light-enhanced generative adversarial network. The generator uses an attention-guided U-Net as the backbone network, which can pay more attention to the darker areas on the image, thereby improving the light-enhancement effect; the discriminator uses a global-local dual-scale discriminant structure, and takes the true / false determination of the image in the local area and the true / false determination of the whole image as the basis for the final true / false determination of the image at the same time, avoiding the situation of local overexposure or underexposure in the optimized image.

[0067] Finally, set the loss function of the light-enhanced generative adversarial network. This loss function is composed of three parts: adversarial loss, perceptual loss, and color loss. Among them, the adversarial loss includes the global and local losses of the generator and the global and local losses of the discriminator; this loss function can not only well meet the training optimization of the generative adversarial network, but also the perceptual loss and color loss play a very important role in improving the effect of the optimized image.

[0068] Such asFigure 9 As shown, it is the optimization effect of the vehicle panoramic view after light enhancement by the method of the present invention. The results show that this method has a significant effect on light enhancement of the vehicle panoramic view with poor lighting, and can make the originally less contrast and difficult-to-identify markings and other image details clearly visible, and there is no overexposure or underexposure phenomenon after light enhancement. As Figure 10 shown, the research process of this method is given.

[0069] The preferred embodiments of the embodiments of the present invention have been described above with reference to the accompanying drawings, and thus do not limit the scope of rights of the embodiments of the present invention. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present invention shall be within the scope of rights of the embodiments of the present invention.

Claims

1. A vehicle panoramic surround view illumination enhancement method based on a generative adversarial network, characterized by: The method is to input the vehicle panoramic surround view under poor lighting conditions into the lighting enhancement generative adversarial network to generate a lighting enhanced vehicle panoramic surround view; The illumination enhancement generative adversarial network is formed by performing multiple rounds of adversarial training using the panoramic view of a vehicle with good illumination and the panoramic view of a vehicle with low illumination with randomly reduced brightness as training data sets; the illumination enhancement generative adversarial network is composed of two parts, a generator and a discriminator, and the loss function is composed of adversarial loss, perceptual loss and color loss; wherein: The generator has an attention-guided U-Net network structure, and by learning the statistical characteristics of the real data provided by the training data set, it can generate samples similar to the real data from the input random noise vector to deceive the discriminator; The discriminator has a global-local dual-scale discriminant structure, which is used to discriminate input samples and output a probability value indicating the possibility of the input sample being a real sample, so as to distinguish the real sample from the sample generated by the generator; The loss function is: LOSS = L GAN +L per +L color Among them, L GAN is the adversarial loss, including the global and local losses of the generator and the global and local losses of the discriminator; L per is the perceptual loss, which is used to represent the distance between the features of the illumination-enhanced image and the features of the well-illuminated image; L color is the color loss, which is used to represent the difference between the lighting-enhanced image and the well-lit image on each channel.

2. The method for enhancing illumination of a vehicle panoramic surround view based on a generative adversarial network according to claim 1, characterized in that: The U-Net network structure is composed of a downsampling part, an attention map, an efficient channel attention (ECA) module, and an upsampling part. The downsampling part is composed of N 3×3 convolutional layers and N-1 maximum pooling layers, and a maximum pooling layer is arranged between each two adjacent 3×3 convolutional layers for extracting features. After being processed by the efficient channel attention (ECA) module, the feature representation capability is further improved and then multiplied with the attention map adjusted to an appropriate size; the upsampling part is composed of N groups of combination layers replacing the deconvolution layer, and each group of combination layers is composed of bilinear upsampling layers and convolutional layers arranged in sequence; the activation function used in each layer of the U-Net network is the LeakyReLu function.

3. The method for enhancing illumination of a vehicle panoramic surround view based on a generative adversarial network according to claim 2, characterized in that: The attention map normalizes the brightness of each pixel on the input image, that is, represents its brightness as a value in the range of 0 to 1, and uses 1 to subtract the normalized value to obtain a grayscale image of the original image.

4. The method for enhancing illumination of a vehicle panoramic surround view based on a generative adversarial network according to claim 2, characterized in that: The efficient channel attention (ECA) module has steps such as global average pooling (GAP), one-dimensional convolution, and Sigmoid activation function. It adaptively captures the dependency between channels through a simple and efficient one-dimensional convolution, avoids the complex multi-layer perceptron (MLP) structure in the traditional attention mechanism, reduces network complexity and computational burden, and can further improve the feature representation capability of the network.

5. The method for enhancing illumination of a vehicle panoramic surround view based on a generative adversarial network according to claim 1, characterized in that: The adversarial loss L GAN , Perceptual loss L per and color loss L color Respectively expressed as: in, are the global and local losses of the generator and the global and local losses of the discriminator respectively; x, y are the images with good lighting and low lighting respectively, G(y) is the lighting enhanced image; φ i,j represents the value of the pixel (i, j) on the image; W, H are the width and height of the image respectively; C is the number of channels, C = 3; c is the cth channel of the image; φ i,j,c Represents the value of the pixel (i, j) on the c channel of the image.

6. The method for enhancing illumination of a vehicle panoramic surround view based on a generative adversarial network according to claim 1, characterized in that: The global-local dual-scale discrimination structure performs local true-false judgment on M corresponding image blocks randomly cropped from the illumination-enhanced image and its corresponding well-illuminated image, and uses the true-false judgment of the image in the local area and the true-false judgment of the image as a whole as the basis for the final true-false judgment of the image, so as to avoid local overexposure or underexposure in the optimized image.

7. The method for enhancing illumination of a vehicle panoramic surround view based on a generative adversarial network according to claim 1, characterized in that: The well-illuminated vehicle panoramic surround view is obtained by collecting 360° panoramic images around the vehicle by fisheye cameras deployed around the vehicle under sufficient light and clear ground markings. The images are subjected to distortion correction and perspective transformation to form a local bird's-eye view of the vehicle, and image fusion is performed on the overlapping parts of the local bird's-eye view to complete image stitching to form a vehicle panoramic surround view.

8. The method for enhancing illumination of a vehicle panoramic surround view based on a generative adversarial network according to claim 1, characterized in that: The low-light vehicle panoramic surround view is obtained by reducing the brightness of the vehicle panoramic surround view under good lighting conditions, so as to obtain a low-light vehicle panoramic surround view that corresponds one-to-one to the vehicle panoramic surround view under good lighting conditions; wherein the brightness reduction parameters are randomly selected.

9. The method for enhancing illumination of a vehicle panoramic surround view based on a generative adversarial network according to claim 1, characterized in that: The specific process of the adversarial training includes the following steps: S101, taking any one group of corresponding images of the vehicle panoramic surround view dataset with good lighting conditions and the vehicle panoramic surround view dataset with low lighting conditions as a current training group; S102, inputting the low-light panoramic surround view in the current training group into the generator to obtain a panoramic surround view of the vehicle after light enhancement; S103, inputting the illumination-enhanced vehicle panoramic surround view and the illumination-well-enhanced vehicle panoramic surround view in the current training group into the discriminator for discrimination, so that the generator and the discriminator are subjected to adversarial training.

10. The method for enhancing illumination of a vehicle panoramic surround view based on a generative adversarial network according to claim 9, characterized in that: The end condition of the adversarial training is that the training ends when the generator can generate realistic samples and the probability value of the discriminator's correct judgment is 0.5.

Images