Automatic driving multi-view image GAN optimization fusion and reconstruction method and system

Abstract

The invention discloses an automatic driving multi-view image GAN optimization fusion and reconstruction method and system, and relates to the technical field of automatic driving, and the method comprises the following steps: 1, image collection and synchronization; 2, preprocessing the image; step 3, initializing the GAN model; 4, optimizing an evolutionary algorithm; step 5, shielding detection and repair; step 6, performing multi-view fusion; and step 7, outputting and applying. According to the method, the quality and consistency of multi-view image fusion can be improved, the shielding condition is effectively processed, the integrity of the image is ensured, the distortion correction effect in the image is improved, the parallax error between different views is reduced, the authenticity and accuracy of the reconstructed image are ensured, the calculation efficiency is improved, and the real-time processing requirement is supported.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and in particular relates to a GAN optimization fusion and reconstruction method and system for autonomous driving multi-view images. Background Art

[0002] Existing autonomous vehicles capture surrounding environment information through multiple cameras, but traditional image processing methods have the following limitations: 1. Occlusion problem When an object is partially obscured by other objects, traditional methods have difficulty accurately restoring the information of the occluded area. For example, on urban roads, pedestrians, vehicles, or other obstacles may partially obstruct road signs or traffic conditions ahead. Traditional image restoration methods, such as those based on texture synthesis or simple interpolation, often fail to generate natural and realistic content in the occluded area, resulting in obvious discontinuity and distortion in the fused image. This not only affects the quality of the image, but may also mislead the perception module of the autonomous driving system, thereby affecting the accuracy of decision-making.

[0003] 2. Distortion Correction Traditional correction algorithms may not be able to completely eliminate image deformation caused by wide-angle lenses or other factors. Although wide-angle lenses can provide a wider field of view, they also introduce significant radial and tangential distortion, especially in the edge areas of the image. Traditional distortion correction methods usually rely on fixed camera intrinsic parameter matrices and distortion coefficients, but in practical applications, these parameters may drift due to environmental changes (such as temperature and humidity), resulting in poor correction effects. In addition, for some complex scenes, such as light changes at the entrance or exit of a tunnel, traditional methods are difficult to adaptively adjust, which affects the authenticity and consistency of the image.

[0004] 3. Parallax Error The image registration from different perspectives is not accurate enough, resulting in a decrease in the quality of the fused image. Parallax error is a common problem in the multi-view image fusion process. Even with the use of techniques such as feature point matching and homography transformation, slight deviations between different perspectives can still lead to inaccurate image alignment. Especially in dynamic environments, changes in vehicle speed and camera installation positions can further exacerbate parallax errors. This not only affects the spatial consistency of the fused image, but can also lead to depth estimation errors in 3D reconstruction, reducing the system's ability to understand the surrounding environment.

[0005] 4. Computational efficiency Traditional methods usually require a lot of computing resources, have poor real-time performance, and are not suitable for the rapid response requirements of autonomous driving. Complex image processing algorithms, such as denoising, brightness adjustment, distortion correction, and multi-view fusion of high-resolution images, require a lot of computing operations. Traditional CPU implementations are difficult to meet the requirements of real-time processing, especially in the case of high frame rates and multi-camera inputs. Although GPU acceleration can improve computing efficiency, there are still memory bandwidth limitations and power consumption issues, which limit its widespread application in vehicle environments. In addition, some traditional methods lack effective parallelization design, further reducing the real-time performance of the system. Summary of the invention

[0006] In response to the above problems, the present invention provides a GAN optimization fusion and reconstruction method and system for multi-view images in autonomous driving.

[0007] The present invention is achieved through the following technical solutions: The GAN optimization fusion and reconstruction method for multi-view images in autonomous driving includes the following steps: Step 1: Image acquisition and synchronization By installing multiple fisheye cameras on each vehicle, they cover the front, left, right and rear fields of view respectively; using a synchronous controller to receive trigger signals from the main control unit via the CAN bus, all cameras are activated at the same time for shooting; Step 2: Image Preprocessing Perform preliminary processing on the collected original images, including denoising, brightness adjustment, color correction, and distortion correction; Step 3: GAN model initialization Construct the DCGAN architecture, use the deep convolutional generative adversarial network as the basic architecture, and combine the residual block to improve the feature expression ability; Step 4: Optimize the evolutionary algorithm of the GAN model Step 5: Occlusion detection and repair Use U-Net or other semantic segmentation networks to automatically identify occluded areas and mark their boundaries; call the GAN model optimized by the evolutionary algorithm to predict and fill in the occluded parts based on the information of adjacent non-occluded areas; Step 6: Multi-view fusion Using the principle of multi-view geometry, image data from all viewpoints are combined to construct a 3D point cloud or panorama. Use the optical flow method or direct linear transformation (DLT) algorithm to map each perspective image to a unified coordinate system to reduce parallax error; Integrate all processed images to form a coherent overall visual effect; Step 7: Output and Application The fused image is converted into a format suitable for the autonomous driving system to facilitate subsequent processing; the image data is transmitted to the perception layer or other related components in real time through a high-speed interface to provide support for path planning and obstacle detection functions.

[0008] Preferably, the denoising in step 2 uses bilateral filtering to remove image noise and keep edges clear; the mathematical expression of the bilateral filter is: Where w(y) is the weight function, defined as: g and f are Gaussian kernel functions in the spatial domain and grayscale domain, respectively.

[0009] The parameters are set to and , to balance smoothing and edge preservation.

[0010] Preferably, the brightness and contrast adjustment is based on an adaptive histogram equalization technique to improve the visual effect of the image; the image contrast is enhanced by histogram equalization of a local area, which is suitable for situations where the lighting is uneven.

[0011] Preferably, the distortion correction is to apply the camera internal and external parameter matrix and the distortion coefficient to correct the image distortion through the `cv::undistort` function in the OpenCV library.

[0012] Preferably, in step 3, a DCGAN architecture is constructed, wherein the generator is composed of multiple transposed convolutional layers, and the discriminator is composed of conventional convolutional layers; residual connections are introduced to enhance the learning ability of the network; the input of the generator G is a random vector z, the output is a synthetic image G(z), and the task of the discriminator D is to distinguish between the real image x and the generated image G(z); the loss function L is defined as: The generator and discriminator are trained alternately until a stable state is reached.

[0013] Preferably, the optimization of the evolutionary algorithm of the GAN model in step 4 comprises the following steps: (1) Initialize the population: Set the initial population size N. Each individual represents a set of GAN weight parameters, using real number encoding. Assuming that the generator G and the discriminator D have WG and WD weight parameters respectively, each individual can be represented as a vector of length (WG + WD). Assign initial values ​​to each individual based on experience or random initialization methods to form an initial population. (2) Fitness function definition: Design a fitness function F(I) to evaluate the quality of the images generated by GAN, including but not limited to PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index); the fitness function can be defined as: Where I represents an image, is a balance factor, usually set to 0.5 to weigh the importance of PSNR and SSIM. For specific application scenarios, you can adjust it as needed The value of (3) Genetic manipulation: Selection operation: Roulette selection is used to select parent individuals to enter the next generation according to the fitness ratio of the individuals; assuming that the fitness of an individual is fi, the probability of its being selected is: Crossover operation: Use single-point crossover to randomly select two parent individuals and exchange their gene fragments at a certain point; for two individuals P1 and P2, assuming that crossover is performed at the kth position, the new individuals C1 and C2 can be expressed as: Mutation operation: Introduce Gaussian mutation, that is, add a small perturbation that follows Gaussian distribution to certain gene positions; assume that the mutation probability is pm and the mutation intensity is , then the mutated gene g' can be expressed as: in The mean is 0 and the variance is is a Gaussian random variable.

[0014] Autonomous driving multi-view image GAN optimization fusion and reconstruction system, including A multi-view image acquisition module, which includes multiple cameras for front, side and rear views, covering a full 360-degree field of view, and is equipped with a synchronous trigger mechanism to ensure that all cameras shoot at the same time; A preprocessing module, which is responsible for performing preliminary processing such as denoising, brightness adjustment, and color correction on the original image, and performs distortion correction using camera calibration technology and a radial distortion model; An evolutionary algorithm-optimized GAN training module, wherein the evolutionary algorithm-optimized GAN training module includes a generator and a discriminator optimized by a genetic algorithm, uses a deep convolutional generative adversarial network as a basic architecture, and combines a residual block to improve feature expression capabilities; Occlusion detection and repair module: Automatically identify occluded areas based on deep learning algorithms; Image fusion module: Adopts the principle of multi-view geometry and combines the 3D model generated by GAN to seamlessly fuse images from different viewpoints; uses the optical flow method or direct linear transformation algorithm for image registration to reduce parallax errors; The output and application module is used to output the fused image.

[0015] Compared with the prior art, the present invention has the following advantages: 1. Improve the quality and consistency of multi-view image fusion By introducing the generative adversarial network (GAN) optimized by the evolutionary algorithm, the present invention can better handle various challenges in the multi-view image fusion process, including occlusion repair, distortion correction, and parallax error. The GAN model generates realistic images by learning a large amount of real scene data, ensuring that images from different perspectives are highly consistent and coherent.

[0016] 2. Effectively handle occlusion and ensure image integrity The present invention uses deep learning technology, especially the U-Net architecture, to automatically identify occluded areas and fill in the missing parts with the GAN model optimized by the evolutionary algorithm. This method can not only generate natural and realistic occluded area content, but also maintain the overall structure and details of the image, providing reliable data support for subsequent perception and decision-making.

[0017] 3. Improved distortion correction in images The present invention adopts advanced distortion correction algorithm, combined with adaptive calibration technology and multi-scale feature extraction method, which can dynamically adjust camera parameters in different environments to ensure the effect of distortion correction. At the same time, by introducing convolutional neural network (CNN), the present invention can learn a more robust distortion correction model, which is suitable for complex and changeable actual driving scenes.

[0018] 4. Reduce the parallax error between different perspectives to ensure the authenticity and accuracy of the reconstructed image The present invention uses the principle of multi-view geometry and the optical flow method, combined with the direct linear transformation (DLT) algorithm, to accurately align images from different viewpoints. In addition, by introducing constraints such as smoothness and coherence, the present invention can effectively reduce parallax errors and ensure the authenticity and accuracy of three-dimensional reconstruction. In particular, in a dynamic environment, the present invention can adjust the alignment parameters in real time to adapt to changes in vehicle speed and camera installation position.

[0019] 5. Improve computing efficiency and support real-time processing requirements In order to improve computing efficiency, the present invention adopts a method that combines hardware acceleration (such as GPU, FPGA) and software optimization. By optimizing the network architecture and algorithm design, the present invention can significantly reduce the amount of calculation and memory usage while ensuring image quality. In addition, the present invention also introduces a parallel processing strategy, fully utilizing the advantages of multi-core processors, achieving efficient real-time processing, and meeting the rapid response requirements of the autonomous driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings of the present invention are used to provide a further understanding of the embodiments of the present invention, constitute a part of this specification, and exemplarily illustrate the embodiments of the present invention and the description thereof, and are used to explain the principles of the present invention.

[0021] Figure 1 Workflow diagram of the GAN optimization fusion and reconstruction method for multi-view images in autonomous driving; Figure 2 Optimize workflow diagram for evolutionary algorithm; Figure 3 This is the workflow diagram for occlusion detection and repair; Figure 4 Structural block diagram of the GAN optimization fusion and reconstruction system for multi-view images in autonomous driving. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Embodiment 1 See also Figure 1-Figure 3 , the GAN optimization fusion and reconstruction method of multi-view images for autonomous driving includes the following steps: Step 1: Image acquisition and synchronization For each vehicle, four fisheye cameras (FOV = 180°) are installed to cover the front, left, right and rear fields of view respectively. The synchronous controller receives the trigger signal from the main control unit through the CAN bus and activates all cameras to shoot at the same time. Each camera shoots at a frequency of 30 frames per second to ensure the continuity of real-time data flow.

[0024] Step 2: Image Preprocessing 2.1 Denoising and smoothing: Bilateral filtering is used to remove image noise and keep edges clear. The mathematical expression of the bilateral filter is: Where w(y) is the weight function, defined as: g and f are Gaussian kernel functions in the spatial domain and grayscale domain, respectively.

[0025] The parameters are set to and , to balance smoothing and edge preservation.

[0026] 2.2 Brightness and contrast adjustment: Improve the visual effect of the image based on the adaptive histogram equalization (CLAHE) technology. This technology enhances the image contrast by histogram equalization in the local area, which is suitable for the situation of uneven lighting.

[0027] 2.3 Distortion Correction: Apply the camera internal and external parameter matrix and distortion coefficients to correct image distortion through the `cv::undistort` function in the OpenCV library.

[0028] Assume that the intrinsic parameter matrix of a camera is: And the distortion coefficient ,The distortion correction process maps the image from the nonlinear space back to the linear space according to these parameters, thereby eliminating the distortion.

[0029] In actual driving environments, vehicles travel on roads with large changes in light, such as tunnel entrances and exits. At this time, the adaptive histogram equalization (CLAHE) technology can significantly improve the brightness and contrast of the image, making subsequent image processing more reliable.

[0030] Step 3 GAN model initialization Network architecture selection: Construct a DCGAN architecture, where the generator consists of multiple transposed convolution layers and the discriminator consists of regular convolution layers. Introduce residual connections to enhance the learning ability of the network. The input of the generator G is a random vector z, the output is a synthetic image G(z), and the task of the discriminator D is to distinguish between the real image x and the generated image G(z). The loss function L is defined as: The generator and discriminator are trained alternately until a stable state is reached.

[0031] During the training phase, a large number of annotated multi-view image datasets are used, including scenes under different weather conditions and lighting changes. Through continuous iterative training, GAN gradually learns to generate realistic images, especially in dealing with complex backgrounds and occlusions.

[0032] Step 4: Optimize the evolutionary algorithm of the GAN model, as follows Figure 2 As shown: (1) Initialize the population: Set the initial population size N, for example, N = 50. Each individual represents a set of GAN weight parameters, using real number encoding. Assuming that the generator G and the discriminator D have WG and WD weight parameters respectively, each individual can be represented as a vector of length (WG + WD). Assign initial values ​​to each individual based on experience or random initialization methods to form an initial population.

[0033] (2) Fitness function definition: Design the fitness function F(I) to evaluate the quality of the images generated by GAN, including but not limited to indicators such as PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index). The fitness function can be defined as: Where I represents an image, is a balance factor, usually set to 0.5 to weigh the importance of PSNR and SSIM. For specific application scenarios, you can adjust it as needed The value of .

[0034] (3) Genetic manipulation: Selection operation: Roulette Wheel Selection is used to select parent individuals to enter the next generation according to the fitness ratio of the individuals. Assuming that the fitness of an individual is fi, the probability of its being selected is: Crossover operation: Use single-point crossover. Randomly select two parent individuals and exchange their gene fragments at a certain point. For example, for two individuals P1 and P2, assuming that crossover is performed at the kth position, the new individuals C1 and C2 can be expressed as: Mutation operation: Introduce Gaussian mutation, that is, add a small perturbation that follows Gaussian distribution to certain gene positions. Assume that the mutation probability is pm and the mutation intensity is , then the mutated gene g' can be expressed as: in The mean is 0 and the variance is is a Gaussian random variable.

[0035] Specifically, in the occlusion repair under the highway environment, the evolutionary algorithm is used to optimize the GAN model. The specific parameters are as follows: Population size: N = 50 Iterations: T = 1000 Crossover probability: pc = 0.8 Mutation probability: pm = 0.1 Mutation Strength: = 0.01 After 1,000 iterations, we finally found a set of optimal GAN ​​weight parameters. The images generated using this parameter have significantly improved the restoration effect in occluded areas, especially when dealing with complex backgrounds and lighting changes. In the test set, PSNR increased by an average of 10% and SSIM increased by an average of 8%.

[0036] Image fusion in urban road environment: Population size: N = 100 Iterations: T = 2000 Crossover probability: pc = 0.9 Mutation probability: pm = 0.2 Mutation Strength: = 0.02 After 2000 iterations, we finally found a set of optimal GAN ​​weight parameters. The fused images generated using this parameter have significantly improved consistency and authenticity across different perspectives. In particular, they perform well in dealing with parallax errors and illumination changes. In the test set, PSNR increased by an average of 12% and SSIM by an average of 10%.

[0037] Step 5: Occlusion detection and repair, as follows Figure 3 As shown: 5.1 Occlusion region labeling: Use U-Net or other semantic segmentation networks to automatically identify occlusion regions and mark their boundaries. U-Net is a convolutional neural network commonly used in medical image analysis. It has an encoder-decoder structure that can capture contextual information well. In occlusion detection, U-Net accepts an input image I and outputs a binary mask M, which indicates the location of the occlusion region.

[0038] 5.2 Missing part generation: Call the GAN model optimized by the evolutionary algorithm to predict and fill the occluded part based on the information of the adjacent non-occluded area. Constraints such as smoothness and coherence can be added during the generation process to ensure that the generated results are natural and reasonable. The generator G accepts the occlusion mask M and the original image I as input and outputs the repaired image I': In urban road environments, vehicles often encounter occlusions from pedestrians and other obstacles. After detecting the occluded area through U-Net, the GAN model can intelligently fill in the missing parts and generate a complete image. This not only improves the consistency and completeness of the image, but also provides more accurate data for subsequent perception and decision-making.

[0039] Step 6: Multi-view fusion 6.1 3D reconstruction: Using the principle of multi-view geometry, we combine the image data from all viewpoints to construct a 3D point cloud or panorama. In this stage, we use the SfM (Structure from Motion) technology to extract 3D structural information from 2D images. Suppose we have n images from different viewpoints. , calculate the position of three-dimensional points through triangulation.

[0040] 6.2 Image registration: Use the optical flow method or direct linear transformation (DLT) algorithm to map each view image to a unified coordinate system to reduce parallax errors. For overlapping areas, use weighted averaging or median filtering to make a smooth transition.

[0041] Assume that the set of points with the same name between the two images I1 and I2 is , use the DLT algorithm to solve the basic matrix F, and then get the essential matrix E and relative posture .

[0042] 6.3 Final fusion: Integrate all processed images to form a coherent overall visual effect. This stage also takes into account factors such as lighting changes to ensure that the color and brightness of the fused image are consistent.

[0043] In highway scenarios, vehicles need to process images from four directions at the same time. Through multi-view fusion technology, images from different perspectives are seamlessly stitched into a panoramic picture, helping the autonomous driving system to better understand the surrounding environment and make reasonable driving decisions.

[0044] Step 7: Export and Apply 7.1 Format conversion: Convert the fused image into a format suitable for the autonomous driving system, such as JPEG, PNG, etc., to facilitate subsequent processing.

[0045] 7.2 Real-time transmission: Image data is transmitted to the perception layer or other related components in real time through high-speed interfaces (such as USB 3.0, PCIe, etc.) to provide support for functions such as path planning and obstacle detection.

[0046] The fused image is encoded into JPEG format and transmitted to the onboard computer in real time through the PCIe interface. After receiving the image, the perception layer of the autonomous driving system further performs target detection, classification and tracking to provide the necessary environmental information for the decision-making layer.

[0047] Embodiment 2 See also Figure 4 , autonomous driving multi-view image GAN optimization fusion and reconstruction system, including Multi-view image acquisition module: including front view, side view, rear view and other cameras. The installation position is carefully designed to ensure full 360-degree coverage. It is also equipped with a synchronous trigger mechanism to ensure that all cameras shoot at the same time.

[0048] Preprocessing module: responsible for the preliminary processing of the original image such as denoising, brightness adjustment, color correction, etc., and performs distortion correction using camera calibration technology and radial distortion models (such as the Brown-Conrady model) for correction.

[0049] Evolutionary algorithm optimized GAN training module: Contains a generator and discriminator optimized by a genetic algorithm (GA), uses a deep convolutional generative adversarial network (DCGAN) as the basic architecture, and combines residual blocks (ResNet Blocks) to improve feature expression capabilities.

[0050] Occlusion detection and repair module: Automatically identify occluded areas based on deep learning algorithms, such as the U-Net architecture, which can effectively segment occluded areas and predict their boundaries.

[0051] Image fusion module: Adopting the principle of multi-view geometry and combining it with the 3D model generated by GAN, images from different viewpoints are seamlessly merged. The optical flow method or direct linear transformation (DLT) algorithm is used for image registration to reduce parallax errors.

[0052] The output and application module is used to output the fused image.

[0053] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions. The sentence "includes an element defined by ... does not exclude the existence of other identical elements in the process, method, article or device including the element".

[0054] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A GAN optimization fusion and reconstruction method for multi-view images in autonomous driving, characterized in that: The steps include: Step 1: Image acquisition and synchronization By installing multiple fisheye cameras on each vehicle, they cover the front, left, right and rear fields of view respectively; using a synchronous controller to receive trigger signals from the main control unit via the CAN bus, all cameras are activated at the same time for shooting; Step 2: Image Preprocessing Perform preliminary processing on the collected original images, including denoising, brightness adjustment, color correction, and distortion correction; Step 3: GAN model initialization Construct the DCGAN architecture, use the deep convolutional generative adversarial network as the basic architecture, and combine the residual block to improve the feature expression ability; Step 4: Optimize the evolutionary algorithm of the GAN model Step 5: Occlusion detection and repair Use the U-Net segmentation network to automatically identify occluded areas and mark their boundaries; call the GAN model optimized by the evolutionary algorithm to predict and fill in the occluded parts based on the information of adjacent non-occluded areas; Step 6: Multi-view fusion Using the principle of multi-view geometry, image data from all viewpoints are combined to construct a 3D point cloud or panorama. Use the optical flow method or direct linear transformation (DLT) algorithm to map each perspective image to a unified coordinate system to reduce parallax error; Integrate all processed images to form a coherent overall visual effect; Step 7: Output and Application The fused image is converted into a format suitable for the autonomous driving system to facilitate subsequent processing; the image data is transmitted to the perception layer or other related components in real time through a high-speed interface to provide support for path planning and obstacle detection functions.

2. The autonomous driving multi-view image GAN optimization fusion and reconstruction method according to claim 1, characterized in that: The denoising in step 2 uses bilateral filtering to remove image noise and keep the edges clear; the mathematical expression of the bilateral filter is: Where w(y) is the weight function, defined as: g and f are Gaussian kernel functions in the spatial domain and grayscale domain, respectively; The parameters are set to and , to balance smoothing and edge preservation.

3. The autonomous driving multi-view image GAN optimization fusion and reconstruction method according to claim 1, characterized in that: The brightness and contrast adjustment in step 2 is based on the adaptive histogram equalization technology to improve the visual effect of the image; the image contrast is enhanced by histogram equalization of the local area, which is suitable for the situation of uneven lighting.

4. The autonomous driving multi-view image GAN optimization fusion and reconstruction method according to claim 1, characterized in that: The distortion correction in step 2 is to apply the camera internal and external parameter matrix and distortion coefficients to correct image distortion through the `cv::undistort` function in the OpenCV library.

5. The autonomous driving multi-view image GAN optimization fusion and reconstruction method according to claim 1, characterized in that: In step 3, a DCGAN architecture is constructed, in which the generator consists of multiple transposed convolutional layers and the discriminator consists of conventional convolutional layers; residual connections are introduced to enhance the learning ability of the network; the input of the generator G is a random vector z, the output is a synthetic image G(z), and the task of the discriminator D is to distinguish between the real image x and the generated image G(z); the loss function L is defined as: The generator and discriminator are trained alternately until a stable state is reached.

6. The autonomous driving multi-view image GAN optimization fusion and reconstruction method according to claim 1, characterized in that: The optimization of the evolutionary algorithm for the GAN model in step 4 includes the following steps: (1) Initialize the population: Set the initial population size N. Each individual represents a set of GAN weight parameters, using real number encoding. Assuming that the generator G and the discriminator D have WG and WD weight parameters respectively, each individual can be represented as a vector of length (WG + WD). Assign initial values ​​to each individual based on experience or random initialization methods to form an initial population. (2) Fitness function definition: Design a fitness function F(I) to evaluate the quality of the images generated by GAN, including but not limited to PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index); the fitness function can be defined as: Where I represents an image, is a balance factor, usually set to 0.5 to weigh the importance of PSNR and SSIM; for specific application scenarios, it can be adjusted as needed The value of (3) Genetic manipulation: Selection operation: Roulette selection is used to select parent individuals to enter the next generation according to the fitness ratio of the individuals; assuming that the fitness of an individual is fi, the probability of its being selected is: Crossover operation: Use single-point crossover to randomly select two parent individuals and exchange their gene fragments at a certain point; for two individuals P1 and P2, assuming that crossover is performed at the kth position, the new individuals C1 and C2 can be expressed as: Mutation operation: Introduce Gaussian mutation, that is, add a small perturbation that follows Gaussian distribution to certain gene positions; assume that the mutation probability is pm and the mutation intensity is , then the mutated gene g' can be expressed as: in The mean is 0 and the variance is is a Gaussian random variable.

7. An autonomous driving multi-view image GAN optimization fusion and reconstruction system, used to implement the autonomous driving multi-view image GAN optimization fusion and reconstruction method according to any one of claims 1 to 6, characterized in that: include A multi-view image acquisition module, which includes multiple cameras for front, side and rear views, covering a full 360-degree field of view, and is equipped with a synchronous trigger mechanism to ensure that all cameras shoot at the same time; A preprocessing module, which is responsible for performing preliminary processing such as denoising, brightness adjustment, and color correction on the original image, and performs distortion correction using camera calibration technology and a radial distortion model; An evolutionary algorithm-optimized GAN training module, wherein the evolutionary algorithm-optimized GAN training module includes a generator and a discriminator optimized by a genetic algorithm, uses a deep convolutional generative adversarial network as a basic architecture, and combines a residual block to improve feature expression capabilities; Occlusion detection and repair module: Automatically identify occluded areas based on deep learning algorithms; Image fusion module: Adopts the principle of multi-view geometry and combines the 3D model generated by GAN to seamlessly fuse images from different viewpoints; uses the optical flow method or direct linear transformation algorithm for image registration to reduce parallax errors; The output and application module is used to output the fused image.

Images