Binocular illumination data enhancement method and device, equipment, storage medium

Abstract

The application relates to the binocular vision field, in particular to a binocular light data enhancement method and device, equipment and a storage medium. A plurality of groups of image data collected by a first structured light camera, a second structured light camera and a binocular camera are acquired. Corresponding disparity true values are determined according to each group of image data and a calibration matrix. The calibration matrix is used for converting the spatial position relationship of the first structured light camera, the second structured light camera and the binocular camera. A generative adversarial network is trained according to each group of image data and the corresponding disparity true values. According to the trained generative adversarial network, a plurality of enhanced image data corresponding to each group of image data are generated according to preset light parameters. The application introduces an image collected by a structured light camera with a cross-view angle to extract a real scene basic light as a reference base for light migration, thereby improving the reliability of model training and the application process. In addition, the disparity true values are introduced in the training process of the generative adversarial network, so that the image disparity generated by the trained network is invariable.

Description

Technical Field

[0001] This application relates to the field of binocular vision, and includes, but is not limited to, a binocular illumination data enhancement method, apparatus, device, and storage medium. Background Technology

[0002] Depth estimation networks for binocular stereo vision rely on large-scale, diverse training data. Existing data augmentation methods mainly fall into two categories: one is pixel-level transformation (flipping, cropping, color perturbation, etc.), which cannot simulate changes in physical lighting properties such as light source direction and intensity; the other is synthetic data generation based on graphics rendering, which requires the construction of a complete 3D scene, resulting in high computational costs and differences between synthetic images and real-world images in terms of statistical characteristics such as texture and noise distribution.

[0003] Significant progress has been made in the field of image relighting based on generative adversarial networks (GANs). Image-to-image translation frameworks, such as pix2pix, model relighting as a pixel-level mapping task from the input image to the target lighting image. Through adversarial training between a U-Net generator and a PatchGAN discriminator, the transformation relationships between different lighting domains are learned. These methods have demonstrated good results in tasks such as monocular face relighting and scene lighting transfer. However, existing relighting methods are all designed for monocular images and do not consider the geometric consistency constraints of binocular stereo image pairs. Summary of the Invention

[0004] In view of this, the binocular illumination data enhancement method, apparatus, device, and storage medium provided in the embodiments of this application can perform accurate and reliable data enhancement processing on binocular images through generative adversarial networks.

[0005] The binocular illumination data enhancement method, apparatus, device, and storage medium provided in this application embodiment are implemented as follows: One aspect of this application provides a binocular illumination data enhancement method, the method comprising: Acquire multiple sets of image data collected by the first structured light camera, the second structured light camera, and the binocular camera; The corresponding parallax true value is determined based on each set of image data and calibration matrix. The calibration matrix is ​​used to transform the spatial position relationship between the first structured light camera, the second structured light camera and the binocular camera. Generative adversarial networks are trained based on the graph data and corresponding disparity ground truth values ​​of each group. Based on the trained generative adversarial network, multiple enhanced image data corresponding to each group of image data are generated according to preset lighting parameters.

[0006] In one possible implementation, each set of image data includes a top-view binocular left image and a top-view binocular right image acquired by the binocular camera, two reference images acquired by the first structured light camera and the second structured light camera respectively, and a depth map acquired by the first structured light camera as the main structured light camera.

[0007] In one possible implementation, determining the corresponding disparity ground truth value based on each set of graph data and calibration matrix includes: Determine the corresponding 3D point cloud based on the depth map; The three-dimensional point cloud is projected onto the camera coordinate system corresponding to the left eye camera in the binocular camera according to the calibration matrix to obtain the true three-dimensional coordinates of each point in the camera coordinate system. The true disparity value is determined based on the actual three-dimensional coordinates of each point.

[0008] In one possible implementation, training the generative adversarial network based on each set of graph data and the corresponding disparity ground truth includes: Target segmentation is performed on each group of image data to obtain the left image mask corresponding to the top-view binocular left image and the reference image mask corresponding to the reference image; The left image mask and the reference image mask are aligned across viewpoints to obtain an effectively aligned mask. Based on the reference image and the aligned effective mask, cross-view illumination decoupling is performed to obtain illumination condition coding; The generator network determines the generated binocular left image and the generated binocular right image based on the top-view binocular left image, the top-view binocular right image, the left image mask, and the lighting condition encoding. The generated image loss is determined based on the image data in this set, the left image mask, the illumination condition encoding, the generated binocular left image, and the generated binocular right image, and the parameters of the generating network are adjusted according to the generated image loss.

[0009] In one possible implementation, the step of performing cross-view illumination decoupling based on the reference image and the aligned effective mask to obtain illumination condition coding includes: The reference image and the aligned effective mask are encoded by a basic illumination encoder to obtain a basic illumination vector; An active lighting vector is obtained by encoding the active lighting parameters using an active lighting encoder. The illumination condition code is obtained by fusing the base illumination vector and the active illumination vector.

[0010] In one possible implementation, the step of determining the generated binocular left image and the generated binocular right image by the generative network based on the top-view binocular left image, the top-view binocular right image, the left image mask, and the illumination condition encoding includes: Multi-scale content features of the top-view binocular left image and the top-view binocular right image are extracted using a binocular encoder; The left heavy illumination image and the right heavy illumination image are determined by the illumination modulation decoder based on the multi-scale content features and the illumination condition coding; A masking and blending operation is performed on the top-view binocular left image, the top-view binocular right image, the left image mask, the left heavy illumination image, and the right heavy illumination image to obtain the generated binocular left image and the generated binocular right image.

[0011] In one possible implementation, determining the generated image loss based on the current set of image data, the left image mask, the illumination condition encoding, the generated binocular left image, and the generated binocular right image, and adjusting the parameters of the generation network based on the generated image loss, includes: The geometric loss is determined based on the data in this set of images, the left image mask, the generated binocular left image, and the generated binocular right image. By combining one of the top-view binocular left image, the top-view binocular right image, the generated binocular left image, and the generated binocular right image with the illumination condition encoding, the corresponding real image score is obtained by inputting the network into the condition discrimination network. Calculate the adversarial loss based on the type of the input image and the corresponding real image score; The generated image loss is determined based on the weighted sum of the geometric loss and the adversarial loss, and the parameters of the generating network are adjusted based on the generated image loss.

[0012] Another aspect of the embodiments of this application provides a binocular illumination data enhancement device, the device comprising: The data acquisition module is used to acquire multiple sets of image data collected by the first structured light camera, the second structured light camera, and the binocular camera; The parameter calculation module is used to determine the corresponding parallax true value based on each set of image data and calibration matrix. The calibration matrix is ​​used to transform the spatial position relationship between the first structured light camera, the second structured light camera and the binocular camera. The network training module is used to train and generate adversarial networks based on the graph data of each group and the corresponding disparity ground truth. The image generation module is used to generate multiple enhanced image data corresponding to each group of image data based on the trained generative adversarial network and preset illumination parameters.

[0013] In one possible implementation, each set of image data includes a top-view binocular left image and a top-view binocular right image acquired by the binocular camera, two reference images acquired by the first structured light camera and the second structured light camera respectively, and a depth map acquired by the first structured light camera as the main structured light camera.

[0014] In one possible implementation, the parameter calculation module is further configured to: Determine the corresponding 3D point cloud based on the depth map; The three-dimensional point cloud is projected onto the camera coordinate system corresponding to the left eye camera in the binocular camera according to the calibration matrix to obtain the true three-dimensional coordinates of each point in the camera coordinate system. The true disparity value is determined based on the actual three-dimensional coordinates of each point.

[0015] In one possible implementation, the network training module is further configured to: Target segmentation is performed on each group of image data to obtain the left image mask corresponding to the top-view binocular left image and the reference image mask corresponding to the reference image; The left image mask and the reference image mask are aligned across viewpoints to obtain an effectively aligned mask. Based on the reference image and the aligned effective mask, cross-view illumination decoupling is performed to obtain illumination condition coding; The generator network determines the generated binocular left image and the generated binocular right image based on the top-view binocular left image, the top-view binocular right image, the left image mask, and the lighting condition encoding. The generated image loss is determined based on the image data in this set, the left image mask, the illumination condition encoding, the generated binocular left image, and the generated binocular right image, and the parameters of the generating network are adjusted according to the generated image loss.

[0016] In one possible implementation, the network training module is further configured to: The reference image and the aligned effective mask are encoded by a basic illumination encoder to obtain a basic illumination vector; An active lighting vector is obtained by encoding the active lighting parameters using an active lighting encoder. The illumination condition code is obtained by fusing the base illumination vector and the active illumination vector.

[0017] In one possible implementation, the network training module is further configured to: Multi-scale content features of the top-view binocular left image and the top-view binocular right image are extracted using a binocular encoder; The left heavy illumination image and the right heavy illumination image are determined by the illumination modulation decoder based on the multi-scale content features and the illumination condition coding; A masking and blending operation is performed on the top-view binocular left image, the top-view binocular right image, the left image mask, the left heavy illumination image, and the right heavy illumination image to obtain the generated binocular left image and the generated binocular right image.

[0018] In one possible implementation, the network training module is further configured to: The geometric loss is determined based on the data in this set of images, the left image mask, the generated binocular left image, and the generated binocular right image. By combining one of the top-view binocular left image, the top-view binocular right image, the generated binocular left image, and the generated binocular right image with the illumination condition encoding, the corresponding real image score is obtained by inputting the network into the condition discrimination network. Calculate the adversarial loss based on the type of the input image and the corresponding real image score; The generated image loss is determined based on the weighted sum of the geometric loss and the adversarial loss, and the parameters of the generating network are adjusted based on the generated image loss.

[0019] The electronic device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0020] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method provided in this application embodiment.

[0021] The binocular illumination data enhancement method, apparatus, device, and storage medium provided in this application acquire multiple sets of image data collected by a first structured light camera, a second structured light camera, and a binocular camera. The corresponding disparity ground truth is determined based on each set of image data and a calibration matrix. The calibration matrix is ​​used to transform the spatial positional relationship of the first structured light camera, the second structured light camera, and the binocular camera. A generative adversarial network (GAN) is trained based on each set of image data and the corresponding disparity ground truth. The trained GAN generates multiple enhanced image data corresponding to each set of image data according to preset illumination parameters. This application introduces images acquired by structured light cameras with cross-view perspectives to extract the real-world baseline illumination of the scene, serving as a reference base for illumination transfer, thereby improving the reliability of model training and application. Furthermore, disparity ground truth is introduced during the training process of the GAN to ensure that the disparity of the images generated by the trained network remains unchanged. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a binocular illumination data enhancement method according to an embodiment of this application is shown; Figure 2 A schematic diagram of a binocular illumination data enhancement device according to an embodiment of this application is shown; Figure 3 A schematic diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0026] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0027] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0028] The binocular illumination data enhancement method of this application embodiment can be executed by any electronic device, including but not limited to mobile phones, wearable devices (such as smartwatches, smart bracelets, smart glasses, etc.), tablet computers, laptops, vehicle terminals, PCs (Personal Computers), etc. The function implemented by this method can be achieved by a processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the electronic device includes at least a processor and a storage medium.

[0029] The binocular illumination data augmentation method of this application can be used in any application scenario that requires binocular illumination data augmentation to expand binocular image training data. For example, it can be used to expand the binocular image training data input to the deep learning model during disordered grasping in industrial applications; or to expand the binocular image training data input to the model for obstacle recognition in service / home robots, etc.

[0030] Related technologies have the following significant drawbacks in terms of illumination enhancement for training data for binocular depth estimation: First, there is insufficient diversity in lighting conditions. Real-world scene data acquisition typically uses a fixed light source, and each acquisition only captures one type of lighting condition. Pixel-level enhancement cannot simulate changes in physical parameters such as lighting direction and angle of incidence, resulting in training data with a single lighting dimension and poor generalization ability of depth estimation networks.

[0031] Second, there is a lack of binocular geometric consistency. Pix2pix, a typical monocular image-to-image translation method, achieves illumination transformation through end-to-end mapping, but its design is only for a single image. When such methods are applied to the left and right binocular images separately, independent processing leads to inconsistent luminance changes at corresponding points in the left and right eyes. Taking the recent multi-view relighting method SyncLight as an example, this method achieves consistent illumination editing across multiple viewpoints through a multi-view diffusion transformer, but its core design is geared towards multi-camera broadcasting, stereoscopic film and television applications, and achieves multi-view collaborative relighting through the propagation of editing signals from a single reference viewpoint, without specifically addressing the parallax preservation problem in binocular stereo vision. Existing technologies as a whole lack a relighting mechanism for binocular image geometric consistency; inconsistent relighting between the left and right eyes will cause stereo matching failure, and the original ground truth depth cannot be reused.

[0032] Third, cross-view information is not fully utilized. Real-world acquisition systems are often equipped with multi-view auxiliary cameras, whose images contain rich ambient lighting information, but existing methods do not effectively utilize this cross-view lighting to guide the relighting generation of binocular images.

[0033] Fourth, there is a lack of locally controllable lighting editing capabilities. In robot vision tasks such as object recognition and grasping, the changes in the appearance of the target object's surface under different lighting conditions have the most significant impact on the accuracy of depth estimation. However, existing relighting methods typically perform global lighting transformations on the entire image, lacking a localized and controllable lighting editing mechanism for the target region. This makes it impossible to simulate directional lighting only on the target object's surface while keeping the background region unchanged.

[0034] Therefore, there is an urgent need for a reliable, realistic, and accurate binocular illumination data enhancement solution.

[0035] The binocular illumination data enhancement scheme of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0036] Figure 1 A flowchart illustrating a binocular illumination data enhancement method according to an embodiment of this application is shown. Figure 1 As shown, the binocular illumination data enhancement method of this application embodiment may include the following steps S10-S40.

[0037] For ease of description, the binocular illumination data enhancement method of this application embodiment is described using an electronic device as the execution subject. It should be understood that the execution subject of this application embodiment can also be a processor or chip in an electronic device, and this application embodiment does not impose any limitations.

[0038] Step S10: Acquire multiple sets of image data collected by the first structured light camera, the second structured light camera, and the binocular camera.

[0039] In one possible implementation, the electronic device can acquire multiple sets of image data captured by a first structured light camera, a second structured light camera, and a binocular camera. The first structured light camera, the second structured light camera, and the binocular camera form an image acquisition system. The binocular camera has a top-down view, and the viewpoints of the two structured light cameras intersect horizontally. During each image acquisition process, the top-down binocular camera acquires a top-down binocular left image of the target area. And the top-down view of both eyes (right image) The two images mentioned above are grayscale images, which can be used as input sources for image enhancement and training inputs for generative adversarial networks. The first and second structured light cameras acquire two grayscale reference images of the scene from intersecting viewpoints. and the corresponding depth map It is used to provide cross-view lighting guidance information and stereo depth truth.

[0040] Optionally, the first structured light camera included in the image acquisition system can be used as the main structured light camera. The electronic device can control the image acquisition system to acquire image data multiple times. Each set of image data acquired can include a top-view binocular left image and a top-view binocular right image acquired by the binocular camera, two reference images acquired by the first structured light camera and the second structured light camera respectively, and a depth map acquired by the first structured light camera as the main structured light camera.

[0041] In some embodiments, the electronic device can pre-calibrate the extrinsic parameters of the three cameras in the image acquisition system offline to obtain a calibration matrix. This calibration matrix is ​​used to transform the spatial positional relationship of the first structured light camera, the second structured light camera, and the binocular camera. The calibration matrix includes a transformation matrix from the first structured light camera to the second structured light camera. And the transformation matrix from the first structured light camera to the left eye camera in the binocular camera system. Therefore, the calibration matrix determination process can be achieved in two stages. The first stage involves stereo calibration of the first and second structured light cameras to obtain the rotation matrix. Translation vector The second stage involves stereo calibration of the first structured light camera and the left eye camera in the binocular camera system to obtain the rotation matrix. Translation vector .

[0042] Step S20: Determine the corresponding disparity true value based on the data of each set of images and the calibration matrix.

[0043] In one possible implementation, after a calibration matrix is ​​pre-determined, the electronic device can simultaneously acquire multiple sets of image data using three cameras. After each acquisition, the depth map is converted into the disparity ground truth in a binocular coordinate system using the calibration matrix. This process allows the 3D points in the depth map to be mapped to the binocular camera coordinate system through rotation and translation transformations, thereby converting them into disparity ground truth. Used for subsequent geometric consistency constraints.

[0044] Optionally, the electronic device can determine the corresponding 3D point cloud based on the depth map. The 3D point cloud is projected onto the camera coordinate system corresponding to the left eye of the binocular camera using a calibration matrix to obtain the true 3D coordinates of each point in the camera coordinate system. The corresponding parallax ground truth value is then determined based on the true 3D coordinates of each point.

[0045] Step S30: Train an adversarial network based on the graph data of each group and the corresponding disparity ground truth.

[0046] In one possible implementation, after determining multiple sets of image data and the corresponding ground truth disparity values ​​for each set, the electronic device trains a generative adversarial network (GAN) based on the image data and the corresponding ground truth disparity values. This GAN includes a generator network and a conditional discriminator network, and the training process primarily trains the network parameters of the GAN. This training process can begin by segmenting the image data for each set to obtain a left image mask corresponding to the top-view binocular left image and a reference image mask corresponding to the reference image. Then, the left image mask and the reference image mask are aligned across viewing angles to obtain an effective aligned mask. Based on the reference image and the effective aligned mask, cross-view illumination decoupling is performed to obtain illumination conditional codes. The generator network then determines the generated binocular left image and the generated binocular right image based on the top-view binocular left image, the top-view binocular right image, the left image mask, and the illumination conditional codes. Finally, the generated image loss is determined based on the current set of image data, the left image mask, the illumination conditional codes, the generated binocular left image, and the generated binocular right image, and the parameters of the generator network are adjusted based on the generated image loss.

[0047] Optionally, the electronic device first performs target segmentation and cross-view mask alignment for each set of image data. The target segmentation process can be implemented through a target segmentation network, which is a pre-trained semantic segmentation network, for top-view binocular left images. and reference images acquired by structured light cameras Pixel-level segmentation is performed separately. This process is applied to the top-view binocular left image. The segmented output left image mask This is used to identify the area where a target object is located from a top-down view. (Referencing a reference image.) Reference mask for the split output structured light camera This is used to identify the regions where the same target object is located under cross-viewpoints. The target segmentation network freezes its parameters during training and does not participate in backpropagation for generative adversarial training. The cross-viewpoint mask alignment process can utilize the calibration matrix obtained from multi-camera extrinsic calibration, and align the mask from the left image... Projected onto the structured light camera's viewpoint, and compared with the reference image mask. Find the intersection to obtain the aligned effective mask of the structured light camera's effective target region. This operation ensures that only the target area in the reference image that is co-viewed with the binocular field of view is used for illumination information extraction, eliminating background interference from cross-viewpoints, so that the illumination features extracted by the subsequent basic illumination encoder only reflect the real illumination environment of the target surface.

[0048] In some embodiments, after determining the image mask, the electronic device encodes the reference image and the aligned effective mask using a basic illumination encoder to obtain a basic illumination vector. An active illumination vector is then obtained by encoding the image using an active illumination encoder according to preset illumination parameters. Finally, the basic illumination vector and the active illumination vector are fused to obtain the illumination conditional code.

[0049] This lighting condition encoding process can be achieved through a basic lighting encoder. Active light encoder This basic lighting encoder is implemented using three modules: a base layer, a blending layer, and a fusion layer. Image of the target region from a structured light camera after cross-view mask alignment As input, the light vector is progressively downsampled through four convolutional blocks, each containing Conv2d-BatchNorm-ReLU and max pooling. Finally, it is output as a 256-dimensional basic illumination vector after global average pooling and a fully connected layer. This encoder is jointly optimized with the generator during training, learning to ignore scene textures and retain only lighting attributes. Active Lighting Encoder Receive user-preset active lighting parameters, which may include the azimuth angle of the light source. Angle of elevation and light source intensity The normalized illumination parameters are expressed in vector form. A 64-dimensional active illumination vector is output through a two-layer fully connected network (3→64→64). The activation function is ReLU. The fusion layer will... and The vectors are concatenated into a 320-dimensional vector, then passed through a fully connected layer (320→256) and ReLU activation to output the illumination condition code. This is used for layer-by-layer injection into the subsequent generated network.

[0050] In some embodiments, the generator network employs an encoder-decoder architecture, which may include a binocular encoder and an illumination modulation decoder. The electronic device can extract multi-scale content features from the top-view binocular left image and the top-view binocular right image using the binocular encoder. The illumination modulation decoder determines the left-heavy illumination image and the right-heavy illumination image based on the multi-scale content features and illumination condition encoding. A masking and blending operation is performed on the top-view binocular left image, the top-view binocular right image, the left image mask, the left-heavy illumination image, and the right-heavy illumination image to obtain the generated binocular left image and the generated binocular right image.

[0051] The binocular encoder comprises two branches, left and right, which respectively process the top-view binocular left image. And the top-down view of both eyes (right image) Multi-scale content features are extracted. Each branch consists of 5 convolutional blocks, each containing Conv2d-BatchNorm-ReLU and downsampling with a stride of 2, with the number of channels decreasing sequentially from 64 to 128 to 256 to 512. Weights are shared between the left and right branches to ensure consistent content features are extracted at corresponding disparity positions. The illumination modulation decoder progressively upsamples from the deepest features, for a total of 5 levels. Each level contains bilinear upsampling (×2), concatenation of skip connection features with the corresponding level from the encoder, and a Conv2d-BatchNorm-ReLU convolutional block. An AdaIN illumination modulation layer (illumination conditional coding) is inserted after each convolutional layer. Scaling factor generated via fully connected layer and offset factor By performing a linear transformation on each channel of the feature map after normalization, illumination information can be injected at each scale.

[0052] The mask blending operation uses the mask in the left image obtained from the target segmentation network. , output the decoder , Pixel-level fusion with the original image yields the generated binocular left image. And generate the right image of binoculars This operation ensures that only the area where the target object is located in the top-down binocular view undergoes lighting transformation, while the pixel values ​​of the background area are completely consistent with the original image, and the generating network only needs to focus on the lighting simulation of the target area.

[0053] During training, after the electronic device determines the generated binocular left and right images corresponding to each set of image data, it can determine the generated image loss based on the image data of this set, the left image mask, the illumination conditional encoding, and the generated binocular left and right images. The parameters of the generation network are then adjusted based on this loss. This process determines the geometric loss based on the image data of this set, the left image mask, and the generated binocular left and right images. One item from the top-view binocular left image, top-view binocular right image, generated binocular left image, and generated binocular right image, along with the illumination conditional encoding, is input into the conditional discriminant network to obtain the corresponding ground truth image score. Adversarial loss is calculated based on the type of input image and the corresponding ground truth image score. The generated image loss is determined by the weighted sum of the geometric loss and the adversarial loss, and the parameters of the generation network are adjusted based on this loss.

[0054] Optionally, the geometric loss during training may include three constraints: disparity consistency constraint, left-right eye brightness consistency constraint, and background invariance constraint. The disparity consistency constraint is determined using a pre-trained differentiable disparity estimation network, PSMNet, to generate the binocular left image. And generate the right image of binoculars Input: parallax The true disparity value obtained by calibration transformation. Calculate smoothed L1 loss , After each forward propagation of the generator network, the generated binocular image pairs are fed into the stereo matching network with frozen parameters. The output is supervised by the ground truth disparity, and the generator network is constrained from changing the disparity structure of the target region from the perspective of stereo geometry.

[0055] Left and right gaze consistency constraints utilize parallax truth values Generate binocular right image By warping the bilinear sampling to the left eye view, and Calculate L1 loss , Based on the ground truth parallax, we verify whether the luminance changes of corresponding pixels in the left and right eyes are synchronized after relighting. If the generator network modifies the illumination of the same object point in the left and right eyes inconsistently, warping will produce a large luminance error, thus forcing the generator network to learn the stereo consistency of illumination.

[0056] Background invariance constraints enforce pixel identity in non-target regions. , This mechanism operates directly in the pixel space, penalizing any deviations from the target region to ensure the generator network strictly outputs the original values ​​in the background region without introducing artifacts. In each iteration of the generator, this mechanism calculates three geometric constraint losses, which are then weighted and summed with the adversarial loss before backpropagation to update the generator network's parameters.

[0057] Alternatively, the adversarial loss can be determined using a conditional discriminant network. The PatchGAN structure is adopted, consisting of 5 convolutional blocks: Conv2d-BatchNorm-LeakyReLU(0.2). The number of channels in each convolutional block is 64→128→256→512→1, with a stride of 2 for each block. The last layer outputs... The score map represents the true / false judgment result of a local block in the input image at each location. The input to the conditional discriminant network is the joint input of the image and the illumination conditions. The 256-dimensional illumination conditions are encoded. The image is expanded into a feature map with the same spatial size as the input image through a fully connected layer, and then this feature map is concatenated with the image along the channel dimension. Taking a single-channel grayscale image as an example, the concatenated dimension is 257 channels, and both are fed into the conditional discriminant network. This design enables the conditional discriminant network to simultaneously perceive the lighting conditions when judging authenticity, forcing the relit image output by the generator network to be both realistic and natural within the target area, and consistent with the specified lighting parameters.

[0058] The adversarial loss between the conditional discriminant network and the generator network adopts the hinge loss form. , ,in For real images, To generate an image. In the generator's total loss, the total geometric consistency loss... and the loss of combat Weighted summation: .

[0059] Step S40: Based on the trained generative adversarial network, generate multiple enhanced image data corresponding to each group of image data according to preset illumination parameters.

[0060] In one possible implementation, after determining the trained generative adversarial network (GAN), the electronic device can generate multiple enhanced image data corresponding to each set of image data based on the trained GAN and preset illumination parameters. This process involves sampling different light source azimuth angles multiple times for each set of original binocular images within the preset illumination parameter range. Angle of elevation and light source intensity The scene is generated in batches by successively inputting into the generator network. Binocular image pairs under different lighting conditions are used as augmented image data. Since the background region is masked and blended to maintain its original value, and the target region's disparity is geometrically constrained to maintain its invariance, the ground truth depth values ​​of all generated images are consistent with the original values. Figure 1 It can be reused directly.

[0061] Optionally, an augmented training set is formed by combining the original stereo images, enhanced image data, and corresponding ground truth depth values, and then input into the stereo depth estimation network for training. This training set covers various lighting directions and intensities while preserving scene content and geometric structure, enabling the depth estimation network to learn robust depth inference capabilities in response to lighting changes.

[0062] In some embodiments, this solution can be used for binocular depth estimation data augmentation in industrial robot grasping scenarios. The following will use the application scenario of an industrial robot grasping various metal and plastic parts in a bin as an example to describe this solution in detail.

[0063] Optionally, the process can begin with hardware system configuration: Top-down binocular camera: Intel RealSense D435, resolution 1024×720, frame rate 30fps, baseline length 50mm. Structured light camera: Two Mech-Mind cameras, positioned at the upper left and upper right of the bin, respectively, with an intersecting viewpoint with the binocular camera, resolution 1280×1024. Computing platform: NVIDIA RTX 4090 GPU. Light source: Fixed strip LED light source at the top of the scene; the actual light source position will not be changed during the training phase.

[0064] Then, system calibration and data acquisition are performed. This process can be completed in two steps using a 15mm checkerboard calibration board: first, the transformation matrix between the two structured light cameras is calibrated, and then the transformation matrix from the structured light camera to the left binocular eye is calibrated, with reprojection errors controlled within 0.1 pixels. After calibration, 1000 sets of data are simultaneously acquired in the bin scene. Each set includes binocular left and right images, two structured light images, and one depth map, and the depth map is converted into the parallax ground truth in the binocular coordinate system.

[0065] Next, segmentation network training and fine-tuning, along with mask generation, are performed. This process annotates target grasping part masks on 500 binocular left-side images. The pre-trained Mask R-CNN is then fine-tuned, achieving an mIoU of 92.6%. The target mask for the left-side image is then obtained through inference on all the data. and structured light target mask The effective mask after alignment is obtained by intersecting the extrinsic parameters through projection. This is used for training the generative network, which employs a shared-weight binocular encoder-decoder architecture. Each layer of the decoder is injected with illumination conditional encoding via AdaIN. The basic illumination encoder extracts 256 dimensions from the structured light target region. The active light encoder transmits the lighting parameters. Mapped to 64 dimensions The two are fused together as a conditional code. The discriminative network uses PatchGAN, receiving the joint input of the image and the conditional code. Training employs the WGAN-GP strategy, with a batch size of 8, a learning rate of 0.0002, and 200 training epochs. Loss weights: , , , After training convergence, five sets of illumination parameters were sampled for each set of original data, generating a total of 5000 sets of multi-illumination binocular images. The background region of the generated images was consistent with the original image pixel by pixel, and the average EPE of the disparity and ground truth was 0.87 pixels. The augmented data and the original data were merged to train the PSMNet depth estimation network. On 100 sets of multi-illumination test data, the EPE decreased from 2.34 pixels to 1.21 pixels (2.58 pixels for the pix2pix independent reillumination method), an improvement of 48.3%; under extreme side lighting conditions, the EPE decreased from 4.67 pixels to 1.89 pixels (5.12 pixels for the pix2pix independent reillumination method).

[0066] In other embodiments, this solution can also be used to enhance binocular depth estimation data for service robots in indoor scenarios. Service robots are equipped with top-down binocular cameras and operate under mixed indoor lighting conditions of natural light and multiple light sources. Due to variations in the direction of natural light at different times and the fixed positions of indoor lighting fixtures, the collected training data suffers from insufficient illumination coverage, leading to a decrease in the accuracy of the depth estimation network under specific lighting conditions. This invention is applied to the depth estimation task of service robots on objects of different materials in indoor environments to improve the depth estimation accuracy under specific lighting conditions.

[0067] In this scenario, the top-view binocular camera remains unchanged. Two structured light cameras are now fixedly positioned on either side of the robot's working area, continuously acquiring reference images from a cross-view perspective. The computing platform is switched to NVIDIA Jetson AGXOrin, with the inference network optimized with TensorRT to achieve a latency of 18ms. The target segmentation network is replaced with DeepLabV3+ using the lightweight MobileNetV3 backbone to adapt for edge device inference, achieving an mIoU of 89.1%. The active lighting parameters are adjusted according to the indoor scene: elevation angle... The system simulates the angle range between ceiling lights and natural light from side windows. For various interior materials (wooden tabletops, metal tableware, ceramic cups, plastic storage boxes), corresponding category labels are added to the segmented network.

[0068] Experimental results show that on a multi-material test set of 200 sets, the depth estimation network trained with augmented data achieves an EPE of 1.45 pixels, a 46.5% improvement compared to 2.71 pixels trained with the original data. The reduction in depth estimation error is particularly significant in material boundary regions, with the EPE decreasing from 3.82 pixels to 1.93 pixels. The inference latency of the generator network on Jetson AGX Orin is 18ms / pair, meeting the requirements for real-time data augmentation.

[0069] Based on the aforementioned technical features, this application's embodiments achieve low-cost, high-quality expansion of binocular depth estimation training data in the illumination dimension through three core mechanisms: cross-view illumination decoupling, semantic mask-guided local generation, and binocular geometric consistency constraints. In terms of accuracy and generalization, the enhanced depth estimation network exhibits low disparity prediction error under both normal and extreme side lighting conditions, significantly improving the model's robustness to illumination changes. Regarding ground truth reuse, the triple constraints of disparity, luminance, and background force the generated image to maintain the original scene's stereo geometry, resulting in low disparity error. This allows all enhanced samples to directly reuse the original depth ground truth, completely avoiding the high cost of manual re-annotation. In terms of imaging quality, the mask blending mechanism ensures that the background region strictly maintains the original pixels, eliminating artifacts easily introduced by global re-illumination methods and making the network training signal purer. Simultaneously, active illumination parameter encoding provides users with the flexible and controllable ability to generate directional illumination samples with arbitrary azimuth, elevation, and intensity as needed. The overall solution has strong hardware adaptability and is widely applicable to diverse visual perception scenarios such as industrial disordered grasping, indoor navigation of service robots, warehousing and logistics, and autonomous driving. It provides a highly practical solution for training a robust depth estimation model with limited real-world data.

[0070] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0071] Based on the foregoing embodiments, this application provides a binocular illumination data enhancement device, which includes various modules and units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0072] Figure 2 A schematic diagram of a binocular illumination data enhancement device according to an embodiment of this application is shown. Figure 2 As shown, the binocular illumination data enhancement device according to an embodiment of this application includes: Data acquisition module 20 is used to acquire multiple sets of image data collected by the first structured light camera, the second structured light camera and the binocular camera; The parameter calculation module 21 is used to determine the corresponding parallax true value based on each set of image data and calibration matrix. The calibration matrix is ​​used to convert the spatial position relationship between the first structured light camera, the second structured light camera and the binocular camera. Network training module 22 is used to train and generate adversarial networks based on the graph data of each group and the corresponding disparity ground truth. The image generation module 23 is used to generate multiple enhanced image data corresponding to each group of image data according to preset illumination parameters based on the trained generative adversarial network.

[0073] In one possible implementation, each set of image data includes a top-view binocular left image and a top-view binocular right image acquired by the binocular camera, two reference images acquired by the first structured light camera and the second structured light camera respectively, and a depth map acquired by the first structured light camera as the main structured light camera.

[0074] In one possible implementation, the parameter calculation module 21 is further configured to: Determine the corresponding 3D point cloud based on the depth map; The three-dimensional point cloud is projected onto the camera coordinate system corresponding to the left eye camera in the binocular camera according to the calibration matrix to obtain the true three-dimensional coordinates of each point in the camera coordinate system. The true disparity value is determined based on the actual three-dimensional coordinates of each point.

[0075] In one possible implementation, the network training module 22 is further configured to: Target segmentation is performed on each group of image data to obtain the left image mask corresponding to the top-view binocular left image and the reference image mask corresponding to the reference image; The left image mask and the reference image mask are aligned across viewpoints to obtain an effectively aligned mask. Based on the reference image and the aligned effective mask, cross-view illumination decoupling is performed to obtain illumination condition coding; The generator network determines the generated binocular left image and the generated binocular right image based on the top-view binocular left image, the top-view binocular right image, the left image mask, and the lighting condition encoding. The generated image loss is determined based on the image data in this set, the left image mask, the illumination condition encoding, the generated binocular left image, and the generated binocular right image, and the parameters of the generating network are adjusted according to the generated image loss.

[0076] In one possible implementation, the network training module 22 is further configured to: The reference image and the aligned effective mask are encoded by a basic illumination encoder to obtain a basic illumination vector; An active lighting vector is obtained by encoding the active lighting parameters using an active lighting encoder. The illumination condition code is obtained by fusing the base illumination vector and the active illumination vector.

[0077] In one possible implementation, the network training module 22 is further configured to: Multi-scale content features of the top-view binocular left image and the top-view binocular right image are extracted using a binocular encoder; The left heavy illumination image and the right heavy illumination image are determined by the illumination modulation decoder based on the multi-scale content features and the illumination condition coding; A masking and blending operation is performed on the top-view binocular left image, the top-view binocular right image, the left image mask, the left heavy illumination image, and the right heavy illumination image to obtain the generated binocular left image and the generated binocular right image.

[0078] In one possible implementation, the network training module 22 is further configured to: The geometric loss is determined based on the data in this set of images, the left image mask, the generated binocular left image, and the generated binocular right image. By combining one of the top-view binocular left image, the top-view binocular right image, the generated binocular left image, and the generated binocular right image with the illumination condition encoding, the corresponding real image score is obtained by inputting the network into the condition discrimination network. Calculate the adversarial loss based on the type of the input image and the corresponding real image score; The generated image loss is determined based on the weighted sum of the geometric loss and the adversarial loss, and the parameters of the generating network are adjusted based on the generated image loss.

[0079] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0080] It should be noted that, in the embodiments of this application... Figure 2 The module division of the binocular illumination data enhancement device shown is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or be integrated into one unit by two or more units. The integrated units can be implemented in hardware, as software functional units, or a combination of both.

[0081] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0082] Figure 3 A schematic diagram of an electronic device according to an embodiment of this application is shown. For example... Figure 3 As shown in the figure, this application provides an electronic device, which can be a server, and its internal structure diagram can be as follows. Figure 3 As shown, the electronic device includes a processor 320, a memory, and a transceiver 340 connected via a system bus 310. The processor 320 provides computing and control capabilities. The memory includes a non-volatile storage medium 331 and internal memory 332. The non-volatile storage medium 331 stores an operating system, computer programs, and a database. The internal memory 332 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium 331. The database stores data. The transceiver 340 communicates with external terminals via a network connection. The computer program is executed by the processor 320 to implement the aforementioned methods.

[0083] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor 320, implements the steps of the method provided in the above embodiments.

[0084] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0085] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0086] In one possible implementation, the apparatus provided in this application can be implemented as a computer program, which can be configured as follows: Figure 3 The device operates on the electronic device shown. The memory of the electronic device can store the various program modules that make up the above-described apparatus. The computer program composed of the various program modules causes the processor 320 to execute the steps of the methods in the various embodiments of this application described in this specification.

[0087] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0088] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, phrases such as "in one possible implementation," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0089] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0090] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0092] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0093] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0094] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0095] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0096] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0097] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0098] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0099] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for binocular illumination data augmentation, characterized in that, The method includes: Acquire multiple sets of image data collected by the first structured light camera, the second structured light camera, and the binocular camera; The corresponding parallax true value is determined based on each set of image data and calibration matrix. The calibration matrix is ​​used to transform the spatial position relationship between the first structured light camera, the second structured light camera and the binocular camera. Generative adversarial networks are trained based on the graph data and corresponding disparity ground truth values ​​of each group. Based on the trained generative adversarial network, multiple enhanced image data corresponding to each group of image data are generated according to preset lighting parameters.

2. The method according to claim 1, characterized in that, Each set of image data includes a top-view binocular left image and a top-view binocular right image acquired by the binocular camera, two reference images acquired by the first structured light camera and the second structured light camera respectively, and a depth map acquired by the first structured light camera, which serves as the main structured light camera.

3. The method according to claim 2, characterized in that, The step of determining the corresponding disparity ground truth value based on each set of image data and calibration matrix includes: Determine the corresponding 3D point cloud based on the depth map; The three-dimensional point cloud is projected onto the camera coordinate system corresponding to the left eye camera in the binocular camera according to the calibration matrix to obtain the true three-dimensional coordinates of each point in the camera coordinate system. The true disparity value is determined based on the actual three-dimensional coordinates of each point.

4. The method according to claim 2, characterized in that, The step of training a generative adversarial network based on the graph data of each group and the corresponding disparity ground truth includes: Target segmentation is performed on each group of image data to obtain the left image mask corresponding to the top-view binocular left image and the reference image mask corresponding to the reference image; The left image mask and the reference image mask are aligned across viewpoints to obtain an effectively aligned mask. Based on the reference image and the aligned effective mask, cross-view illumination decoupling is performed to obtain illumination condition coding; The generator network determines the generated binocular left image and the generated binocular right image based on the top-view binocular left image, the top-view binocular right image, the left image mask, and the lighting condition encoding. The generated image loss is determined based on the image data in this set, the left image mask, the illumination condition encoding, the generated binocular left image, and the generated binocular right image, and the parameters of the generating network are adjusted according to the generated image loss.

5. The method according to claim 4, characterized in that, The step of performing cross-view illumination decoupling based on the reference image and the aligned effective mask to obtain illumination condition encoding includes: The reference image and the aligned effective mask are encoded by a basic illumination encoder to obtain a basic illumination vector; An active lighting vector is obtained by encoding the active lighting parameters using an active lighting encoder. The illumination condition code is obtained by fusing the base illumination vector and the active illumination vector.

6. The method according to claim 4, characterized in that, The step of determining the generated binocular left image and the generated binocular right image through a generative network based on the top-view binocular left image, the top-view binocular right image, the left image mask, and the lighting condition encoding includes: Multi-scale content features of the top-view binocular left image and the top-view binocular right image are extracted using a binocular encoder; The left heavy illumination image and the right heavy illumination image are determined by the illumination modulation decoder based on the multi-scale content features and the illumination condition coding; A masking and blending operation is performed on the top-view binocular left image, the top-view binocular right image, the left image mask, the left heavy illumination image, and the right heavy illumination image to obtain the generated binocular left image and the generated binocular right image.

7. The method according to claim 4, characterized in that, The step of determining the generated image loss based on the image data of this set, the left image mask, the illumination condition encoding, the generated binocular left image, and the generated binocular right image, and adjusting the parameters of the generation network based on the generated image loss, includes: The geometric loss is determined based on the data in this set of images, the left image mask, the generated binocular left image, and the generated binocular right image. By combining one of the top-view binocular left image, the top-view binocular right image, the generated binocular left image, and the generated binocular right image with the illumination condition encoding, the corresponding real image score is obtained from the input condition discrimination network. Calculate the adversarial loss based on the type of the input image and the corresponding real image score; The generated image loss is determined based on the weighted sum of the geometric loss and the adversarial loss, and the parameters of the generating network are adjusted based on the generated image loss.

8. A binocular illumination data enhancement device, characterized in that, The device includes: The data acquisition module is used to acquire multiple sets of image data collected by the first structured light camera, the second structured light camera, and the binocular camera; The parameter calculation module is used to determine the corresponding parallax true value based on each set of image data and calibration matrix. The calibration matrix is ​​used to transform the spatial position relationship between the first structured light camera, the second structured light camera and the binocular camera. The network training module is used to train and generate adversarial networks based on the graph data of each group and the corresponding disparity ground truth. The image generation module is used to generate multiple enhanced image data corresponding to each group of image data based on the trained generative adversarial network and preset illumination parameters.

9. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images