CycleGAN-based sunny-snowy weather map image data style transfer system and method

Abstract

The application discloses a sunny-snowy weather image data style migration system based on CycleGAN, which comprises the following steps: a network construction module constructs a sunny-snowy weather image data style migration model based on CycleGAN; a mask module creates a mask according to image annotation information of an input image, and the mask is superimposed on the input image to input edge detection of a teacher network and a student network; the input image is subjected to style migration by a first generator to obtain a generated image, and the generated image is superimposed on the mask to input edge detection of the student network; the generated image is generated into a reconstructed image by a second generator, and the reconstructed image is superimposed on the mask to input edge detection of the student network; a model training module trains the constructed sunny-snowy weather image data style migration model; and an inference module receives a sunny weather image to generate a corresponding snowy weather image. The application can solve the technical problem of object loss in the existing style migration technology in sunny-snowy weather image data style migration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a CycleGAN (Cycle-Consistent Adversarial Networks)-based sunny-snowy weather image data style transfer system and method. BACKGROUND

[0002] The development of automatic driving technology cannot be separated from a large amount of real-world scene data support. In order to ensure the safety and reliability of the automatic driving system, the usability of the test and verification system under various possible scene conditions needs to be tested and verified to define the design and operation conditions that can truly reflect the system capability. However, the scarcity of data in harsh working conditions and long-tail scenes poses a challenge, and building a high-quality harsh weather dataset is a highly challenging and labor-, resource- and cost-intensive task. In particular, image data under snow conditions are extremely scarce, which limits the performance improvement of the automatic driving system. Therefore, researchers seek to expand the dataset through innovative methods to enhance the performance of the system in harsh weather. Style transfer technology is a method of converting images from one style to another, which is commonly used in artistic creation or data augmentation. In the field of automatic driving, this technology can be used to generate image data under snow conditions, thereby expanding the dataset. This not only solves the difficulty of collecting snow data, but also provides more abundant training materials for the automatic driving system. In addition, the new image data obtained by style transfer from the existing dataset can use the annotation information of the original dataset, greatly saving the labor, resources and costs required for collecting the dataset in the real world. However, existing style transfer techniques have some limitations when processing images under harsh weather conditions, such as object loss or deformation. The image annotation information in the automatic driving dataset provides precise boundaries for the positions and shapes of traffic participants, which is the key context for understanding traffic scenes. However, in the application of style transfer technology, the context information contained in these bounding boxes is often not fully utilized. If the style transfer algorithm can integrate image annotation information, it will be able to more accurately identify and preserve key traffic elements such as vehicles and pedestrians during the visual style conversion process of the image. In this way, the generated image not only maintains consistency with the style of the source image in terms of vision, but also better fits the actual traffic scene in terms of content, ensuring the authenticity and accuracy of image synthesis. In this way, the application of style transfer technology in the field of automatic driving will be more in-depth and extensive, providing more abundant and reliable visual data for the perception and decision-making of the automatic driving system.

[0003] Currently, the traditional CycleGAN model is an unsupervised image translation framework that enables image style transfer without paired training samples. It can handle various image translation tasks, including style transfer, object transformation, and seasonal transformation. Therefore, it can also be used for weather image data style transfer.

[0004] The technical core includes two mapping functions G and F, where G is responsible for converting images from the source domain X to images with the style of the target domain Y, while F performs the opposite conversion, i.e., converts images from the Y domain back to the X domain.

[0005] G: X→Y

[0006] F: Y→X

[0007] The training of these two mapping functions is guided by the combination of adversarial loss and cycle consistency loss, ensuring the quality of generated images and the reversibility of the conversion. In the CycleGAN model, D Y aims to distinguish between target domain images generated by G and real target domain images, while D X distinguishes between source domain images generated by F and real source domain images. In this way, the generators G and F continuously improve their performance by minimizing these two losses during the training process, generating high-quality images consistent with the target domain style. In addition, the cycle consistency loss of the CycleGAN model ensures that the images can be highly consistent after forward and backward conversion, i.e., F(G(x)) is close to the original image x, and G(F(y)) is close to the original image y.

[0008]

[0009]

[0010] Although the traditional CycleGAN optimizes the quality of generated images through cycle consistency loss, it still has the problem of object loss when applied to style transfer under harsh weather conditions. For example, under snow conditions, the car body may be covered with snow, causing significant changes in color and texture, which interferes with the generator of the CycleGAN model to correctly capture and transfer the style.

[0011] In the prior art, there is also HED NGAN model that combines a generative adversarial network and an image style transfer algorithm of a HED edge detector, is specially designed for converting daytime images into nighttime images. This technical solution trains a GAN generator to simulate the conversion between daytime and nighttime images, while using a differentiable HED edge detector to ensure that the converted nighttime images are consistent with the original daytime images in terms of edge structure. This solution uses a teacher-student architecture, which includes two edge detectors: one HED edge detector as the teacher network, and another HED edge detector as the student network N The edge detector is trained together with the GAN generator. The student network HED N The goal is to learn edge detection capabilities that work well on both daytime and nighttime images, while the teacher network HED provides fixed edge information to guide the training of the student network. In addition, the HED N GAN also introduces an edge consistency loss, which constrains the similarity between the edge map of the generated image and the input image at the pixel level through an L1 loss function, to improve the quality of the generated image.

[0012] HED N The GAN model achieves high consistency in day-night image style transfer by enforcing consistency between the input image and the generated image through edge consistency, but this model uses global edge detection, which requires a large amount of memory and computing power, so it still faces challenges in terms of computational cost and application generalization.

[0013] There is also an improved method based on CycleGAN for performing sunny-snowy weather image data style transfer, which introduces additional semantic information to guide image generation and uses deep supervision signals to optimize the discriminator. Specifically, the improved method based on CycleGAN uses a trained ViT-Adapter model to generate semantic segmentation maps, and uses SFT spatial feature transformation technology to fuse semantic information in a spatially sensitive manner. Additional supervision signals are introduced in the middle layer of the generator, allowing images of different sizes and proportions to be input into the discriminator simultaneously. However, the improved method based on CycleGAN is highly dependent on the accuracy of the trained semantic segmentation model ViT-Adapter, and if ViT-Adapter performs poorly in some complex or unusual scenarios, it will directly affect the quality of the final synthesized snow effect. Secondly, the model is mainly optimized for regular snow conditions, and lacks in-depth research and testing of snow effects under extreme conditions. SUMMARY

[0014] The purpose of the present application is to provide a sunny-snowy weather image data style transfer system and method based on CycleGAN, which can solve the technical problem of object loss in existing style transfer technology in sunny-snowy weather image data style transfer.

[0015] To achieve the above object, the application provides a CycleGAN-based sunny-snowy weather image data style transfer system, which comprises a mask module, a network construction module, a model training module and an inference module, wherein:

[0016] The network construction module comprises a CycleGAN backbone network and an edge detector, and is used for constructing a CycleGAN-based sunny-snowy weather image data style transfer model; the CycleGAN backbone network has a first generator Generator X2Y and a second generator Generator Y2X; the edge detector has a pre-trained teacher network PiDiNet and a student network PiDiNet N ; wherein the mask module creates a mask Mask according to image annotation information of an input image Input image, and the mask Mask and the input image Input image are superimposed and input to the teacher network PiDiNet and the student network PiDiNet N for edge detection; the input image Input image is input to the first generator Generator X2Y for style transfer, to obtain a generated image Translated image, and the generated image Translated image and the mask Mask are superimposed and input to the student network PiDiNet N for edge detection; the generated image Translated image is input to the second generator Generator Y2X to generate a reconstructed image Reconstruction image, and the reconstructed image Reconstruction image and the mask Mask are superimposed and input to the student network PiDiNet N for edge detection again;

[0017] The model training module is used for training the constructed sunny-snowy weather image data style transfer model, and uses a comprehensive loss function composed of an adversarial loss described by formula (1) a cycle consistency loss a local edge consistency loss and a distillation loss to ensure that the generated snowy weather image is highly consistent in structure with the sunny weather image;

[0018]

[0019] wherein λ represents a weight parameter corresponding to the cycle consistency loss;

[0020] ​The reasoning module is configured to apply the trained sunny weather image data style transfer model to a new sunny weather image, receive the sunny weather image, and generate a corresponding snowy weather image.

[0021] Further, the mask module specifically includes:

[0022] The annotation information acquisition unit is configured to extract and parse annotation information corresponding to the original image, and determine whether the annotation information exists <object>If the element is true, the pixel coordinate values of the corners of the traffic participant bounding box are obtained and stored in the list boxes; otherwise, an empty list is directly generated;

[0023] An initial mask setting unit is configured to create a full zero matrix with the same size as the original image, which is used to represent the initial mask mask_0. The information of the traffic participant bounding box in the list boxes is traversed, and the pixel values in the region of the traffic participant bounding box are set to 255 to mark the traffic participant

[0024] A mask image setting unit is configured to copy the single-channel initial mask mask_0 to the red, green and blue color channels of the RGB image, and create a three-channel mask image mask_rgb with a shape of (H, W, 3):

[0025] mask_rgb = np.stack((mask_0, mask_0, mask_0), axis=-1)

[0026] Wherein, axis =-1 represents operation along the last dimension of the array, H is the height, and W is the width.

[0027] Further, the mask module specifically further includes:

[0028] An image preprocessing unit is configured to perform preprocessing operations on the three-channel mask image mask_rgb and the original image synchronously: fix the aspect ratio to 288x512 pixels, then convert the mask and the original image format from tensor to numpy array through the tensor2im function, and apply the three-channel mask image mask_rgb to the original image through bit operation to retain only the internal information of the traffic participant bounding box.

[0029] Further, the model training module learns and imitates the edge detection behavior of the teacher network PiDiNet through the process of knowledge distillation to adapt to the edge detection task under snow conditions, specifically including:

[0030] An image edge detection unit is configured to perform local edge detection on the input image Input image through the teacher network PiDiNet in the forward propagation stage to obtain a first edge detection image E(I teacher-Input ); through the student network PiDiNet N Perform local edge detection on the input image Input image, the generated image Translated image and the reconstructed image Reconstruction image to obtain a second edge detection image E((I student-Input ), and a third edge detection image E((I student-Translated ), a fourth edge detection image E(I student-Reconstruction );

[0031] an edge mapping difference calculation unit for obtaining a local edge consistency loss L edge by calculating edge consistency loss between E(I teacher-Input ) and E((I student-Translated )), and edge consistency loss between E(I teacher-Input ) and E(I student-Reconstruction ); a distillation loss L distillation obtained by calculating edge consistency loss between E(I teacher-Input ) and E((I student-Input ));

[0032] Further, the local edge consistency loss L edge is obtained by calculating edge consistency loss between E(I the distillation loss L distillation of the edge detection module respectively described by formula (2), (3) to obtain a pixel-by-pixel calculation L1 loss;

[0033]

[0034] wherein, is edge consistency loss between E(I teacher-Input ) and E((I student-Translated ), is edge consistency loss between E(I teacher-Input ) and E(I student-Reconstruction ), is edge consistency loss between E(I teacher-Input ) and E((I student-Input )), and α1, α2, α3 are weight parameters of the edge consistency loss terms respectively.

[0035] The application also provides a sunny-snowy weather image data style transfer method based on CycleGAN, which comprises the following steps:

[0036] Step 1: constructing a sunny-snowy weather image data style transfer model based on CycleGAN by a network construction module: the network construction module comprises a CycleGAN backbone network and an edge detector, the CycleGAN backbone network has a first generator Generator X2Y and a second generator Generator Y2X; the edge detector has a pre-trained teacher network PiDiNet and a student network PiDiNet N ; wherein a mask Mask is created according to image annotation information of an input image Input image, and the mask Mask and the input image Input image are superimposed and then input to the teacher network PiDiNet and the student network PiDiNet N Edge detection is performed; the input image Input image is subjected to style transfer by a first generator Generator X2Y to obtain a generated image Translated image, and the generated image Translated image is superimposed with a mask Mask and then input to the student network PiDiNet N Edge detection is performed; the generated image Translated image is subjected to a second generator Generator Y2X to obtain a reconstructed image Reconstruction image, and the reconstructed image Reconstruction image is superimposed with the mask Mask and then input to the student network PiDiNet again N Edge detection is performed;

[0037] Step 2, training the constructed sunny-snowy weather image data style transfer model, using the adversarial loss described by formula (1) Cycle consistency loss Local edge consistency loss And distillation loss The comprehensive loss function is composed of To ensure that the generated snowy weather image is highly consistent in structure with the sunny weather image;

[0038]

[0039] Wherein, λ represents the weight parameter corresponding to the cycle consistency loss;

[0040] Step 3, applying the trained sunny-snowy weather image data style transfer model to a new sunny weather image, receiving the sunny weather image, and generating a corresponding snowy weather image.

[0041] Further, the method for creating a mask Mask according to the image annotation information of the input image Input image specifically includes:

[0042] Step 11, extracting and parsing the annotation information corresponding to the original image, and determining whether the annotation information exists <object>If yes, the pixel coordinate values of the corners of the traffic participant bounding box are obtained and stored in the list boxes; otherwise, an empty list is directly generated;

[0043] Step 12, a full zero matrix with the same size as the original image is created to represent the initial mask mask 0, and the information of the traffic participant bounding box in the list boxes is traversed, and the pixel values in the region of the traffic participant bounding box are set to 255 to mark the traffic participant

[0044] Step 13, the single-channel initial mask mask 0 is copied to the red, green and blue color channels of the RGB image to create a three-channel mask image mask rgb with a shape of (H, W, 3):

[0045] mask rgb=np.stack((mask 0,mask 0,mask 0),axis=-1)

[0046] Wherein, axi s=-1 represents operation along the last dimension of the array, H is the height, and W is the width.

[0047] Further, the method for creating a mask Mask according to the image annotation information of the input image Input image specifically further comprises:

[0048] Step 14, the three-channel mask image mask rgb and the original image are preprocessed synchronously: fixed aspect ratio is reduced to 288*512 pixels, and then the mask and the original image format are converted from tensor to numpy array through the tensor2im function, and the three-channel mask image mask rgb is applied to the original image, and only the internal information of the traffic participant bounding box is retained through bit operation.

[0049] Further, step 2 learns and imitates the edge detection behavior of the teacher network PiDiNet through the process of knowledge distillation to adapt to the edge detection task under snow conditions, specifically comprising:

[0050] Step 21, in the forward propagation stage, the input image Input image is subjected to local edge detection through the teacher network PiDiNet to obtain a first edge detection image E(I teacher-Input ); the student network PiDiNet N is subjected to local edge detection on the input image Input image, the generated image Translated image and the reconstructed image Reconstruction image to obtain a second edge detection image E((I student-Input ), a third edge detection image E((I student-Translated )), the fourth edge detection image E(I) student-Reconstruction );

[0051] Step 22, for E(I) teacher-Input ) and E((I student-Translated The edge consistency loss between E(I) and E(I) teacher-Input ) and E(I student-Reconstruction The edge consistency loss between () is used to obtain the local edge consistency loss. to E(I teacher-Input ) and E((I student-Input The edge consistency loss between )) is used to obtain the distillation loss.

[0052] Furthermore, local edge consistency loss Distillation loss of edge detection module The L1 loss is obtained by calculating it pixel-by-pixel using equations (2) and (3), respectively.

[0053]

[0054] in, for E(I teacher-Input ) and E((I student-Translated Edge consistency loss between )) for E(I teacher-Input ) and E(I student-Reconstruction Edge consistency loss between ) for E(I teacher-Input ) and E((I student-Input The edge consistency loss between )) is α1, α2, and α3, which are the weight parameters of the edge consistency loss term.

[0055] The present invention has the following advantages due to the adoption of the above technical solutions:

[0056] The technical solution proposed in this invention primarily addresses the problem of object loss during style transfer of image data in clear and snowy weather. By introducing edge detection technology, this invention can improve the quality of the generated image and the accuracy of style transfer while maintaining the consistency of traffic participant objects in the image. Furthermore, this invention avoids the high computational cost of global edge detection by focusing on the region within the bounding box, significantly reducing memory and computational requirements. Simultaneously, this invention fully utilizes the annotation information in autonomous driving datasets; by combining local edge detection with annotation information, it further improves the efficiency of dataset utilization and the effect of image style transfer. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the CycleGAN model according to an embodiment of the present invention.

[0058] Figure 2 is a sunny weather style migration effect schematic diagram of an embodiment of the present application.

[0059] Figure 3 is a mask effect schematic diagram of an embodiment of the present application.

[0060] Figure 4 is a model construction and training process schematic diagram of an embodiment of the present application.

[0061] Figure 5 is a structure schematic diagram of a sunny weather image data style migration system of an embodiment of the present application. DETAILED DESCRIPTION

[0062] In the drawings, the same or similar notations are used to indicate the same or similar elements or elements having the same or similar functions. The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0063] In the description of the present application, the terms "center", "longitudinal", "transverse", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the scope of protection of the present application.

[0064] As shown in Figure 5 , the sunny weather image data style migration system based on CycleGAN provided by the embodiment of the present application comprises a mask module, a network construction module, a model training module and an inference module, wherein:

[0065] The network construction module is used to construct a sunny weather image data style migration model based on CycleGAN, and the sunny weather image data style migration model is used to receive sunny weather images and generate corresponding snowy weather images.

[0066] In this embodiment, the network construction module has a CycleGAN backbone network, and based on the framework of the CycleGAN backbone network, the image annotation information is fully utilized to solve the object loss problem that occurs when the image style is migrated under sunny and snowy weather.

[0067] Before model training, images and their annotation information under sunny and snowy conditions need to be extracted from the dataset first to construct the training set and the test set. In an embodiment, in order to perform the sunny-snowy weather image data style transfer task under the framework of CycleGAN, 5250 snowy images and 5250 sunny images are extracted from the CADC dataset and the BDD100K dataset to form a new dataset, and the sunny and snowy weather training sets each contain 4725 images, and the test sets each contain 525 images.

[0068] The CADC dataset is the first automatic driving dataset focusing on cold driving environment in the world, containing 56000 camera images, covering 75 scenes, providing 10 annotation categories, suitable for researching and developing target detection algorithms for automatic driving vehicles in extreme weather conditions such as ice and snow, and can also be used for sunny-snowy weather image data style transfer. According to the number of corresponding laser radar points, at least 20 corresponding image data with laser radar points are obtained, and images with poor quality are manually deleted, to obtain 5250 snowy images.

[0069] The BDD100K dataset is one of the largest and most diverse automatic driving datasets, containing 100,000 images, and detailed annotations, covering rich geographical, environmental and weather conditions, and is an important resource for evaluating and developing image recognition technologies for automatic driving vehicles. According to the annotations in the dataset, images that meet the sunny and white weather conditions are extracted, and images with poor quality are manually deleted, to obtain 5250 sunny images.

[0070] It should be noted that the extracted image types require that there is at least one traffic participant in the image. The quantity requirement includes that the number of images in the dataset in the field cannot be too small, and needs to be sufficient. The training set and the test set are divided according to the commonly used division ratio in the field, for example, 9:1.

[0071] Further, in order to perform the sunny-snowy weather image data style transfer task under the framework of CycleGAN, the embodiment takes the sunny weather style as domain X and the snowy weather style as domain Y, and performs re-arrangement and size pre-processing on the images to adapt to the requirements of the CycleGAN backbone network on the structure of the dataset. The re-arrangement method includes: dividing the original CycleGAN dataset into training set A, training set B, test set A and test set B. On the basis of the original CycleGAN scheme, the use of image annotation information is added, and the dataset is divided into training set A, training set B, test set A, test set B, training set annotation information A, training set annotation information B, test set annotation information A and test set annotation information B.

[0072] In addition, the relevant annotation information of each image is also parsed and converted, for example: the bounding box annotation of the CADC dataset is a 3D bounding box, and the file format is.json. The bounding box annotation of the BDD100K dataset is a 2D bounding box, and the file format is.json. When parsing and converting, the bounding box annotation of the CADC dataset is converted into a 2D bounding box, and the annotation files of the CADC and BDD100K are converted from the.json format into the.xml format, and finally saved as a standardized XML format file. At this time, each XML file corresponds to the annotation information of an image, and the annotation information mainly includes the root element <annotation>, elements, <object>Elements. Among them, the root element <annotation>a root tag containing all annotation information of the whole image, an element describing detailed information of a single image, <object>The element describes each individual traffic actor object in the image, whose position information is saved in the form of coordinates in the corresponding <object>under the element <bbox>In the middle.

[0073] The pixels of the dataset images are uniformly adjusted to 720x1280, but CycleGAN defaults to input images of 256x256 pixels in three channels. Therefore, the images in the dataset are scaled and cropped to adapt to the image size of the network input. In addition, considering that high-resolution images require higher computing resources, the aspect ratio of the original image is maintained, and the image size is adjusted to 288x512 pixels.

[0074] The mask module creates a mask Mask according to the image annotation information of the input image Input image, marks the position of the road user through the bounding box, and applies the mask to the image to retain the area information within the bounding box. In one embodiment, the mask module specifically includes an annotation information acquisition unit, an initial mask setting unit, and a mask image setting unit, wherein:

[0075] The annotation information acquisition unit is used to extract and parse the annotation information corresponding to the original image, and judge whether the annotation information exists <object>If the element is present, then get the pixel coordinate values of the corners of the traffic participant bounding box, the origin (0, 0) of the coordinate system is usually the top-left corner of the image, and store them in the list boxes. The traffic participant bounding box is a rectangular frame that can just frame the traffic participant on the image, and is represented by four values xmin, xmax, ymin, ymax in the annotation information, and the value of the traffic participant bounding box is saved in the XML file:

[0076] boxes.append([xmin, xmax, ymin, ymax])

[0077] Wherein, xmin is the x-coordinate of the top-left corner of the traffic participant bounding box, xmax is the x-coordinate of the bottom-right corner of the traffic participant bounding box, ymin is the y-coordinate of the top-left corner of the traffic participant bounding box, and ymax is the y-coordinate of the bottom-right corner of the traffic participant bounding box.

[0078] Annotation information does not exist <object>If the element is 0, it indicates that there is no traffic participant in the original image, and the image annotation is directly generated as an empty list.

[0079] The initial mask setting unit is used to create a full zero matrix with the same size as the original image, which is used to represent the initial mask mask_0:

[0080] mask_0 = np.zeros((720, 1280), dtype=np.uint8)

[0081] Where (720, 1280) is the shape of the matrix, indicating that this matrix has 720 rows and 1280 columns. Specifically, it is used to represent a mask for an image with 720 pixels high and 1280 pixels wide. dtype=np.uint8 specifies that the data type of each element in the matrix is an unsigned 8-bit integer.

[0082] Traverse the information of the traffic participant bounding box in the list boxes, and set the pixel value in the area of the traffic participant bounding box to 255 to mark the traffic participant.

[0083] mask_0[ymin:ymax, xmin:xmax] = 255

[0084] mask_0[ymin:ymax,xmin:xmax] = 255 indicates that the value of a specific area in a two-dimensional array named mask_0 is being set to 255. [ymin:ymax,xmin:xmax]: This part is a slicing operation that selects a sub-region in mask_0. ymin and ymax define the starting and ending row indices of the selected region on the y-axis (vertical direction). xmin and xmax define the starting and ending column indices of the selected region on the x-axis (horizontal direction).

[0085] The mask image setting unit is used to copy the single-channel initial mask mask_0 to the red, green, and blue color channels of the RGB image, creating a three-channel mask image mask_rgb with shape (H, W, 3):

[0086] mask_rgb = np.stack((mask_0, mask_0, mask_0), axis=-1)

[0087] Where, the single-channel mask is usually a two-dimensional array, (mask_0, mask_0, mask_0): this is a tuple containing three mask_0 arrays. This means there are three identical mask arrays. The role of this code is to copy the single-channel mask mask_0 three times as the red, green and blue color channels of the RGB image, thereby creating a three-channel mask image with a shape of (H, W, 3). The axis parameter is used to specify the dimension on which to operate, such as stacking, slicing or reshaping, etc. axis = -1 means operating along the last dimension of the array, H is the height, and W is the width.

[0088] In one embodiment, the mask module specifically further comprises an image preprocessing unit:

[0089] The image preprocessing unit is used to synchronize the three-channel mask image mask_rgb with the original image for preprocessing operation: fixed aspect ratio reduction to 288x512 pixels, and then through the tensor2im function to convert the mask and the original image format from tensor to numpy (English full name for Numerical Python) array, and apply the three-channel mask image mask_rgb to the original image, through bitwise operation only to retain the information inside the traffic participant bounding box, that is, only to retain the information inside the bounding box:

[0090] img_bitwise = cv2.bitwise_and(img_numpy, mask_numpy)

[0091] The original image, the mask visualization image, and the original image and mask superimposed image are as shown in Figure 3 The first row is the original image, the second row is the mask image mask_rgb, and the third row is the superimposed image obtained by superimposing the original image and the mask.

[0092] In this embodiment, the network construction module also has an edge detector, which is jointly trained with the CycleGAN backbone network through the method of knowledge distillation to adapt to the edge detection task under snow conditions. In this embodiment, the basic structure of the CycleGAN backbone network is followed in the whole model construction, and the edge detector based on knowledge distillation is combined to realize high-quality migration of image style.

[0093] Taking the CycleGAN backbone network as the backbone network, selecting PiDiNet as the edge detector for local edge detection inside the bounding box, and jointly training the first generator Generator X2Y and Generator Y2X through the method of knowledge distillation. The edge detector has a pre-trained teacher network PiDiNet and a student network PiDiNet N , the output is all local edge detection image. The teacher network can guide the student network to perform high-precision edge detection under sunny and snowy conditions. Subsequently, the edge consistency loss is performed on the edge detection images of the input image, the generated image and the reconstructed image to prevent the loss of traffic participants such as cars and pedestrians during the style transfer process.

[0094] As shown in Figure 1 In order to facilitate understanding of the process of style transfer, the upper half and the lower half of the dashed line in the figure are symmetrical in structure. Taking the sunny domain to the snowy domain style transfer process as an example, the upper half Input image represents a sunny image, Translated image represents a generated snowy image, and Reconstruction image represents a reconstructed sunny image, realizing the function of transferring the image style from sunny to snowy. The lower half Input image represents a snowy image, Translated image represents a generated sunny image, and Reconstruction image represents a reconstructed snowy image, realizing the function of transferring the image style from snowy to sunny.

[0095] In the figure, the CycleGAN backbone network has a first generator Generator X2Y and a second generator Generator Y2X. Although two first generators Generator X2Y and two second generators Generator Y2X are shown in the figure, in fact, there is only one Generator X2Y, which is a generator for transferring images from domain X to domain Y, and in this embodiment, it can be understood as a generator for transferring images from sunny to snowy. There is also one Generator Y2X, which is a generator for transferring images from domain Y to domain X, and in this embodiment, it can be understood as a generator for transferring images from snowy to sunny.

[0096] In operation, the mask Mask is superimposed on the input image Input image and input to the teacher network PiDiNet and the student network PiDiNet N for edge detection. The input image Input image is subjected to style transfer by the first generator Generator X2Y to obtain a generated image Translated image, and the generated image Translated image is superimposed on the mask Mask and input to the student network PiDiNet N Edge detection is performed; a translated image is generated by a second generator Generator Y2X, a reconstruction image is generated, and the reconstruction image is superimposed with a mask Mask and then input to the student network PiDiNet again N Edge detection is performed.

[0097] If the style transfer from sunny to rainy weather is performed, only sufficient image data needs to be collected to construct a new data set, and then the present technical solution can be applied.

[0098] Through the present embodiment, the quality and consistency of the generated image are improved, the computing cost is reduced, the annotation information in the autonomous driving data set is fully utilized, and strong technical support is provided for the performance improvement of autonomous driving in adverse weather conditions.

[0099] PiDiNet is selected as the edge detector, and is jointly trained with the improved CycleGAN model through the method of knowledge distillation to optimize the edge detection effect of the snowy image. The architecture of PiDiNet includes an efficient backbone network and a task-specific edge structure. The backbone network uses a depth separable convolution structure and a shortcut connection to improve the running efficiency, while the edge structure is used to generate an edge map from each stage and provide deep supervision. In addition, PiDiNet uses a compact dilated convolution module CDCM and a compact spatial attention module CSAM to enrich multi-scale edge information and eliminate background noise. The final edge map is generated by fusing four single-channel feature maps, and then passing through a convolution layer and a sigmoid function.

[0100] The model training module is used to train the constructed sunny-snowy weather image data style transfer model, and uses the comprehensive loss function provided by formula (1) Combined with the adversarial loss Cycle consistency loss Local edge consistency loss And distillation loss To ensure that the generated snowy image is highly consistent in structure with the sunny weather image, and further optimize the quality of the generated image and the accuracy of the style transfer.

[0101]

[0102] wherein λ represents a weight parameter corresponding to the cycle consistency loss, the initial weight parameter is set according to the experience in the existing literature, and is continuously adjusted and optimized according to the specific task, data set, and model performance.

[0103] In one embodiment, the model training module learns and mimics the edge detection behavior of the teacher network PiDiNet through a knowledge distillation process to adapt to the edge detection task under snowy conditions. The model training module specifically includes an image edge detection unit and an edge mapping difference calculation unit, wherein:

[0104] The image edge detection unit is used to perform local edge detection on the input image (Inputimage) through the teacher network (PiDiNet) during the forward propagation stage, and obtain the first edge detection image (E(I)). teacher-Input ); via the student network PiDiNet N Local edge detection is performed on the input image, the generated image, and the reconstructed image to obtain the second edge detection image E((I)). student-Input )), the third edge detection image E((I student-Translated )), the fourth edge detection image E(I) student-Reconstruction The edge mapping difference calculation unit is used to calculate E(I). teacher-Input ) and E((I student-Translated The edge consistency loss between E(I) and E(I) teacher-Input ) and E(I student-Reconstruction The edge consistency loss between () is used to obtain the local edge consistency loss. to E(I teacher-Input ) and E((I student-Input The edge consistency loss between )) is used to obtain the distillation loss.

[0105] To ensure the student network maintains consistent edge structure when converting sunny images to snowy images, the model training process calculates the edge mapping difference between the original sunny image and the generated snowy image. This difference is measured using an L1 loss function and minimized during training, ensuring that the generated snowy image visually retains similar edge features to the original image. Furthermore, the student network's loss function includes not only edge detection loss but also knowledge distillation loss. The edge detection loss ensures the student network can effectively detect edges, while the knowledge distillation loss encourages the student network to mimic the teacher network's decisions and outputs. During training, the student network's parameters are updated by minimizing the total loss, which is a combination of the edge detection loss and the knowledge distillation loss.

[0106] In addition to the loss executed in the original CycleGAN, edge consistency loss and distillation loss are executed for the local edge detection results of the input image Input image, the generated image Translated image, and the reconstructed image Reconstruction image, mainly composed of three parts, respectively, the pixel-wise L1 loss of the local edge detection result of the input image by the teacher network PiDiNet and the student network PiDiNet N The pixel-wise L1 loss of the local edge detection result of the input image by the teacher network PiDiNet and the student network PiDiNet N The pixel-wise L1 loss of the local edge detection result of the input image by the teacher network PiDiNet and the student network PiDiNet N The pixel-wise L1 loss of the local edge detection result of the input image by the teacher network PiDiNet and the student network PiDiNet Figure 2 As shown in the following table, after the model training phase is completed, the Generator X2Y and the Generator Y2X can be used to perform style transfer on sunny and snowy weather image data.

[0107] In one embodiment, the adversarial loss The representation of the adversarial loss is as follows:

[0108]

[0109] wherein, represents the adversarial loss of the generator G that migrates the picture style from the domain X to the domain Y, represents the adversarial loss of the generator F that migrates the picture style from the domain Y to the domain X.

[0110] In one embodiment, and are represented by the following formula:

[0111]

[0112] wherein, represents the adversarial loss of the generator G that migrates the picture style from the domain X to the domain Y, which is composed of two parts: the first part represents the discriminator D that gives the expected value of the positive prediction that the real sample y comes from the domain Y Y The second part represents the discriminator D that gives the expected value of the negative prediction that the image G(x) converted by the generator G from the sample x in the domain X to the domain Y Y The second part The adversarial loss represents that the generator F migrates the picture style from the domain Y to the domain X, which is composed of two parts: the first part The discriminator D represents that the real sample x comes from the domain X X The second part gives the expected value of the positive prediction The discriminator D represents that the generator F converts the sample y in the domain Y to the image F(y) in the domain X X The second part gives the expected value of the negative prediction.

[0113] In one embodiment, the cycle consistency loss The representation of the cycle consistency loss is the existing content in the original CycleGAN model, and the improvement of the loss function in the present application is to increase the distillation loss and the edge consistency loss, which is represented by the following formula:

[0114]

[0115] Where F(G(x)) is the image generated by the generator G after migrating the original image x from the domain X to the domain Y, and then the migrated image is migrated back to the domain X by the generator F. G(F(y)) is the corresponding image generated by the generator F after migrating the image y from the domain Y to the domain X, and then the migrated image is migrated back to the domain Y by the generator G.

[0116] In one embodiment, the local edge consistency loss Distillation loss of the edge detection module The pixel-by-pixel L1 loss is obtained by equations (2) and (3), respectively;

[0117]

[0118] Where, is the edge consistency loss between E(I teacher-Input ) and E((I student-Translated )), is the edge consistency loss between E(I teacher-Input ) and E(I student-Reconstruction ), is the edge consistency loss between E(I teacher-Input ) and E((I student-Input )), and α1, α2, α3 are weight parameters of the edge consistency loss terms, respectively. The initial weight parameters are set according to the experience in the existing literature, and are continuously adjusted and optimized according to the specific task, data set and model performance.

[0119] Edge consistency loss L edge In addition to the pixel-by-pixel L1 loss, L2 loss, smooth L1 loss, logarithmic loss, etc. loss function can also be used, the form is still according to the formula shown in the paper, only the different calculation method is used to obtain each term.

[0120] The model construction and training process is as shown in Figure 4 First, the images under sunny and snowy conditions are screened and extracted from the CADC and BDD100K datasets, and the images with poor quality are removed through manual screening. Then, the images are uniformly resized and the annotation information is parsed to construct the mask module. In the model structure design, the generator G and F of CycleGAN are followed, which are responsible for the style transfer of sunny to snowy and snowy to sunny, respectively. The discriminators D X and D Y are used to distinguish the source domain and target domain images, the PiDiNet is selected as the edge detector, and the lightweight version of the PiDiNet N is trained through the knowledge distillation method and is trained jointly with the generator to adapt to the edge detection task under the snowy condition. In the loss function design, the adversarial loss, cycle consistency loss, local edge consistency loss and distillation loss are combined to optimize the quality of the generated images and the accuracy of the style transfer. In the model training stage, the input image is converted through the generator, and is evaluated through the discriminator and the edge detector. The total loss is calculated and the parameters are updated through back propagation. Finally, in the model evaluation stage, the quality of the generated image and the accuracy of the style transfer are evaluated to ensure the performance of the model in the sunny and snowy weather image data style transfer task, and the super parameter is adjusted according to the evaluation result to further improve the performance of the model.

[0121] The inference module is used to apply the trained sunny and snowy weather image data style transfer model to new sunny weather images, receive sunny weather images, and generate corresponding snowy images.

[0122] The application also provides a sunny and snowy weather image data style transfer method based on CycleGAN, which comprises:

[0123] Step 1, constructing a sunny and snowy weather image data style transfer model based on CycleGAN through a network construction module: the network construction module has a CycleGAN backbone network and an edge detector, the CycleGAN backbone network has a first generator Generator X2Y and a second generator Generator Y2X; the edge detector has a pre-trained teacher network PiDiNet and a student network PiDiNet N ; wherein the mask Mask is created according to the image annotation information of the input image Input image, and the mask Mask and the input image Input image are superimposed and input to the teacher network PiDiNet and the student network PiDiNet N Edge detection is performed; the input image Input image is input to the first generator Generator X2Y for style transfer to obtain a generated image Translated image, and the generated image Translated image and the mask Mask are superimposed and input to the student network PiDiNet N Edge detection is performed; the generated image Translated image is input to the second generator Generator Y2X to generate a reconstructed image Reconstruction image, and the reconstructed image Reconstruction image and the mask Mask are superimposed and input to the student network PiDiNet again N Edge detection is performed.

[0124] Step 2, training the constructed sunny-snowy weather image data style transfer model, using the adversarial loss described by formula (1) Cycle consistency loss Local edge consistency loss And distillation loss The comprehensive loss function is composed of Ensure that the generated snowy weather image is highly consistent in structure with the sunny weather image.

[0125] Step 3, applying the trained sunny-snowy weather image data style transfer model to a new sunny weather image, receiving the sunny weather image, and generating a corresponding snowy weather image.

[0126] In one embodiment, the method for creating a mask Mask according to the image annotation information of the input image Input image specifically includes:

[0127] Step 11, extracting and parsing the annotation information corresponding to the original image, and judging whether the annotation information exists <object>If yes, the pixel coordinate values of the corners of the traffic participant bounding box are obtained and stored in the list boxes; otherwise, an empty list is directly generated;

[0128] Step 12, a full zero matrix with the same size as the original image is created to represent the initial mask mask 0, and the information of the traffic participant bounding box in the list boxes is traversed to set the pixel value in the area of the traffic participant bounding box to 255 to mark the traffic participant

[0129] Step 13, the single-channel initial mask mask 0 is copied to the red, green and blue color channels of the RGB image to create a three-channel mask image mask_rgb with a shape of (H, W, 3):

[0130] mask_rgb = np.stack((mask_0, mask_0, mask_0), axis = -1)

[0131] Wherein, axis = -1 represents operation along the last dimension of the array, H is the height, and W is the width.

[0132] In an embodiment, the method for creating a mask Mask according to the image annotation information of the input image Input image specifically further includes:

[0133] Step 14, the three-channel mask image mask_rgb and the original image are preprocessed synchronously: fixed aspect ratio is reduced to 288x512 pixels, and then the mask and the format of the original image are converted from tensor to numpy array through the tensor2im function, and the three-channel mask image mask_rgb is applied to the original image, and only the internal information of the traffic participant bounding box is retained through bit operation.

[0134] In an embodiment, step 2 learns and imitates the edge detection behavior of the teacher network PiDiNet through the process of knowledge distillation to adapt to the edge detection task under snow conditions, and step 2 specifically includes:

[0135] Step 21, in the forward propagation stage, the input image Input image is subjected to local edge detection through the teacher network PiDiNet to obtain a first edge detection image E(I teacher-Input ); the student network PiDiNet N is subjected to local edge detection on the input image Input image, the generated image Translated image and the reconstructed image Reconstruction image to obtain a second edge detection image E((I student-Input ), a third edge detection image E((I student-Translated ) and E((I student-Reconstruction ).

[0136] Step 22, obtain local edge consistency loss by edge consistency loss between E(I teacher-Input ) and E((I student_Translated )), and edge consistency loss between E(I teacher-Input ) and E(I student-Reconstruction ). obtain distillation loss by edge consistency loss between E(I teacher-Input ) and E((I student-Input ).

[0137] In the process of technological development, there are often multiple possible solutions to address specific technical challenges. For the problem solved by the present application, although the local edge detection technology has been proven to be effective, it does not exclude other technical means that can also bring similar benefits. For example, attention mechanism can focus on the key parts of the image, which may further improve the accuracy of object recognition. However, whether these alternatives are truly feasible requires in-depth research, development and experimentation. When considering alternatives, the technical complexity, computational cost, implementation difficulty and final results must be weighed to determine whether they can achieve the same technical effect as the present application.

[0138] The prior art in processing images under harsh weather conditions, such as snow, the vehicle body may be covered with snow, resulting in increased difficulty in object recognition, and causing significant changes in color and texture, thereby interfering with the generator of the style transfer technology to correctly capture and transfer the style. In addition, existing style transfer technology mainly operates on the image itself, and does not make full use of the detailed annotation information already existing in the data set, hindering the generation of high-quality style transfer images.

[0139] Compared with the traditional CycleGAN model, the application performs local edge detection on the image annotation information in the autonomous driving dataset, focuses on the area within the bounding box, and thus maintains the consistency of the traffic participant object in the image. This method performs edge consistency loss on the real image, the generated image and the reconstructed image, effectively solves the problem of object loss, and improves the quality and consistency of the generated image. Compared with the second technology, the application avoids the high computational cost caused by global edge detection, and reduces the memory and computing requirements caused by edge detection. In addition, the application specially considers the use of annotation information in the autonomous driving dataset, and better integrates image style transfer and annotation information through local edge detection. The prior art only focuses on the image itself, and does not fully utilize the annotation information of the autonomous driving dataset itself. Therefore, the application provides a more efficient and accurate technical solution for the style transfer of the image in the autonomous driving dataset, while maintaining the advantages of the cycle consistency loss of the CycleGAN framework.

[0140] Finally, it should be pointed out that: the above embodiments are only used to illustrate the technical solutions of the application, and not to limit it. Those skilled in the art should understand that the technical solutions described in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; these modifications or replacements do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solutions of the embodiments of the application.< / object> < / object> < / object> < / bbox> < / object> < / object> < / annotation> < / object> < / annotation> < / object> < / object>

Claims

1. A style transfer system for clear / snowy weather image data based on CycleGAN, characterized in that, It includes a masking module, a network construction module, a model training module, and an inference module, among which: The network building blocks include a CycleGAN backbone network and an edge detector, used to construct a CycleGAN-based style transfer model for clear and snowy weather image data. The CycleGAN backbone network has a first generator (Generator X2Y) and a second generator (Generator Y2X). The edge detector has a pre-trained teacher network (PiDiNet) and a student network (PiDiNet). N The masking module creates a mask based on the image annotation information of the input image. This mask is then overlaid on the input image and input into both the teacher network (PiDiNet) and the student network (PiDiNet). N Edge detection is performed; the input image undergoes style transfer via the first generator (X2Y) to obtain the generated image (Translated image). The generated image (Translated image) is then overlaid with a mask and input into the student network PiDiNet. N Edge detection is performed; the generated image (Translated image) is processed by a second generator (Y2X) to generate a reconstructed image. This reconstructed image is then overlaid with a mask and input back into the student network PiDiNet. N Perform edge detection; The model training module is used to train the constructed style transfer model based on clear and snowy weather image data, using the adversarial loss described by equation (1). Cycle consistency loss Local edge consistency loss and distillation loss Composition of comprehensive loss function Ensure that the generated snowy weather images are structurally highly consistent with those of clear weather images; Where λ represents the weight parameter corresponding to the cycle consistency loss; The inference module is used to apply the trained style transfer model of clear and snowy weather image data to new clear weather images, receiving clear weather images and generating corresponding snowy weather images.

2. The CycleGAN-based style transfer system for clear and snowy weather image data as described in claim 1, characterized in that, The mask module specifically includes: The annotation information acquisition unit is used to extract and parse the annotation information corresponding to the original image and determine whether the annotation information exists. <object>If it is an element, then obtain the pixel coordinates of each corner point of the traffic participant's bounding box and store them in the list `boxes`; otherwise, generate an empty list directly. The initial mask setting unit is used to create an all-zero matrix with the same size as the original image to represent the initial mask mask_0. It iterates through the information of the traffic participant bounding boxes in the list boxes and sets the pixel values ​​within the bounding boxes of traffic participants to 255 to mark the traffic participants. The mask image setting unit is used to copy the initial single-channel mask mask_0 to the red, green, and blue color channels of the RGB image, creating a three-channel mask image mask_rgb with shape (H,W,3). mask_rgb=np.stack((mask_0, mask_0, mask_0), axis=-1) Here, axis = -1 indicates that the operation is performed along the last dimension of the array, H is the height, and W is the width.

3. The CycleGAN-based style transfer system for clear and snowy weather image data as described in claim 2, characterized in that, The mask module also includes: The image preprocessing unit is used to perform preprocessing operations on the three-channel mask image mask_rgb and the original image simultaneously: the aspect ratio is fixed and the image is reduced to 288×512 pixels, and then the format of the mask and the original image is converted from tensor to NumPy array through the tensor2im function. The three-channel mask image mask_rgb is applied to the original image, and only the internal information of the traffic participant bounding box is retained through bit operations.

4. The CycleGAN-based style transfer system for clear and snowy weather image data as described in any one of claims 1-3, characterized in that, The model training module learns and mimics the edge detection behavior of the teacher network PiDiNet through a knowledge distillation process to adapt to edge detection tasks under snowy conditions, specifically including: The image edge detection unit is used to perform local edge detection on the input image Inputimage through the teacher network PiDiNet during the forward propagation stage to obtain the first edge-detected image E(I). teacher-Input ); via the student network PiDiNet N Local edge detection is performed on the input image, the generated image, and the reconstructed image to obtain the second edge detection image E((I)). student-Input )), the third edge detection image E((I student-Translated )), the fourth edge detection image E(I) student-Reconstruction ); Edge mapping difference calculation unit, which is used to calculate E(I) teacher-Input ) and E((I student-Translated The edge consistency loss between E(I) and E(I) teacher-Input ) and E(I student-Reconstruction The edge consistency loss between () is used to obtain the local edge consistency loss. to E(I teacher-Input ) and E((I student-Input The edge consistency loss between )) is used to obtain the distillation loss.

5. The CycleGAN-based style transfer system for clear and snowy weather image data as described in claim 4, characterized in that, Local edge consistency loss Distillation loss of edge detection module The L1 loss is obtained by calculating it pixel-by-pixel using equations (2) and (3), respectively. in, for E(I teacher-Input ) and E((I student-Translated Edge consistency loss between )) for E(I teacher-Input ) and E(I student-Reconstruction Edge consistency loss between ) for E(I teacher-Input ) and E((I student-Input The edge consistency loss between )) is α1, α2, and α3, which are the weight parameters of the edge consistency loss term.

6. A style transfer method for clear / snowy weather image data based on CycleGAN, characterized in that, include: Step 1: Construct a CycleGAN-based style transfer model for clear and snowy weather image data using network building modules. These modules include a CycleGAN backbone network and an edge detector. The CycleGAN backbone network has a first generator (Generator X2Y) and a second generator (Generator Y2X). The edge detector has pre-trained teacher and student networks (PiDiNet). N Specifically, a mask is created based on the image annotation information of the input image. This mask is then overlaid on the input image and input into both the teacher network PiDiNet and the student network PiDiNet. N Edge detection is performed; the input image undergoes style transfer via the first generator (X2Y) to obtain the translated image. The translated image is then overlaid with a mask and input into the student network PiDiNet. N Edge detection is performed; the generated image (Translated image) is processed by a second generator (Y2X) to generate a reconstructed image. This reconstructed image is then overlaid with a mask and input back into the student network PiDiNet. N Perform edge detection; Step 2: Train the style transfer model for the constructed clear and snowy weather image data, using the adversarial loss described by equation (1). Cycle consistency loss Local edge consistency loss and distillation loss Composition of comprehensive loss function Ensure that the generated snowy weather images are structurally highly consistent with those of clear weather images; Where λ represents the weight parameter corresponding to the cycle consistency loss; Step 3: Apply the trained style transfer model for clear and snowy weather image data to new clear weather images, receive clear weather images, and generate corresponding snowy weather images.

7. The style transfer method for clear / snowy weather image data based on CycleGAN as described in claim 6, characterized in that, The methods for creating a mask based on the image annotation information of the input image specifically include: Step 11: Extract and parse the annotation information corresponding to the original image, and determine whether the annotation information exists. <object>If it is an element, then obtain the pixel coordinates of each corner point of the traffic participant's bounding box and store them in the list `boxes`; otherwise, generate an empty list directly. Step 12: Create a zero matrix with the same size as the original image to represent the initial mask mask_0. Iterate through the information of the traffic participant bounding boxes in the list boxes and set the pixel values ​​within the bounding boxes of traffic participants to 255 to mark the traffic participants. Step 13: Copy the initial single-channel mask mask_0 to the red, green, and blue color channels of the RGB image to create a three-channel mask image mask_rgb with shape (H,W,3). mask_rgb=np.stack((mask_0, mask_0, mask_0), axis=-1) Here, axis = -1 indicates that the operation is performed along the last dimension of the array, H is the height, and W is the width.

8. The style transfer method for clear / snowy weather image data based on CycleGAN as described in claim 7, characterized in that, The method for creating a mask based on the image annotation information of the input image also includes: Step 14: Perform preprocessing operations on the three-channel mask image mask_rgb and the original image simultaneously: reduce the aspect ratio to 288×512 pixels, then use the tensor2im function to convert the format of the mask and the original image from tensors to NumPy arrays, and apply the three-channel mask image mask_rgb to the original image, retaining only the internal information of the traffic participant bounding boxes through bit operations.

9. The style transfer method for clear / snowy weather image data based on CycleGAN as described in any one of claims 6-8, characterized in that, Step 2 involves learning and mimicking the edge detection behavior of the teacher network PiDiNet through a knowledge distillation process to adapt to edge detection tasks under snowy conditions. Specifically, this includes: Step 21: In the forward propagation phase, local edge detection is performed on the input image using the teacher network PiDiNet to obtain the first edge detection image E(I). teacher-Input ); via the student network PiDiNet N Local edge detection is performed on the input image, the generated image, and the reconstructed image to obtain the second edge detection image E((I)). student-Input )), the third edge detection image E((I student-Translated )), the fourth edge detection image E(I) student-Reconstruction ); Step 22, for E(I) teacher-Input ) and E((I student-Translated The edge consistency loss between E(I) and E(I) teacher-Input ) and E(I student-Reconstruction The edge consistency loss between () is used to obtain the local edge consistency loss. to E(I teacher-Input ) and E((I student-Input The edge consistency loss between )) is used to obtain the distillation loss.

10. The style transfer method for clear / snowy weather image data based on CycleGAN as described in claim 9, characterized in that, Local edge consistency loss Distillation loss of edge detection module The L1 loss is obtained by calculating it pixel-by-pixel using equations (2) and (3), respectively. in, for E(I teacher-Input ) and E((I student-Translated Edge consistency loss between )) for E(I teacher-Input ) and E(I student-Reconstruction Edge consistency loss between ) for E(I teacher-Input ) and E((I student-Input The edge consistency loss between )) is α1, α2, and α3, which are the weight parameters of the edge consistency loss term. < / object> < / object>

Images