A method for removing shadows from truck longitudinal beam assembly hole images

Through the shadow single-map guide network model, combining adversarial loss and cyclic consistency loss, the image shadow of truck longitudinal beam assembly holes is removed, which solves the problem of shadow impact in traditional technology and improves the accuracy and reliability of image processing.

CN118781029BInactive Publication Date: 2024-11-19CHANGCHUN UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411260724.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When traditional detection technology processes truck longitudinal beam images, especially images with uneven surfaces and complex lighting environments, shadows affect the performance of image processing algorithms. The existing shadow removal technology cannot adapt to complex industrial environments and artifacts often exist after processing. .

Method used

Using the shadow single-map boot network model, the shadow boot module and the single-map network module connected by multiplication are designed with adversarial loss, cyclic consistency loss and identity loss, a two-stage training strategy is designed to learn brightness characteristics and remove shadows, and improve image processing. accuracy and reliability.

Benefits of technology

It realizes efficient removal of image shadows of truck longitudinal beam assembly holes, improves image processing accuracy and reliability, retains key image details, prevents gradient vanishing problems in deep networks, and enhances the robustness and feature abstraction capabilities of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118781029B_ABST
    Figure CN118781029B_ABST
Patent Text Reader

Abstract

The invention discloses a method for removing shadows from an image of an assembly hole of a truck longitudinal beam, and relates to the technical field of image processing, including: obtaining an assembly hole shadow image and inputting it into a shadow single mapping guidance network model to obtain an assembly hole non-shadow image; outputting the assembly hole non-shadow image; wherein the training steps of the shadow single mapping guidance network model are: iterative training of a shadow guidance module under the constraint of a loss function; randomly initializing the module parameters of the trained shadow guidance module to the module parameters of a single mapping network module, and iterative training under the constraint of a loss function until a preset condition is reached, thereby obtaining a shadow single mapping guidance network model. The invention uses the brightness feature in the image shadow feature as a guidance condition, designs a single mapping network module, extracts and learns image features under the guidance of the shadow guidance module, realizes efficient assembly hole image shadow removal, and improves the accuracy and reliability of image processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and more particularly to a method for removing shadows from truck longitudinal beam assembly hole images. Background Art

[0002] As the truck industry develops, quality control of truck longitudinal beams is essential to ensure vehicle safety and durability. However, traditional inspection techniques have limitations in efficiency and accuracy, especially when processing longitudinal beam images with uneven surfaces and complex lighting conditions. Shadows in the image can significantly affect the performance of image processing algorithms.

[0003] Although existing image shadow removal technologies such as brightness adjustment and filtering are feasible to a certain extent, they cannot adapt to complex industrial environments, and the processed images often have artifacts. Therefore, how to accurately and reliably remove the shadow of the truck longitudinal beam assembly hole image is a problem that technicians in this field urgently need to solve. Summary of the invention

[0004] In view of this, the present invention provides a method for removing shadows from truck longitudinal beam assembly hole images, which overcomes the above-mentioned defects.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A method for removing shadows from a truck longitudinal beam assembly hole image, comprising:

[0007] Shadow removal method:

[0008] Get the assembly hole shadow image;

[0009] Inputting the assembly hole shadow image into a trained shadow single mapping guidance network model to obtain an assembly hole non-shadow image; the shadow single mapping guidance network model comprises a multiplication-connected shadow guidance module and a single mapping network module;

[0010] Outputting the non-shadow image of the assembly hole;

[0011] The training steps of the shadow single mapping guided network model are:

[0012] The shadow guidance module is iteratively trained under the constraint of the loss function; the module parameters of the trained shadow guidance module are randomly initialized as the module parameters of the single mapping network module, and iterative training is performed under the constraint of the loss function until the preset conditions are met to obtain the shadow single mapping guidance network model.

[0013] Optionally, before training the shadow single mapping guided network model, a training set needs to be constructed, and the steps for constructing the training set are:

[0014] Acquire assembly hole images of different specifications, and classify the assembly hole images according to whether they have shadows, to construct a shadow image data set and a non-shadow image data set;

[0015] The shadow image dataset and the non-shadow image dataset are expanded, and an unpaired assembly hole image dataset is constructed according to the expanded shadow image dataset and the non-shadow image dataset.

[0016] Optionally, the shadow guidance module and the single mapping network module have the same structure, both including two generation networks and two discrimination networks.

[0017] Optionally, the shadow guidance module is guided by brightness features and learns the mapping relationship between non-shadow features and shadow features through cyclic transformation of the two generative networks and supervision of the two discriminative networks.

[0018] Optionally, the cyclic conversion of the two generating networks comprises:

[0019] Forward process: converting the assembly hole shadow image into the assembly hole non-shadow image through a first generation network, and converting the assembly hole non-shadow image into the assembly hole shadow image through a second generation network and a shadow feature mask;

[0020] Reverse process: randomly extract the shadow feature mask, the second generation network converts the assembly hole non-shadow image into the assembly hole shadow image under the guidance of the shadow feature mask, and the assembly hole shadow image is mapped into the assembly hole non-shadow image through the first generation network.

[0021] Optionally, the two generating networks in the single mapping network module both take the assembly hole shadow image as input, one generating network generates a first assembly hole non-shadow image under the guidance of brightness features in the shadow guiding module, and the other generating network generates a second assembly hole non-shadow image in an encoding and decoding manner.

[0022] Optionally, during the training process of the shadow single mapping guided network model, the shadow guiding module is trained under the constraints of adversarial loss, cycle consistency loss and identity loss; the single mapping network module is trained as a whole using the adversarial loss, the identity loss and the target consistency loss under the guidance of the shadow guiding module.

[0023] Optionally, the loss function of the shadow single mapping guided network model is expressed as:

[0024] ;

[0025] In the formula, represents the loss function of the shadow single map guided network model; represents the generation network; represents the discriminant network; Respectively represent weights; Represents the generative adversarial loss of the generating network and the discriminative network; Represents the difference loss between the generated images of the two generation networks in the shadow guidance module and the original assembly hole shadow image; Represents the consistency loss between the non-shadow image of the assembly hole generated by the generation network in the single mapping network module and the original non-shadow image of the assembly hole.

[0026] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a method for removing shadows from truck longitudinal beam assembly hole images. The brightness features in the image shadow features are used as guiding conditions, and a single mapping network module is designed. Under the guidance of the shadow guidance module, image feature extraction and learning are performed to achieve efficient assembly hole image shadow removal, and improve image processing accuracy and reliability. Through a carefully designed two-stage training strategy, combined with adversarial loss, cycle consistency loss and identity loss, the generalization ability of the network is optimized and the robustness of the model is enhanced; and the single mapping and dual mapping network structures introduced in the present invention can effectively predict and process the complex relationship between shadows and non-shadows, retaining key image details. In addition, the fusion of multiplication connection and residual network further enhances feature abstraction and improves accuracy, while preventing the gradient vanishing problem in deep networks, ensuring the effect of deep learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0028] Figure 1 A schematic diagram of the overall method flow provided by the present invention;

[0029] Figure 2 Schematic diagram of the unpaired assembly hole image dataset S-NS provided by the present invention;

[0030] Figure 3 A schematic diagram of the structure of a shadow guiding module provided by the present invention;

[0031] Figure 4 A schematic diagram of the structure of a single-mapping network module provided by the present invention;

[0032] Figure 5 A schematic diagram of the structure of the generation network provided by the present invention;

[0033] Figure 6 A schematic diagram of the structure of the discrimination network structure and the SCM selection module provided by the present invention;

[0034] Figure 7 A schematic diagram of the shadow single mapping guided network training steps provided by the present invention;

[0035] Figure 8 A schematic diagram of the loss function of the network model provided by the present invention;

[0036] Fig. 9 A schematic diagram of a training loss curve provided by the present invention;

[0037] FIG10 (a) is a shadow image of the assembly holes input by each network model provided by the present invention; FIG10 (b) is an output image of CycleGAN; FIG10 (c) is an output image of TC-GAN; and FIG10 (d) is an output image of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] This embodiment discloses a method for removing shadows from a truck longitudinal beam assembly hole image. Figure 1 As shown, including:

[0040] Shadow removal method:

[0041] Step 1, obtaining the shadow image of the assembly hole;

[0042] Step 2: Input the shadow image of the assembly hole into the trained shadow single mapping guidance network model to obtain a non-shadow image of the assembly hole; the shadow single mapping guidance network model includes a multiplication-connected shadow guidance module and a single mapping network module;

[0043] Step 3, outputting the non-shadow image of the assembly hole;

[0044] Among them, the training steps of the shadow single mapping guided network model are:

[0045] The shadow guidance module is iteratively trained under the constraint of the loss function; the module parameters of the trained shadow guidance module are randomly initialized as the module parameters of the single mapping network module, and iterative training is performed under the constraint of the loss function until the preset conditions are met to obtain the shadow single mapping guidance network model.

[0046] In one embodiment, before training the shadow single mapping guided network model, a training set needs to be constructed. The steps for constructing the training set are:

[0047] Obtain assembly hole images of different specifications, and classify the assembly hole images according to whether they have shadows, and construct shadow image datasets and non-shadow image datasets;

[0048] The shadow image dataset and the non-shadow image dataset are expanded, and the unpaired assembly hole image dataset S-NS is constructed based on the expanded shadow image dataset and the non-shadow image dataset.

[0049] Furthermore, the assembly hole images of various plates with different sizes and thicknesses are collected through the assembly hole image acquisition system to construct an unpaired assembly hole image dataset S-NS for subsequent analysis and application, specifically:

[0050] Step 2-1-1: Collect assembly hole images based on the truck longitudinal beam assembly hole image acquisition system, and classify the assembly hole images into shadows and non-shadows;

[0051] Step 2-1-2: Rotate and mirror-flip the assembly hole image obtained in step 2-1-1 to obtain a new image, save the new image, classify it according to shadow and non-shadow, and create a new Figure 2 The image dataset S-NS shown has non-shadow images on the left and shadow images on the right.

[0052] In one embodiment, the shadow guidance module and the single mapping network module have the same structure, both including two generation networks and two discrimination networks.

[0053] Furthermore, the structure of the shadow guidance module is as follows Figure 3 As shown in the figure, it contains two generating networks and two discriminative networks. Each generating network consists of an encoder and a decoder. The encoder contains three convolutional layers and nine residual blocks. These three convolutional layers reduce the dimension of the original input image (from the unpaired assembly hole image dataset S-NS) by three quarters, and then use nine residual blocks to extract image features. The decoder consists of three deconvolution layers with a stride of two. Each layer in the encoder and decoder is followed by an instance normalization and ReLU activation function to enhance performance.

[0054] The discriminant network structure uses a Markov discriminator, and each convolutional layer is followed by a normalization function and a LeakyReLU activation function.

[0055] exist Figure 3 middle, is the first generated network, Generate a network for the second time; is the first discriminant network, is the second discriminant network; , , All are assembly hole shadow images; , , All are non-shadow images of assembly holes; is the shadow feature mask, is a shadow-free mask, is the shadow mask.

[0056] like Figure 4 As shown, the network structure of the generated network of the single mapping network module includes and Two subnetworks; for assembly hole shadow images , and They are composed of an encoder and a decoder, which are used to learn the mapping relationship between independent shadow and non-shadow images and obtain the corresponding and Residual image; where P(X) and P(Y) represent the data distribution in the shadow domain and the non-shadow domain, respectively; and All are real empty images; and are all shadow-free images; and Send them to the discriminant network respectively and , under the action of the selection module SCM, the shadow-free image with the highest confidence is selected Output.

[0057] exist and In the network, the encoder consists of three convolutional layers and nine residual blocks with a stride of two. The input is the assembly hole shadow image. After that, it is convolutionally downsampled. The decoder contains three deconvolution layers with a stride of two. There is an instance normalization function and a relu activation function after each layer of the encoder and decoder.

[0058] Generate a network structure such as Figure 5 As shown, represents color image input. This embodiment uses black and white images for network training. The third convolutional layer and deconvolutional layer were repeatedly run 3 times during training, and the convolution kernel and step size remained unchanged, so the receptive field size remained unchanged. At the same time, the training parameters were reduced and the judgment of the decision function was improved.

[0059] The discriminant network is a Markov discriminator, including and Two sub-networks, discriminant network and Corresponding to the generation network and , the discriminative network has 5 convolutional layers, each of which is followed by instance normalization and Leaky ReLU activation function.

[0060] In one embodiment, the shadow guidance module is guided by the brightness feature and learns the mapping relationship between the non-shadow features and the shadow features through the cyclic transformation of two generative networks and the supervision of two discriminative networks.

[0061] In one embodiment, the loop transformation of two generating networks includes:

[0062] Forward process: The assembly hole shadow image is converted into the assembly hole non-shadow image through the first generation network, and the assembly hole non-shadow image is converted into the assembly hole shadow image through the second generation network and the shadow feature mask;

[0063] Reverse process: randomly extract shadow feature masks, and the second generation network converts the assembly hole non-shadow image into the assembly hole shadow image under the guidance of the shadow feature mask, and the assembly hole shadow image is mapped into the assembly hole non-shadow image through the first generation network.

[0064] Furthermore, the shadow guidance module converts the input image from the image domain into a feature combination, using the brightness feature as the guidance strategy, and encodes the intrinsic mapping relationship between non-shadow features and shadow features. Import the hole shadow image Convert to non-shadow image of mounting hole , and then the second generated network In the shadow feature mask Under the guidance, it is then converted into a shadow image of the assembly hole , the second generation network Mounting hole shadow image Map to the same mounting hole shadow image , the first discriminant network Used to identify non-shadow images of assembly holes and the converted non-shadow image of the mounting hole .

[0065] The lower part is actually the reverse process of the upper part. First, the shadow feature mask Randomly draw a shadow mask from , input the assembly hole shadow image , the second generation network In the shadow mask Convert it to a mounting hole shadow image under guidance , and then through the first generation network Mapped as non-shaded image of mounting hole , finally, the first generated network In the shadowless mask (No shadow mask is a zero-valued mask), the hole non-shadow image is assembled Convert to the same non-shadow image of the mounting hole , the second discriminant network Used to determine the shadow image of the assembly hole and the converted mounting hole shadow image .

[0066] In one embodiment, the two generating networks in the single mapping network module both take the assembly hole shadow image as input, one generating network generates a first assembly hole non-shadow image under the guidance of the brightness feature in the shadow guiding module, and the other generating network generates a second assembly hole non-shadow image in an encoding and decoding manner.

[0067] Furthermore, for the input assembly hole shadow image , through two generative networks and To learn two independent mappings, the generative network Under the guidance of brightness features in the shadow guidance module, the first assembly hole non-shadow image is obtained. ; Generate network Without using the training parameters in the shadow guidance module, the second assembly hole non-shadow image is obtained by mapping from the shadow image domain to the non-shadow image domain by encoding and decoding. .

[0068] In order to keep the generated first assembly hole non-shadow image and the second assembly hole non-shadow image Feature consistency, using target consistency constraints to act on the generated network and the generative network At both ends, two generating networks are used to generate shadow images of the assembly holes. Different representations are generated to model the assumption that two different shadow difference images correspond to the same non-shadow image of the assembly hole, and the network is generated Prioritizes the target consistency constraint over the generated network .

[0069] Generative Network and It consists of two encoder-decoders with a dual network structure, called shadow residual generation network, which can generate shadow images from the assembly hole shadow image. Two different shadow residual images are generated in .

[0070] During network training, the generated network Under the guidance of the shadow image, the shadow image of the assembly hole is encoded The brightness feature relationship between the non-shadow and shadow residuals; Generate a network Learn to shade images of mounting holes Convert from the image domain to a feature combination. The decoder in the shadow residual generation network decodes the features from the feature domain into a shadow residual image; finally, the shadow image is assembled , non-shadow image of the first assembly hole and the second assembly hole non-shadow image Applying element-wise addition between them, we can recover two independent shadow-free images and Ideally, a shadow-free image and They are exactly the same images under the object consistency loss constraint.

[0071] Discriminant network and Similar to the structure of the generative network, in order to promote the adversarial training of the discriminative network, two real sample data are randomly selected from the non-shadow image dataset for the supervision of the real non-shadow image features. and Send them to the discriminant network respectively and , under the action of the selection module SCM, the shadow-free image with the highest confidence is selected The entire network module consists of two generating networks, two discriminant networks and a selection module SCM. The SCM selection module is located at the end of the entire network. It is an evaluation classifier and its structure is constructed by 4 convolutional layers. Figure 6 As shown, the discriminant network The pixels output in the output correspond to the receptive fields in the real assembly hole image, thereby ensuring the authenticity of the image details and making the shadow edges more obvious compared to the background. The discriminant network will generate the shadow-free image generated by the generating network It is superimposed with the real image according to the channel dimension, processed by 5 convolutional layers, and then sent to the SCM selection module; the sizes of the convolutional layers are 256×256, 128×128, 64×64, 32×32, and 32×32 from left to right; the SCM module (selection module) performs a confidence test on the generated image based on the similarity between the generated image and the real image.

[0072] In one embodiment, during the training process of the shadow single mapping guided network model, the shadow guidance module is trained under the constraints of adversarial loss, cycle consistency loss and identity loss; the single mapping network module is trained as a whole using adversarial loss, identity loss and target consistency loss under the guidance of the shadow guidance module.

[0073] Furthermore, the shadow single map guides the training steps of the network: Figure 7 As shown in the figure, the entire network training is divided into two stages. In the first stage, the shadow guidance module is trained under the constraints of adversarial loss, cycle consistency loss and identity loss; in the second stage, the single mapping network is trained as a whole under the guidance of the guidance module using adversarial loss, identity loss and target consistency loss. The training input of the second stage is the assembly hole shadow image. , in the figure, It is the shadow guidance module The residual block input maps the feature map; It is the shadow guidance module The weight layer of residual blocks; It is the shadow guidance module The weight layer of residual blocks; It is the first The residual block input maps the feature map; It is the first The weight layer of residual blocks; It is the first The weight layer of the residual block; relu is the activation function.

[0074] Among them, the identity loss represents the difference between the output images of the same module when the input is the same;

[0075] The object consistency loss characterizes the difference between the image generated by the module and the real image.

[0076] When the shadow guidance module is trained, the trained network parameters are frozen, and the single mapping network module uses some of the shadow guidance module parameters. The shadow guidance module is a dual mapping network structure. When it is trained, it is objectively considered that it has learned the brightness characteristics of the image, and then the shadow guidance module parameters are randomly initialized to the single mapping network parameters. The input of the single mapping network module is the mapping of the complete features of the image, that is, the data of each channel of the image. Then, under the guidance of the first stage training parameters, the shadow image features of the automobile longitudinal beam assembly hole are completely extracted and learned, and the image shadow removal is completed in the single mapping network.

[0077] The composition of the shadow single mapping guidance network: the shadow guidance module and the single mapping network module are connected by multiplication, and the multiplication connection is located in the residual network of the two modules. The brightness feature domain map of the shadow guidance module is multiplied pixel by pixel with the assembly hole image feature map in the single mapping network module, and the new domain map is sent to the weight layer. The reason for choosing to use the residual network is not only because the residual network can better fit the classification function to obtain higher classification accuracy, but also because the residual network can solve the problem of optimizing training when the number of layers of the network is deepened.

[0078] Among them, a residual layer can be expressed as:

[0079] ;

[0080] In the formula, the residual layer contains the direct mapping part and the residual part, is a direct mapping, is the residual part; It is the shadow guidance module The residual block input maps the feature map; is the residual layer; It is the shadow guidance module The residual block input maps the feature map; It is the shadow guidance module The weight layer of the residual block.

[0081] The entire multiplication connection calculation method can be expressed as:

[0082] ;

[0083] In the formula, It is the first The residual block input maps the feature map; is the residual layer; It is the first The residual block input maps the feature map, is a multiplication concatenation operation, It is the shadow guidance module The residual block inputs the mapping feature map, and after pixel-by-pixel multiplication, it is sent together to the single mapping network module residual blocks, It is the first The weight layer of the residual block is pushed back in sequence, and the pixel-by-pixel multiplication connection covers the entire residual network, and finally the entire feature extraction and learning of the image is completed under the guidance of the shadow brightness feature.

[0084] In one embodiment, the loss function of the shadow single mapping guided network model is expressed as:

[0085] ;

[0086] In the formula, represents the loss function of the shadow single map guided network model; represents the generation network; represents the discriminant network; Respectively represent weights; Represents the generative adversarial loss of the generating network and the discriminative network; Represents the difference loss between the generated images of the two generation networks in the shadow guidance module and the original assembly hole shadow image; Represents the consistency loss between the assembly hole non-shadow map generated by the generation network in the single mapping network module and the true assembly hole non-shadow map.

[0087] Furthermore, if Figure 8 As shown, the entire shadow guides the single mapping network loss function Generate adversarial loss based on the generation network and the discriminator network , two generation networks based on shadow-guided modules and The difference loss between the generated image and the original hole shadow image and the consistency loss between the non-shadow image of the assembly hole generated by the generative network in the single mapping network module and the real image The weighted sum of , the specific formula of the loss function used is as follows:

[0088] ;

[0089] In the formula, represents the loss function of the shadow single map guided network model; represents the generation network; represents the discriminant network; Respectively represent weights, and the entire network parameters are optimized through the minimax strategy; Represents the generative adversarial loss of the generating network and the discriminative network; Represents the difference loss between the generated images of the two generation networks in the shadow guidance module and the original assembly hole shadow image; Represents the consistency loss between the assembly hole non-shadow map generated by the generation network in the single mapping network module and the true assembly hole non-shadow map.

[0090] Among them, the generation adversarial loss based on the generation network and the discriminative network , in order to train each sub-network, the network adopts a general adversarial constraint to enforce the optimization of the adversarial loss of the generator and the discriminator. Specifically, let P(X) and P(Y) represent the data distribution of the shadow domain D(X) and the non-shadow domain D(Y), respectively, and the adversarial loss of each sub-network is It can be expressed as:

[0091] ;

[0092] In the formula, Represents the generative adversarial loss of the generating network and the discriminative network; represents the generation network; represents the discriminant network; ε represents the expectation operation; represents the j-th assembly hole non-shadow image, j∈1,2, indicating that two assembly hole non-shadow images are selected from the assembly hole non-shadow image dataset to guide the generation of assembly hole non-shadow images; is a sample randomly drawn from the shadow domain D(X), i.e., the input assembly hole shadow image.

[0093] Two generation networks based on shadow-guided modules and The difference loss between the generated image and the original assembly hole shadow image Specifically, samples are extracted from the P(X) data distribution sample extracted from the shadow domain D(X) , used to generate the network for shadow local feature extraction, so that the output images of the two generated sub-networks are consistent. The specific formula of the loss function used is as follows:

[0094] ;

[0095] In the formula, represents the generating network; ε represents the expectation operation.

[0096] Consistency loss between the non-shadowed assembly hole map generated by the network from the single-map network module and the true non-shadowed assembly hole map Specifically, samples are drawn from the P(Y) data distribution sample extracted from the non-shadow domain D(Y) , add it to the loss judgment to make the generated assembly hole non-shadow image similar to the real assembly hole non-shadow image. The specific formula of the loss function used is as follows:

[0097] .

[0098] In the formula, represents the generating network; ε represents the expectation operation.

[0099] During the training process, Pytorch was used to build the model, and the Adam optimizer with a learning rate of 0.0002 was used to optimize the entire network parameters. The prepared automobile longitudinal beam assembly hole dataset was input, the number of training epochs was set to 200, and the training step batch_size was set to 8. All images were rescaled to 256×256 as input, and linear decay was used to gradually reduce the learning rate to zero. Finally, the training was carried out on a Windows 10 system computer, and the training time was about 36 hours. The graphics card was NVIDIA RTX3090 GPU. Fig. 9 As shown in the figure, the loss of the shadow-guided single-map generative adversarial network (SGSM-GAN) has been relatively stable from the 100th to the 120th iteration, and the network has reached the best state, proving that the shadow removal algorithm for the automobile longitudinal beam assembly hole image has achieved the optimal shadow removal effect.

[0100] In the unpaired assembly hole image dataset S-NS, assembly hole images that have not been trained by the network are selected and used as test sample inputs. CycleGAN, TC-GAN and the method SGSM-GAN of this embodiment are used to process the assembly hole shadow images. Some experimental results are shown in Figures 10(a) to 10(d):

[0101] Figures 10 (a) to 10 (d) show the comparison of shadow removal results of the network model of the present invention, CycleGAN and TC-GAN on the automobile longitudinal beam assembly hole dataset. Figure 10 (a) represents the assembly hole shadow image input by each network model, and Figures 10 (b), 10 (c) and 10 (d) respectively represent the partial output results of the three network models of CycleGAN, TC-GAN and the network of this embodiment. It can be observed that the shadow removal effect of the network model of this embodiment is quite impressive, and there is almost no residue in the shadow area of ​​the assembly hole. The CycleGAN network model does learn some features of the shadow-free image, but it is still unable to distinguish the edge area of ​​the assembly hole, resulting in defects and the inability to complete the shadow removal task. The texture features of the TC-GAN network model appear clearer and can identify the real assembly hole edge, but there are still shadow residues in the darkest area of ​​the shadow area, proving that the TC-GAN network has not perfectly learned the brightness features of the non-shadow image of the assembly hole.

[0102] In summary, the present invention constructs a data set by collecting two sets of images, and uses a truck longitudinal beam assembly hole image acquisition system to collect assembly hole images. Subsequently, a designed generative adversarial network model is used for training, in particular, a simple and effective shadow single mapping guided network. The network can successfully constrain the mapping relationship between shadow images and shadow-free images in an unsupervised situation, pay more attention to the extraction of local features, and thus improve the effect of shadow removal.

[0103] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0104] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for removing shadows from truck longitudinal beam assembly hole images, characterized in that: include: Shadow removal method: Get the assembly hole shadow image; Inputting the assembly hole shadow image into a trained shadow single mapping guidance network model to obtain an assembly hole non-shadow image; the shadow single mapping guidance network model comprises a multiplication-connected shadow guidance module and a single mapping network module; Outputting the non-shadow image of the assembly hole; The shadow guidance module and the single mapping network module have the same structure, both including two generating networks and two discriminative networks; The shadow guidance module is guided by the brightness feature and learns the mapping relationship between the non-shadow feature and the shadow feature through the cyclic conversion of the two generating networks and the supervision of the two discriminative networks; The loop transformation of the two generating networks includes: Forward process: converting the assembly hole shadow image into the assembly hole non-shadow image through a first generation network, and converting the assembly hole non-shadow image into the assembly hole shadow image through a second generation network and a shadow feature mask; Reverse process: randomly extracting the shadow feature mask, converting the assembly hole non-shadow image into the assembly hole shadow image under the guidance of the shadow feature mask by the second generation network, and mapping the assembly hole shadow image into the assembly hole non-shadow image through the first generation network; The two generating networks in the single mapping network module both take the assembly hole shadow image as input, one generating network generates a first assembly hole non-shadow image under the guidance of brightness features in the shadow guiding module, and the other generating network generates a second assembly hole non-shadow image in an encoding and decoding manner; The training steps of the shadow single mapping guided network model are: The shadow guidance module is iteratively trained under the constraint of the loss function; the module parameters of the trained shadow guidance module are randomly initialized as the module parameters of the single mapping network module, and iterative training is performed under the constraint of the loss function until a preset condition is met to obtain the shadow single mapping guidance network model; Among them, during the training process of the shadow single mapping guided network model, the shadow guidance module is trained under the constraints of adversarial loss, cycle consistency loss and identity loss; the single mapping network module is trained as a whole using the adversarial loss, the identity loss and the target consistency loss under the guidance of the shadow guidance module.

2. The method for removing shadows from truck longitudinal beam assembly hole images according to claim 1, characterized in that: Before the shadow single mapping guide network model is trained, a training set needs to be constructed. The steps for constructing the training set are as follows: Acquire assembly hole images of different specifications, and classify the assembly hole images according to whether they have shadows, to construct a shadow image data set and a non-shadow image data set; The shadow image dataset and the non-shadow image dataset are expanded, and an unpaired assembly hole image dataset is constructed according to the expanded shadow image dataset and the non-shadow image dataset.

3. The method for removing shadows from truck longitudinal beam assembly hole images according to claim 1, characterized in that: The expression of the loss function of the shadow single mapping guided network model is: L SGGAN (G i ,D i )=λ1·(Loss GAN (G i ,D i ))+λ2·(Loss SG (G i ))+λ3·Loss idenity (G i )); Where, L SGGAN represents the loss function of the shadow single map guided network model; G i represents the generating network; D i represents the discriminant network; λ1, λ2, λ3 represent weights respectively; Loss GAN Represents the generation adversarial loss of the generating network and the discriminating network; Loss SG Represents the difference loss between the generated images of the two generation networks in the shadow guidance module and the original assembly hole shadow image; Loss identity Represents the consistency loss between the non-shadow image of the assembly hole generated by the generation network in the single mapping network module and the original non-shadow image of the assembly hole.