A reflection interference removing method and system based on reflection region detection and edge attention

Abstract

The present application belongs to the technical field of computer, and particularly relates to a reflection interference removing method and system based on reflection region detection and edge attention, which comprises the following steps: adopting the idea of a generative adversarial network on a main body, and taking a mixed image containing reflection interference after pretreatment as the input of the model; inputting the input mixed image containing reflection interference into a reflection region detection network module, and passing through a reflection feature enhancement network module to obtain image multi-scale Gaussian-Laplacian features and finally participate in the synthesis of image gradient features of strengthened reflection; taking the obtained image gradient features as the input of the entire generative network, extracting features through a shallow feature extraction network, transmitting the extracted features to a discriminative network in a skip connection mode, discriminating the quality of the generated image after reflection interference removal while obtaining the output image after reflection interference removal, guiding the generation of the next round generator until the output quality meets the requirements. The present application can remove the influence of reflection interference on images in a complex light external scene, improve the quality of the license images collected by the system, and make the images meet the image quality requirements of other downstream tasks.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of computer technology, and particularly relates to a method and system for de-reflection interference based on reflection region detection and edge attention. Background Technology

[0002] In practical certificate image acquisition systems, reflectors and glass lenses are introduced to ensure image quality. However, the introduction of reflectors and glass lenses inevitably introduces reflection interference into the acquired images. Reflection interference refers to the phenomenon where external light reflects off the objective glass and reflectors, causing glare, highlights, and artifacts in the captured certificate images, hindering image recognition. The main reasons for reflection interference in the system's acquired images are twofold: Firstly, the multispectral light, especially visible light, of the certificate image acquisition equipment has low penetrability. When passing through the thick objective glass, the light refracts on the glass surface, creating blurry ghosting. Secondly, in strong ambient light, external light reflects off the objective glass and is imaged on the reflector, forming a mirrored reflection on the objective glass surface. Although the system's imaging effect can be intelligently adjusted by changing the camera's exposure time to control ambient light levels, adjusting the exposure time cannot suppress reflection interference when the ambient light is too strong, and it may even reduce the overall image quality of the system.

[0003] In engineering practice, removing reflection interference from ID card images using non-deep learning methods is difficult and ineffective. Firstly, during the operation of the ID card image acquisition system, only a single image is acquired for each spectrum. Single-image dereflection algorithms typically require additional user-defined prior conditions to assist in the dereflection interference task, which is time-consuming and labor-intensive. Furthermore, these prior conditions may fail in different types of reflection scenarios. Secondly, due to the uncertainty of ambient lighting and external background, the scale differences of reflective areas in the single ID card image acquired by the system are large and difficult to predict. The reflection intensity varies in some areas, the edges of reflective areas are blurred, and the location of reflective areas is difficult. Thirdly, when the external ambient light is too strong, artifacts and other image interference that severely affect ID card recognition will appear in the captured ID card image, suppressing the expression of image edge features and making image recovery difficult.

[0004] Based on the above analysis, the problems and shortcomings of the existing technology are as follows: the existing methods are difficult to avoid the reflection problem in image imaging, and it is known that the existing traditional image processing methods using non-deep learning have weak ability to remove reflection interference, have great limitations, poor effect, and great difficulty. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method and system for de-reflection interference based on reflection region detection and edge attention.

[0006] This invention is implemented as follows: a method for de-reflection interference based on reflection region detection and edge attention, comprising the following steps:

[0007] S1: The main body adopts the idea of ​​generative adversarial network, and the mixed image containing reflection interference is preprocessed and used as the input of the model;

[0008] S2: The mixed image containing reflection interference input in S1 is input into the reflection region detection network module, and then passes through the reflection feature enhancement network module to obtain the multi-scale Gaussian-Laplacian features of the image and finally participate in the synthesis of image gradient features to enhance reflection.

[0009] S3: The image gradient features obtained in S2 are used as the input of the entire generator network, and the shallow feature extraction network extracts features.

[0010] S4: The features extracted in S3 are transmitted to the discriminator network via skip connections. While obtaining the output image after de-reflection interference, the quality of the generated de-reflection interference image is judged to guide the generation of the next generator until the output quality meets the requirements.

[0011] Furthermore, S1 specifically includes:

[0012] The task of removing image reflection interference usually considers an image containing reflection interference as a mixture of transmitted and reflected images. When using traditional methods to calculate the mixing ratio of the two through some image features or to separate them using a mixing matrix, there are significant limitations in processing image reflection interference removal for ID cards. Specifically, it cannot handle images with uneven reflection intensity and has poor generalization ability. Therefore, a deep learning-based image reflection interference removal algorithm is adopted to achieve this task.

[0013] Since generative adversarial networks can effectively enhance image details during the generation stage, and can also guide the image generation effect by adjusting the attention distribution, improvements are made based on generative adversarial networks.

[0014] The mixed images containing reflection interference acquired by the certificate collection hardware system will undergo image resizing and instance normalization to ensure that the images input to the network are of consistent size and do not interfere with each other. Instance normalization refers to the process of calculating the mean and standard deviation for each input feature map in a single channel, and then using the mean and standard deviation of each channel to normalize the elements within that single channel. Instance normalization has significant advantages in image style transfer. For a feature map x with a batch size of N and length, width, and number of channels h, w, and c, respectively, its specific mathematical definition is as follows:

[0015]

[0016] Where x hw Let represent a single-channel image with a scale of h×w, where μ(x) and σ(x) are the mean and variance, respectively, γ and β are variables learned during training, and ε is a small constant.

[0017] Furthermore, S2 specifically includes:

[0018] Based on the characteristic that the gradient of the reflected image is sparser than that of the transmitted image in the mixed image, a reflection region detection network was designed. The mixed image is processed by the multi-scale reflection region detection network to obtain the multi-scale Gaussian-Laplacian features of the image. Then, the reflection features are further refined by the reflection feature enhancement network that integrates channel attention, and the reflection region probability map and the reflection image prediction map are output. In this way, the next stage of image generation is guided by suppressing the reflection-dominant region and enhancing the transmission-dominant region.

[0019] Based on the phenomenon that the edge features of projected images are usually more pronounced than those of reflected images, a reflection feature enhancement network based on edge attention was designed to strengthen the edges of the projected image. The reflection feature enhancement module has two inputs: one is the multi-scale Gaussian-Laplacian features of the original image obtained from the reflection region detection module. out Secondly, the image completely passes through the output I of the reflection area detection module. RD I out Feature enhancement is achieved through four cascaded squeeze-excitation residual modules, followed by I... RD Weighted product is used to obtain image gradient features that enhance reflections;

[0020] Removing reflection interference can be seen as the task of separating the reflected image and the transmitted image in a mixed image. In traditional interference removal algorithms that use image transformation, the selected prior conditions are highly dependent on the scene. In a specific scene, the prior conditions may fail, thus directly affecting the reflection removal effect.

[0021] To ensure the acquisition of multi-scale image gradient features, a multi-scale feature extraction module is used in the network structure. The input image is scaled by multi-scale global pooling, and the image size is scaled to 1 / 2, 1 / 4, 1 / 8 and 1 / 16 of the original scale, respectively. The original scale image and the multi-scale image obtained by pooling are then processed by Gaussian-Laplacian convolution to obtain multi-scale image gradient features.

[0022] Laplacian convolution is a common edge detection method that obtains edge features of an image by performing second-order differentiation. However, it is sensitive to image noise and discrete points. To overcome this problem, Gaussian smoothing is usually performed before Laplacian convolution. The calculation method of its convolution kernel is as follows:

[0023]

[0024] Where σ 2 σ is the standard deviation, and its selection is positively correlated with the image smoothing effect. It can be adjusted based on the network performance. 2 Fine-tuning of values;

[0025] The obtained multi-scale Gaussian-Laplacian feature maps are upsampled to their original scale using bilinear interpolation and then stitched together; To represent the image at the corresponding scale after the downsampling operation, use This represents the upsampled image at the corresponding scale; the output image at this time is:

[0026]

[0027] Three squeeze-excited residual modules are used to enhance the output image I. out The multi-scale Gaussian-Laplacian features are described. The squeeze-excitation residual module models the interdependencies between image channels through two steps: squeezing and excitation. Squeezing is achieved by performing global average pooling on the feature map to obtain the global compressed feature vector of the current feature map. For the input image X∈H×W×C, the squeezing step can be written as the following formula:

[0028]

[0029] Z c This refers to the output related to the c-th channel, with input x. c The activation comes from the convolutional layer; the activation is achieved by obtaining the weights of each channel in the feature map through two fully connected layers, activating it using the sigmoid function, and then using the weighted feature map as the input to the next layer of the network. The activation process can be written as the following formula:

[0030]

[0031] T1 and T2 are two linear layers, and the feature intensity of each channel is adaptively adjusted through the network's global loss function.

[0032] After obtaining the enhanced image features, the PReLU function is used to activate the image features. The formula for the PReLU function can be written as:

[0033]

[0034] Where y i It is the input of the i-th channel, a i Control the slope of the negative half-axis, with an initial value of a. i =0.2, and its value can be adaptively updated through backpropagation during subsequent training. The process of feature enhancement and activation is recorded as f. LOG Finally, the final reflection region detection probability map I is obtained by using a 3×3 convolution kernel and Sigmoid activation normalization. RD I RD It can be represented as:

[0035] I RD =Sigmoid(Conv(f LOG (I out )))

[0036] In this way, the mixed image can obtain preliminary gradient features of the reflection image through the reflection region detection module, providing basic clues for predicting the reflection image. To better predict the reflection image, a reflection feature enhancement module is used to further refine the gradient features of the reflection image. The reflection feature enhancement module requires two inputs: one is the I obtained from the original image through the multi-scale Gaussian-Laplacian feature extraction module. out Secondly, the final output I of the reflection area detection module. RD Feature enhancement is achieved through four cascaded squeeze-excitation residual modules, and then combined with I RD Weighted products are used to further enhance the gradient features of the reflection image;

[0037] To enhance the network's edge representation capabilities, an edge attention module is introduced. Traditional edge detection schemes typically extract only the binary contours of the image. However, in de-reflection interference tasks, obtaining binary edge contours is ineffective for subsequent transmission image reconstruction because even in areas of strong reflection, indistinguishable hard boundaries may exist, interfering with edge detection of the actual transmission image. Therefore, the mean absolute color difference (MAD) method is chosen to obtain image edge representation. The color difference between a pixel and its surrounding pixels is calculated in each of the RGB channels of the input image, and a weighted average is taken. The image edge is estimated based on the MAD, calculated using the following formula:

[0038]

[0039] Where (x,y) are the coordinates of the corresponding pixel in the image, and c refers to the RGB color space channel of the image. The edge map of the mixed image obtained in this way has a more obvious suppression effect on the hard boundaries of the reflective region than the binarized edge map obtained by converting the grayscale image, because the color gamut of the hard boundaries of the reflective region is usually relatively simple and has a small impact on the average absolute color difference. The obtained edge map is introduced into the network as an additional channel input to add edge attention to the network.

[0040] Furthermore, S3 specifically includes:

[0041] The generative network consists of convolutional layers, ReLU activation layers, instance normalization layers, and a coordinated attention layer. The generative network extracts features from the input image. To enhance feature extraction capabilities, a coordinated attention module is introduced. This module performs pooling operations along both row and column dimensions, aggregating vertical and horizontal features into two separate feature maps. These feature maps are then encoded into two attention maps, each capturing long-range dependencies in either row or column space. This attention mechanism efficiently integrates global image features, acquiring not only cross-channel features but also coordinate-sensitive information, aiding in the model's identification and localization of features of interest. In the feature extraction stage of the mixed image, coordinate localization of the reflection region is crucial for removing reflection interference. Channel attention addresses feature acquisition, while spatial attention addresses feature localization. Coordinated attention provides both spatial and channel attention to the network with only a slight increase in computational complexity, enabling the generative network to efficiently acquire transmission image features.

[0042] After obtaining the enhanced transmission image features separated by S2, these features are used as input to a generative adversarial network (GAN) for subsequent feature reconstruction and output. A GAN is a generative model that combines the characteristics of both generative and discriminative models. This network consists of a generator network and a discriminator network. The generator is responsible for generating new data samples based on the feature distribution of the sample data, while the discriminator provides feedback to further optimize the generator. Through this interaction between the two neural networks, the network learns and ultimately guides the data samples generated by the generator to be as close as possible to the real data samples. This network can be used in unsupervised or semi-supervised environments, greatly reducing the requirements for sample data.

[0043] The basic process of a Generative Adversarial Network (GAN) is as follows: the generator network G learns from the random input z of the model, and outputs generated samples G(z). These generated samples and real samples G(x) are then input into a discriminator. The discriminator labels the real samples as 1 and the generated samples as 0. The goal is to train a discriminator D that maximizes the accuracy of sample classification. Based on this, the generator G is optimized by minimizing the adversarial loss log(1-D(G(z))), thereby reducing the accuracy of the discriminator's classification. The two network's evaluation functions are completely opposite, and a zero-sum game is played by minimizing and maximizing the evaluation function V(G,D), as shown in the formula:

[0044]

[0045] Where x represents the real sample data, p data (x) represents the true sample data distribution, z represents the random input data, and p z (z) represents the prior distribution of the random input data; when p z =p data At that time, a zero-sum game has an optimal solution.

[0046] Furthermore, S4 specifically includes:

[0047] The main function of the discriminative network is to expand the feature vectors of the acquired feature maps, ultimately obtaining a generated image with the same scale as the input image. In addition, skip connections are introduced between the generator and discriminative networks, which can directly transfer image data from the generator network to the discriminative network, reducing the loss of image detail information. Furthermore, a feature transformer consisting of 6 cascaded residual modules is added between the generator and discriminative networks. Its main function is to convert the image features extracted by the discriminative network into corresponding feature maps, and it can effectively alleviate the gradient vanishing problem during training.

[0048] The decoder consists of deconvolutional layers, ReLU activation layers, normalization layers, Tanh activation layers, etc. The main function of the decoder is to expand the feature vectors obtained from the feature transformer and skip connections to obtain a generated image with the same scale as the input image. At the same time, it participates in the discrimination process of the entire generative adversarial network and improves the precision and generation quality of the entire network through the adversarial learning process.

[0049] Another objective of this invention is to provide a reflection interference removal system based on reflection region detection and edge attention, which implements the aforementioned reflection interference removal method. This system includes an image preprocessing module, a reflection region detection network module, a reflection feature enhancement network module, a generative adversarial network module (including a generative network module), and a discriminant network module.

[0050] The image preprocessing module is used to normalize the size of images after they are input into the network.

[0051] The reflection region detection network module, connected to the image preprocessing module, is used to obtain multi-scale Gaussian-Laplacian features of the image in the reflection region detection network.

[0052] The reflection feature enhancement network module, connected to the reflection region detection network module, is used to further refine the reflection features through the reflection feature enhancement network that incorporates channel attention, and outputs the probability map of the reflection region and the prediction map of the reflection image to the generative adversarial network.

[0053] The generative adversarial network module includes a generative network module and a discriminative network module, which guide image generation by suppressing the reflection-dominant region and enhancing the transmission-dominant region through the joint action of the generative and discriminative networks.

[0054] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the aforementioned de-reflection interference method based on reflection region detection and edge attention.

[0055] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the aforementioned de-reflection interference method based on reflection region detection and edge attention.

[0056] Another objective of this invention is to provide a certificate image data processing terminal, which is used to implement the certificate image acquisition system based on the reflection region detection and edge attention de-reflection interference method.

[0057] Another objective of this invention is to provide a cross-platform computer application that can be deployed on Windows, Linux, and most domestic operating systems, and is used to perform the steps of the aforementioned reflection region detection and edge attention-based de-reflection interference method in conjunction with specific hardware.

[0058] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0059] First, (1) the present invention provides a method for removing reflection interference based on reflection region detection and edge attention, which can be used for image reflection interference removal in certificate acquisition systems under complex lighting scenes. Based on the gradient difference between the reflection-dominant region and the transmission-dominant region in a mixed image containing reflection interference, reflection region detection is introduced into the image generation network to adjust the network's attention to specific regions; and because of problems such as blurred text and unclear details caused by indistinct edge features in mixed images, an edge attention module is introduced into the network to retain as much image detail edge information as possible.

[0060] Compared to similar algorithms, the algorithm in this chapter is superior in terms of model size, running speed, and technical indicators.

[0061] (2) The present invention proposes a de-reflection interference method based on reflection region detection and edge attention, which can provide a strong theoretical basis and technical support for downstream work. For a long time, many currently used certificate acquisition systems have only completed basic image acquisition tasks. If further detailed analysis of the images acquired by the certificate acquisition equipment is desired, it is necessary to use a solution or equipment other than the certificate acquisition system. There are two main reasons for this phenomenon. The first reason is that this type of certificate acquisition system does not have the ability to adaptively resist reflection interference, and is also more or less affected by external light sources, resulting in reflection interference. This makes it difficult or even impossible to perform a series of downstream tasks such as machine-readable code recognition, full-frame recognition, and chip reading on the acquired certificate images, such as those under visible light. The second reason is that the currently feasible de-reflection interference algorithms based on traditional image processing methods are highly dependent on the usage scenario. If they are not suitable for the environment of the certificate acquisition system, the processed image will be of poor quality or the processing time will be too long, failing to meet the system parameter requirements.

[0062] The proposed de-reflection interference method based on reflection region detection and edge attention adopts an end-to-end network structure based on generative adversarial networks. This makes it relatively easy to embed as a module into the system, minimizing its impact on the overall document collection system structure and facilitating deployment. Furthermore, the use of a deep learning framework for de-reflection interference allows for module parameter training before system deployment. This results in lower time consumption compared to traditional image processing methods during actual production deployment, saving time for downstream tasks and enabling the document collection system to complete more personalized and modular collection and recognition of documents within a limited timeframe.

[0063] Secondly, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0064] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0065] The expected benefits and commercial value of this technical solution are considerable. Firstly, it solves the problem of glare and bright spots in the images captured by document acquisition equipment under complex lighting conditions. This significantly improves downstream tasks such as full-frame document reading and authentication, where reflection interference in the captured images leads to decreased reading and authentication success rates. This allows the document acquisition equipment to integrate image acquisition, reading, and authentication, achieving multi-functionality and reusability. Its quality and advantages will attract airports, border control, customs, and various enterprises and institutions with relevant needs to choose the integrated equipment resulting from this technical solution, eliminating the need to purchase multiple devices with different functions to complete corresponding tasks. This truly realizes the integration and intelligence of document acquisition equipment, making its functions seamless and streamlined. Secondly, this technical solution provides a new approach to solving this type of reflection interference problem. It can be applied to different scenarios, not just document acquisition systems, showing great application potential and the potential for adoption by other industries, thus generating continuous benefits. Furthermore, patent protection for the technical solution will further ensure competitive advantage and market share, bringing long-term commercial value and competitive advantage to enterprises. In summary, this technical solution has high expected returns and commercial value, and is expected to become an important technical and commercial standard for document collection systems at airports and border inspection ports.

[0066] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0067] For a certificate acquisition system, the phenomenon of reflection interference in captured images under certain lighting conditions is a common problem. Currently, most systems rely on traditional image processing methods, and some even lack adaptive reflection interference removal capabilities. Furthermore, traditional image processing methods may only be effective against one type of reflection interference under complex external lighting conditions, lacking generalization ability. The method proposed in this invention, based on reflection region detection and reflection edge attention, can adaptively handle various reflection phenomena under different light source conditions in practical applications, demonstrating good generalization. Prior to this, no deep learning-based methods or solutions had emerged that could robustly address reflection interference in certificate acquisition systems. Moreover, this invention achieves significant breakthroughs in key technologies such as feature extraction and feature fusion, laying a solid foundation for further research on reflection interference removal in certificate acquisition systems and for more downstream tasks and applications relying on the improved image quality and integrity after reflection interference removal. Therefore, this invention fills a technological gap in feature extraction and fusion, as well as reflection interference removal, both domestically and internationally, providing a new technical solution and development direction for certificate acquisition systems.

[0068] Fourth, the technological advancements in reflection removal methods based on reflection region detection and edge attention are mainly reflected in the following aspects:

[0069] 1. Introducing generative deep neural networks improves the generalization of the model: This method is based on generative adversarial networks and solves the problem of the limitation of traditional image processing methods when solving this type of dereflection interference task by calculating the mixing ratio of reflected and transmitted images through some image features or using a mixing matrix for separation. This improves the robustness and generalization of the entire model.

[0070] 2. Construction of a Reflection Region Detection Module: This module learns multi-scale Gaussian-Laplacian features in the blended image to obtain image gradient features, detects and locates regions in the blended image dominated by the reflective image, and obtains a reflection interference probability map. This probability map indicates the probability that a certain region in the image is dominated by the reflective image. This reflection region detection module effectively filters out local noise caused by reflection interference. Compared to traditional image processing methods, which may have flawed prior conditions in specific scenarios, this method achieves better reflection removal results.

[0071] 3. Introducing an edge-attention-based reflection feature enhancement network module: This method uses the mean absolute color difference (MAD) to obtain image edge representations. It calculates the color difference between a pixel and its surrounding pixels in each of the RGB channels of the input image and then calculates a weighted average. The image edge is estimated based on the MAD. The edge map obtained in this way, compared to the binarized edge map obtained from a grayscale image, shows a more significant suppression effect on hard boundaries of reflective regions. Introducing the obtained edge map as an additional channel input into the network adds edge attention and improves the network's feature extraction capability.

[0072] 4. Using multidimensional loss functions to improve output image quality: The loss function of the dereflection interference network mainly comes from two aspects. One is the pixel loss between the generated transmission image and the reference transmission image, and the other part is the perceptual loss and adversarial loss from the network. Pixel loss can optimize the quality of the generated image and make the generated image obtain a higher peak signal-to-noise ratio. Adversarial loss helps to improve the intuitive effect of the network output image. Overall, using a weighted average of multiple loss functions can improve the quality of the generated image.

[0073] Reflection removal methods based on reflection region detection and edge attention have achieved significant technological advancements in network construction, reflection region detection, reflection feature enhancement, and output image quality improvement. These advancements not only improve the accuracy and reliability of the task but also provide new ideas and more effective technical means for deep learning methods to solve the reflection removal problem. Attached Figure Description

[0074] Figure 1 This is a flowchart of the reflection interference removal method based on reflection region detection and edge attention provided in the embodiments of the present invention;

[0075] Figure 2 This is a structural diagram of the certificate acquisition system based on the reflection region detection and edge attention de-reflection interference method provided in the embodiments of the present invention;

[0076] Figure 3 This is a schematic diagram of the network implementation of the reflection region detection module provided in an embodiment of the present invention;

[0077] Figure 4 This is a schematic diagram of the network implementation of the edge attention reflection feature enhancement module provided in an embodiment of the present invention;

[0078] Figure 5 This is a schematic diagram of an image generation network based on generative adversarial networks provided in an embodiment of the present invention;

[0079] Figure 6These are the experimental results of the overall network provided in the embodiments of the present invention, including ablation experiments within the network and comparative experiments with similar algorithms. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0081] like Figure 1 As shown, this embodiment of the invention provides a method for de-reflection interference based on reflection region detection and edge attention, including the following steps:

[0082] S1: The main body adopts the idea of ​​generative adversarial network, and the mixed image containing reflection interference is preprocessed and used as the input of the model;

[0083] S2: The mixed image containing reflection interference input in S1 is input into the reflection region detection network module, and then passes through the reflection feature enhancement network module to obtain the multi-scale Gaussian-Laplacian features of the image and finally participate in the synthesis of image gradient features to enhance reflection.

[0084] S3: The image gradient features obtained in S2 are used as the input of the entire generator network, and the shallow feature extraction network extracts features.

[0085] S4: The features extracted in S3 are transmitted to the discriminator network via skip connections. While obtaining the output image after de-reflection interference, the quality of the generated de-reflection interference image is judged to guide the generation of the next generator until the output quality meets the requirements.

[0086] S1 specifically includes:

[0087] The task of removing image reflection interference usually considers an image containing reflection interference as a mixture of transmitted and reflected images. When using traditional methods to calculate the mixing ratio of the two through some image features or to separate them using a mixing matrix, there are significant limitations in processing image reflection interference removal for ID cards. Specifically, it cannot handle images with uneven reflection intensity and has poor generalization ability. Therefore, a deep learning-based image reflection interference removal algorithm is adopted to achieve this task.

[0088] Since generative adversarial networks can effectively enhance image details during the generation stage, and can also guide the image generation effect by adjusting the attention distribution, improvements are made based on generative adversarial networks.

[0089] The mixed images containing reflection interference acquired by the certificate collection hardware system will undergo image resizing and instance normalization to ensure that the images input to the network are of consistent size and do not interfere with each other. Instance normalization refers to the process of calculating the mean and standard deviation for each input feature map in a single channel, and then using the mean and standard deviation of each channel to normalize the elements within that single channel. Instance normalization has significant advantages in image style transfer. For a feature map x with a batch size of N and length, width, and number of channels h, w, and c, respectively, its specific mathematical definition is as follows:

[0090]

[0091]

[0092] Where x hw Let represent a single-channel image with a scale of h×w, where μ(x) and σ(x) are the mean and variance, respectively, γ and β are variables learned during training, and ε is a small constant.

[0093] Furthermore, S2 specifically includes:

[0094] Based on the characteristic that the gradient of the reflected image is sparser than that of the transmitted image in the mixed image, a reflection region detection network was designed. The mixed image is processed by the multi-scale reflection region detection network to obtain the multi-scale Gaussian-Laplacian features of the image. Then, the reflection features are further refined by the reflection feature enhancement network that integrates channel attention, and the reflection region probability map and the reflection image prediction map are output. In this way, the next stage of image generation is guided by suppressing the reflection-dominant region and enhancing the transmission-dominant region.

[0095] Based on the phenomenon that the edge features of projected images are usually more pronounced than those of reflected images, a reflection feature enhancement network based on edge attention was designed to strengthen the edges of the projected image. The reflection feature enhancement module has two inputs: one is the multi-scale Gaussian-Laplacian features of the original image obtained from the reflection region detection module. out Secondly, the image completely passes through the output I of the reflection area detection module. RD I out Feature enhancement is achieved through four cascaded squeeze-excitation residual modules, followed by I... RD Weighted product is used to obtain image gradient features that enhance reflections;

[0096] Removing reflection interference can be seen as the task of separating the reflected image and the transmitted image in a mixed image. In traditional interference removal algorithms that use image transformation, the selected prior conditions are highly dependent on the scene. In a specific scene, the prior conditions may fail, thus directly affecting the reflection removal effect.

[0097] To ensure the acquisition of multi-scale image gradient features, a multi-scale feature extraction module is used in the network structure. The input image is scaled by multi-scale global pooling, and the image size is scaled to 1 / 2, 1 / 4, 1 / 8 and 1 / 16 of the original scale, respectively. The original scale image and the multi-scale image obtained by pooling are then processed by Gaussian-Laplacian convolution to obtain multi-scale image gradient features.

[0098] Laplacian convolution is a common edge detection method that obtains edge features of an image by performing second-order differentiation. However, it is sensitive to image noise and discrete points. To overcome this problem, Gaussian smoothing is usually performed before Laplacian convolution. The calculation method of its convolution kernel is as follows:

[0099]

[0100] Where σ 2 σ is the standard deviation, and its selection is positively correlated with the image smoothing effect. It can be adjusted based on the network performance. 2 Fine-tuning of values;

[0101] The obtained multi-scale Gaussian-Laplacian feature maps are upsampled to their original scale using bilinear interpolation and then stitched together; To represent the image at the corresponding scale after the downsampling operation, use This represents the upsampled image at the corresponding scale; the output image at this time is:

[0102]

[0103] Three squeeze-excited residual modules are used to enhance the output image I. out The multi-scale Gaussian-Laplacian features are described. The squeeze-excitation residual module models the interdependencies between image channels through two steps: squeezing and excitation. Squeezing is achieved by performing global average pooling on the feature map to obtain the global compressed feature vector of the current feature map. For the input image X∈H×W×C, the squeezing step can be written as the following formula:

[0104]

[0105] Z c This refers to the output related to the c-th channel, with input x. c The activation comes from the convolutional layer; the activation is achieved by obtaining the weights of each channel in the feature map through two fully connected layers, activating it using the sigmoid function, and then using the weighted feature map as the input to the next layer of the network. The activation process can be written as the following formula:

[0106]

[0107] T1 and T2 are two linear layers, and the feature intensity of each channel is adaptively adjusted through the network's global loss function.

[0108] After obtaining the enhanced image features, the PReLU function is used to activate the image features. The formula for the PReLU function can be written as:

[0109]

[0110] Where y i It is the input of the i-th channel, a i Control the slope of the negative half-axis, with an initial value of a. i =0.2, and its value can be adaptively updated through backpropagation during subsequent training. The process of feature enhancement and activation is recorded as f. LOG Finally, the final reflection region detection probability map I is obtained by using a 3×3 convolution kernel and Sigmoid activation normalization. RD I RD It can be represented as:

[0111] I RD =Sigmoid(Conv(f LOG (I out )))

[0112] In this way, the mixed image can obtain preliminary gradient features of the reflection image through the reflection region detection module, providing basic clues for predicting the reflection image. To better predict the reflection image, a reflection feature enhancement module is used to further refine the gradient features of the reflection image. The reflection feature enhancement module requires two inputs: one is the I obtained from the original image through the multi-scale Gaussian-Laplacian feature extraction module. out Secondly, the final output I of the reflection area detection module. RD Feature enhancement is achieved through four cascaded squeeze-excitation residual modules, and then combined with I RD Weighted products are used to further enhance the gradient features of the reflection image;

[0113] To enhance the network's edge representation capabilities, an edge attention module is introduced. Traditional edge detection schemes typically extract only the binary contours of the image. However, in de-reflection interference tasks, obtaining binary edge contours is ineffective for subsequent transmission image reconstruction because even in areas of strong reflection, indistinguishable hard boundaries may exist, interfering with edge detection of the actual transmission image. Therefore, the mean absolute color difference (MAD) method is chosen to obtain image edge representation. The color difference between a pixel and its surrounding pixels is calculated in each of the RGB channels of the input image, and a weighted average is taken. The image edge is estimated based on the MAD, calculated using the following formula:

[0114]

[0115] Where (x,y) are the coordinates of the corresponding pixel in the image, and c refers to the RGB color space channel of the image. The edge map of the mixed image obtained in this way has a more obvious suppression effect on the hard boundaries of the reflective region than the binarized edge map obtained by converting the grayscale image, because the color gamut of the hard boundaries of the reflective region is usually relatively simple and has a small impact on the average absolute color difference. The obtained edge map is introduced into the network as an additional channel input to add edge attention to the network.

[0116] S3 specifically includes:

[0117] The generative network consists of convolutional layers, ReLU activation layers, instance normalization layers, and a coordinated attention layer. The generative network extracts features from the input image. To enhance feature extraction capabilities, a coordinated attention module is introduced. This module performs pooling operations along both row and column dimensions, aggregating vertical and horizontal features into two separate feature maps. These feature maps are then encoded into two attention maps, each capturing long-range dependencies in either row or column space. This attention mechanism efficiently integrates global image features, acquiring not only cross-channel features but also coordinate-sensitive information, aiding in the model's identification and localization of features of interest. In the feature extraction stage of the mixed image, coordinate localization of the reflection region is crucial for removing reflection interference. Channel attention addresses feature acquisition, while spatial attention addresses feature localization. Coordinated attention provides both spatial and channel attention to the network with only a slight increase in computational complexity, enabling the generative network to efficiently acquire transmission image features.

[0118] After obtaining the enhanced transmission image features separated by S2, these features are used as input to a generative adversarial network (GAN) for subsequent feature reconstruction and output. A GAN is a generative model that combines the characteristics of both generative and discriminative models. This network consists of a generator network and a discriminator network. The generator is responsible for generating new data samples based on the feature distribution of the sample data, while the discriminator provides feedback to further optimize the generator. Through this interaction between the two neural networks, the network learns and ultimately guides the data samples generated by the generator to be as close as possible to the real data samples. This network can be used in unsupervised or semi-supervised environments, greatly reducing the requirements for sample data.

[0119] The basic process of a Generative Adversarial Network (GAN) is as follows: the generator network G learns from the random input z of the model, and outputs generated samples G(z). These generated samples and real samples G(x) are then input into a discriminator. The discriminator labels the real samples as 1 and the generated samples as 0. The goal is to train a discriminator D that maximizes the accuracy of sample classification. Based on this, the generator G is optimized by minimizing the adversarial loss log(1-D(G(z))), thereby reducing the accuracy of the discriminator's classification. The two network's evaluation functions are completely opposite, and a zero-sum game is played by minimizing and maximizing the evaluation function V(G,D), as shown in the formula:

[0120]

[0121] Where x represents the real sample data, p data (x) represents the true sample data distribution, z represents the random input data, and p z (z) represents the prior distribution of the random input data; when p z =p data At that time, a zero-sum game has an optimal solution.

[0122] This invention employs the concept of generative adversarial networks (GANs). Taking a mixed image containing reflection interference as input, the network predicts a transmission image without reflection interference. The specific structure is shown in the attached figure. Figure 5 As shown, the overall architecture of this network is a generative adversarial network (GAN). Based on the characteristic that the gradient of the reflected image is sparser than that of the transmitted image in the mixed image, a reflection region detection network and a reflection feature enhancement network are designed in step S2. Furthermore, in some de-reflection interference algorithms, due to excessively deep neural network layers and multiple downsampling layers, the quality of the final output image is lower than that of the input image, resulting in a less than expected de-reflection interference effect. To address this issue, a skip connection is used between the generator and discriminator of the GAN in this invention. This transfers low-level features obtained by the shallow feature extraction network to the output layer network, avoiding the gradient vanishing problem, accelerating the training process, and effectively improving the quality of the output de-reflection interference image.

[0123] S4 specifically includes:

[0124] The main function of the discriminative network is to expand the feature vectors of the acquired feature maps, ultimately obtaining a generated image with the same scale as the input image. In addition, skip connections are introduced between the generator and discriminative networks, which can directly transfer image data from the generator network to the discriminative network, reducing the loss of image detail information. Furthermore, a feature transformer consisting of 6 cascaded residual modules is added between the generator and discriminative networks. Its main function is to convert the image features extracted by the discriminative network into corresponding feature maps, and it can effectively alleviate the gradient vanishing problem during training.

[0125] The decoder consists of deconvolutional layers, ReLU activation layers, normalization layers, Tanh activation layers, etc. The main function of the decoder is to expand the feature vectors obtained from the feature transformer and skip connections to obtain a generated image with the same scale as the input image. At the same time, it participates in the discrimination process of the entire generative adversarial network and improves the precision and generation quality of the entire network through the adversarial learning process.

[0126] The detailed image and intermediate data processing of the reflection removal method based on reflection region detection and edge attention involves multiple steps. The aim is to improve the overall quality of the de-reflection image by learning the separation process of the reflection and projection images and extracting and fusing key features of the reflection image through deep learning. The following is a detailed description of each step of the method:

[0127] (1) Data input and preprocessing

[0128] Image data preprocessing:

[0129] Size Resizing: The original images affected by reflection interference are resized uniformly. If necessary, qualitative cropping is performed to ensure that all input images have the same dimensions, which facilitates subsequent feature separation and feature fusion.

[0130] Instance normalization: The image is normalized to ensure that each input sample image has an independent normalization parameter, effectively avoiding mutual influence between training samples.

[0131] (2) Image feature separation and extraction

[0132] By utilizing a reflectivity region detection network and an edge-attention-based reflectivity feature enhancement network, the transmissive image is separated from the preprocessed mixed image, and its edge features are enhanced. These features can effectively distinguish between reflective and transmissive elements in the mixed image and better reflect the feature data of the transmissive image that needs to be separated.

[0133] (3) Feature fusion generation

[0134] Generative Adversarial Network Construction: Design and implement a Generative Adversarial Network (GAN) consisting of a generator network, a feature converter, and a discriminator network. It can take the pre-extracted input image after feature separation and enhancement as input and learn to produce a better output image free from reflection interference.

[0135] Encoder: The main component of the generator network, its function is to expand and fuse the feature vectors of the input projected image to obtain a generated image with the same scale as the input image after size resizing in step S1, and provide it to subsequent modules for learning and output.

[0136] Feature converter: Converts the image features extracted by the encoder into corresponding feature maps.

[0137] (4) Image generation and discrimination

[0138] Decoder: Expands the feature vectors of the acquired feature map and generates the final output image; as the main component of the discriminator, it participates in the judgment of the quality of the output image and guides the training process of the entire network.

[0139] Through the detailed image separation, image feature extraction, fusion and reconstruction process described above, this method can fully capture the differences between reflected and transmitted images in the mixed image, improve the ability to separate and extract transmitted image features, thereby improving the quality and robustness of the entire network structure in removing reflection interference. In particular, when this method is used in the certificate collection system, it can well adapt to the task of removing reflection interference caused by global and local strong light and polarized light environments.

[0140] Example 1: Reflection interference in a real border crossing environment.

[0141] The hardware device using the method of this invention operates in a real border inspection port environment, acquiring images of documents and certificates passing through the pilot site during daily traffic. Because the device is in a real port environment, there is external lighting that can affect image quality and easily cause reflection interference; therefore, this embodiment is effective. The entire de-reflection interference process includes:

[0142] 1. Image preprocessing. The image, after size normalization preprocessing, is input into the entire network.

[0143] 2. Reflection region detection. Used to obtain multi-scale Gaussian-Laplacian features of the image.

[0144] 3. Reflection Feature Enhancement. By suppressing the reflection-dominant region and enhancing the transmission-dominant region, a reflection region probability map and a reflection image prediction map are output.

[0145] 4. Feature Fusion and Image Generation. The output reflection region probability map and reflection image prediction map, enhanced by reflection features, are input into the generative adversarial network to obtain the output image after reflection interference is removed. This image guides the generation of the next generator in the next round until the output quality meets the requirements.

[0146] Example 2: Reflection interference in a laboratory environment with strong light.

[0147] In a laboratory environment, the external environment is changed to strong light, making the reflected interference stronger than in typical indoor or outdoor environments. The entire process for removing reflected interference is as follows:

[0148] 1. Image preprocessing. The image, after size normalization preprocessing, is input into the entire network.

[0149] 2. Reflection Region Detection. The image passes through the reflection region detection module to obtain multi-scale Gaussian-Laplacian features.

[0150] 3. Reflection Feature Enhancement. A reflection feature enhancement network based on edge attention further refines the reflection features, outputting a reflection region probability map and a reflection image prediction map. This, in turn, guides the next stage of image generation by suppressing reflection-dominant regions and enhancing transmission-dominant regions.

[0151] 4. Feature Fusion and Image Generation. The output reflection region probability map and reflection image prediction map, enhanced by reflection features, are input into the generative adversarial network (GAN). This process obtains the de-reflection interference-free output image under strong reflective light conditions. Simultaneously, the quality of the generated de-reflection interference-free image is judged to guide the generation of the next round of the generator, until the output quality meets the requirements.

[0152] These two embodiments demonstrate the broad applicability of the reflection interference removal method based on reflection region detection and edge attention in complex external light source application scenarios. From general indoor and outdoor environments to laboratory environments under strong light source conditions, this method can effectively enhance the system's reflection interference removal capability and improve its generalization and robustness.

[0153] like Figure 2 As shown, this embodiment of the invention provides a reflection interference removal system based on reflection region detection and edge attention to implement the aforementioned reflection interference removal method. The system includes an image preprocessing module, a reflection region detection network module, a reflection feature enhancement network module, a generative adversarial network module (including a generative network module), and a discriminant network module.

[0154] The image preprocessing module is used to normalize the size of images after they are input into the network.

[0155] The reflection region detection network module, connected to the image preprocessing module, is used to obtain multi-scale Gaussian-Laplacian features of the image in the reflection region detection network.

[0156] The reflection feature enhancement network module, connected to the reflection region detection network module, is used to further refine the reflection features through the reflection feature enhancement network that incorporates channel attention, and outputs the probability map of the reflection region and the prediction map of the reflection image to the generative adversarial network.

[0157] The generative adversarial network module includes a generative network module and a discriminative network module, which guide image generation by suppressing the reflection-dominant region and enhancing the transmission-dominant region through the joint action of the generative and discriminative networks.

[0158] This invention provides a computer device, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the de-reflection interference method based on reflection region detection and edge attention.

[0159] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the de-reflection interference method based on reflection region detection and edge attention.

[0160] This invention provides a certificate image data processing terminal, which is used to implement the certificate image acquisition system based on the reflection region detection and edge attention de-reflection interference method.

[0161] This invention provides a cross-platform computer application that can be deployed on Windows, Linux, and most domestic operating systems. This application is used to combine specific hardware to complete the steps of the de-reflection interference method based on reflection region detection and edge attention.

[0162] The following are two specific examples demonstrating how to apply a de-reflection interference method based on reflective region detection and edge attention.

[0163] This invention uses a total of four datasets, including one real-world scene reflection image dataset, two incremental datasets, and one test dataset.

[0164] During the model training phase, this invention uses a synthetic dataset, the data source of which is SIR containing three real-world scenarios. 2The dataset contains a total of 454 pairs of images and 2200 pairs of synthetic images from real-world scenes. This includes 90 pairs of images synthesized from indoor and outdoor scenes using Gaussian blur, and 200 pairs of images from the Nature dataset taken through portable glass. These data together form the training dataset, named RSRR2944. This dataset contains pairs of transmitted images and synthetic images that combine reflected images, which are of good guiding significance for model training.

[0165] To improve training effectiveness, this method additionally uses two incremental datasets. These datasets enhance the images during pre-training, including one dataset of real ID card reflective images and one dataset of synthetic ID card reflective images. The real ID card dataset consists of 615 pairs of visible light ID card images with reflective interference actually captured by the ID card acquisition system mentioned in this invention, along with the corresponding directly captured projected ID card images. The synthetic ID card dataset uses a synthesis method, specifically, considering the captured mixed image I as a linear combination of the transmitted image B and the reflected image R. Its mathematical model can be expressed as the following formula:

[0166] I = αB + (1-α)(K*R)

[0167] Where α is a hyperparameter representing transmittance, its value is positively correlated with transmission intensity; the higher the transmission intensity, the closer the acquired image is to a pure transmission image B. K is a Gaussian blur kernel, and * represents a convolution operation. To improve the realism of the reflection image, a Gaussian smoothing kernel of random size from 3 to 15 pixels is used to perform Gaussian blur processing on the reflection image. At the same time, to avoid image overflow, the sum of the two synthesis coefficients is set to 1, and linear superposition is performed to obtain a mixed image containing reflection interference. This invention uses commonly used ID card images (712 images) as transmission images and common ID card collection background images (445 images) as reflection images. The above method is used to synthesize ID card image data containing reflection interference, and data augmentation processing is performed through flipping, rotating, random cropping, etc., to obtain a total of 5520 pairs of synthesized ID card datasets. This incremental dataset is named PPRR6135.

[0168] To effectively evaluate the algorithm's performance, this method selected 800 images from both domestic and international publicly available reflective image test sets and a dataset of reflective images for document collection. 300 images were randomly selected from RSRR2044 as the domestic and international publicly available reflective image test set, and 500 images were extracted from visible light document images actually collected by our hardware system and PPRR6135 as the document collection reflective image test set. The network structure was implemented using the PyTorch deep learning framework, employing the Adam optimizer with optimizer parameters β1 and β2 set to 0.8 and 0.99 respectively. Training was performed with a fixed learning rate lr = 0.0002. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were used to evaluate image quality. The PSNR (unit: dB) was defined based on mean squared error, and image quality was represented by calculating the mean squared error between the input and output images, expressed as the formula:

[0169]

[0170] Where MAXI represents the maximum pixel value of the image. For RGB images, the MSE of each of the three channels needs to be calculated separately and then averaged. Generally, the higher the PSNR value, the better the image quality. Structural similarity is a parameter used to measure the similarity between the generated image and the original transmission image. It measures the similarity of the images from three aspects: brightness, contrast, and structure, and is expressed by the following formula:

[0171]

[0172] Where μ x and μ y These are the average pixel values ​​of the input image x and y, respectively. The variances of inputs x and y are σ, respectively. xy Let c1 and c2 be the covariances of the input images x and y, respectively. Both c1 and c2 are constants, making the SSIM range [0,1]. The closer the SSIM value is to 1, the higher the similarity between the input images x and y. During testing, ablation experiments involving reflection region detection and reflection feature enhancement were conducted to illustrate the impact of the module network on the experiment. The complete experiment achieved a PSNR of 23.11 and an SSIM value of 0.836, demonstrating superior performance compared to similar de-reflection algorithms. Figure 6 exhibit.

[0173] In summary, compared with existing related methods, the method proposed in this invention is the first algorithm model to perform reflection region detection and reflection feature enhancement to remove reflection interference. It is superior in accuracy, generated image quality, and running time, and is currently being tested on our company's hardware products, which is conducive to the practical application of this algorithm.

[0174] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0175] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for reflection interference removal based on reflection region detection and edge attention, characterized in that, Includes the following steps: S1: Preprocess the mixed image containing reflection interference, and use the preprocessed mixed image as the input of the reflection region detection network module; S2: Input the preprocessed mixed image from S1 into the reflection region detection network module to obtain multi-scale Gaussian-Laplacian features of the image, and input the multi-scale Gaussian-Laplacian features and the reflection region detection output into the reflection feature enhancement network module to finally participate in the synthesis of image gradient features that enhance reflection; S3: The image gradient features obtained in S2 are used as the input to the entire generator network, and the shallow feature extraction network extracts features. S4: The features extracted in S3 are transmitted to the discriminator network via skip connections. While obtaining the output image after de-reflection interference, the quality of the generated de-reflection interference image is judged to guide the generation of the next generator network until the output quality meets the requirements. S2 specifically includes: A reflection region detection network is set up. The mixed image is passed through the multi-scale reflection region detection network to obtain the multi-scale Gaussian-Laplacian features of the image. Then, the reflection feature enhancement network module further refines the reflection features and outputs the reflection region probability map and the reflection image prediction map. Then, the next stage of image generation is guided by suppressing the reflection-dominant region and enhancing the transmission-dominant region. The reflection feature enhancement network module integrates channel attention and edge attention. The reflection feature enhancement network module has two inputs: one is the multi-scale Gaussian-Laplacian features of the preprocessed mixed image containing reflection interference, obtained in the reflection region detection network module. Secondly, the image completely passes through the output of the reflection region detection network module. , Feature enhancement is achieved through four cascaded squeeze-excitation residual modules, followed by... Weighted product is used to obtain image gradient features that enhance reflections; The reflection region detection network module uses a multi-scale feature extraction module. The mixed image containing reflection interference after S1 preprocessing is scaled by multi-scale global pooling, and the image size is scaled to 1 / 2, 1 / 4, 1 / 8 and 1 / 16 of the original scale, respectively. The original scale image and the multi-scale image obtained by pooling operation are respectively processed by Gaussian-Laplacian convolution to obtain multi-scale image gradient features. Before performing Laplacian convolution, a Gaussian smoothing operation is performed. The convolution kernel is calculated as follows: ； in The standard deviation is positively correlated with the image smoothing effect and can be used to adjust the network performance. Fine-tuning of values; The acquired multi-scale Gaussian-Laplacian feature maps were upsampled to their original scale and then stitched together using bilinear interpolation. To represent the image at the corresponding scale after the downsampling operation, use This represents the upsampled image at the corresponding scale; the output image at this time is: ； Four squeeze-excited residual modules are used to enhance the output image. The multi-scale Gaussian-Laplacian features are described; the squeeze-excitement residual module models the interdependencies between image channels through two steps: squeezing and excitation. Squeezing is achieved by performing global average pooling on the feature map to obtain a globally compressed feature vector for the current feature map. For the input image... The extrusion step can be written as the following formula: ； in It refers to the first Each channel-related output, input The activation comes from the convolutional layer; the activation is achieved by obtaining the weights of each channel in the feature map through two fully connected layers, activating it using the sigmoid function, and then using the weighted feature map as the input to the next layer of the network. The activation process can be written as the following formula: ； in and It consists of two linear layers, and ultimately the feature intensity of each channel is adaptively adjusted through the network's global loss function; After obtaining the enhanced image features, the PReLU function is used to activate the image features. The formula for the PReLU function can be written as: ； in It is the first Input of each channel, Controlling the slope of the negative half-axis, the initial value is taken as... Its value can be adaptively updated through backpropagation during subsequent training, recording the feature enhancement and activation process as follows. Finally through Convolution kernel, using Activate normalization to obtain the final probability map of reflection region detection. , It can be represented as: ； To enhance the network's edge representation capabilities, an edge attention module is introduced. The mean absolute color difference (MAD) method is used to obtain image edge representations. The color difference between a given pixel and its surrounding pixels is calculated in each of the RGB channels of the input image, and then a weighted average is taken. The image edge is estimated based on the MAD, calculated using the following formula: ； in These are the coordinates of the corresponding pixel in the image. This refers to the RGB color space channels of the image. The edge map of the mixed image obtained in this way has a more obvious effect on suppressing the hard boundaries of the reflective area compared with the binarized edge map obtained by converting the grayscale image. The obtained edge map is introduced into the network as an additional channel input to add edge attention to the network. The basic process of generative adversarial networks (GANs) is that the generator network G processes random inputs from the model. The generator network learns and outputs generated samples. Then the generated samples are compared with the real samples. The input is fed into a discriminator network, which labels real samples as 1 and generated samples as 0. The goal is to train a discriminator network D that maximizes the accuracy of sample classification and minimizes the adversarial loss. To optimize the generator network G, thereby reducing the classification accuracy of the discriminator network, the minimum-maximum estimation function is used. To optimize, the function is: ； in Represents real sample data. Represents the distribution of real sample data. This indicates randomly input data. This represents the optimization of the prior distribution of randomly input data.

2. The de-reflection interference method based on reflection region detection and edge attention as described in claim 1, characterized in that, S1 specifically includes: The image size will be resized and instance normalized for mixed images containing reflection interference captured by the certificate acquisition hardware system to ensure that the images input to the network are of consistent size and do not interfere with each other. Instance normalization refers to the process of calculating the mean and standard deviation for each input feature map in a single channel, and then using the mean and standard deviation of each channel to normalize the elements within that single channel. Instance normalization has significant advantages in image style transfer, especially for batches of N with length, width, and number of channels. and Feature map Its specific mathematical definition is as follows: ； ； ； in Indicates the single-channel lower scale size as The image, and These are the mean and variance, respectively. and For variables learned during the training process, It is a constant with a small value.

3. The de-reflection interference method based on reflection region detection and edge attention as described in claim 1, characterized in that, S3 specifically includes: The generator network consists of convolutional layers, ReLU activation layers, instance normalization layers, and a coordinated attention layer. The generator network extracts features from the input image. To enhance feature extraction capabilities, a coordinated attention module is introduced. This module performs pooling operations along both row and column dimensions, aggregating vertical and horizontal features into two separate feature maps. These feature maps are then encoded into two attention maps, each capturing long-range dependencies in either row or column space. This attention mechanism efficiently integrates global image features, acquiring not only cross-channel features but also coordinate-sensitive information, aiding in the model's identification and localization of features of interest. In the feature extraction stage of the mixed image, coordinate localization of the reflection region is crucial for removing reflection interference. Channel attention addresses feature acquisition, while spatial attention addresses feature localization. Coordinated attention provides both spatial and channel attention to the network with only a slight increase in computational complexity, enabling the generator network to efficiently acquire transmission image features. After obtaining the enhanced image gradient features separated by S2, these features are used as input to the Generative Adversarial Network (GAN) for subsequent feature reconstruction and output. The GAN combines the characteristics of both generative and discriminative models. It consists of a generator network and a discriminator network. The generator network generates new data samples based on the feature distribution of the sample data, while the discriminator network provides feedback to further optimize the generator network. Through this interaction between the two neural networks, the network learns and ultimately guides the data samples generated by the generator network to be as close as possible to the real data samples. This network can be used in unsupervised or semi-supervised environments, greatly reducing the requirements for sample data.

4. The anti-glare interference method based on reflection area detection and edge attention of claim 1, wherein, S4 specifically includes: Skip connections are introduced into the generator network and the discriminator network, which can directly transfer image data from the generator network to the discriminator network, reducing the loss of image detail information. In addition, a feature transformer consisting of 6 cascaded residual modules is added between the generator network and the discriminator network. Its main function is to convert the image features extracted by the discriminator network into corresponding feature maps, and it can effectively alleviate the gradient vanishing problem during training. The decoder consists of deconvolutional layers, ReLU activation layers, normalization layers, Tanh activation layers, etc. The main function of the decoder is to expand the feature vectors obtained from the feature transformer and skip connections to obtain a generated image with the same scale as the input image. At the same time, it participates in the discrimination process of the entire generative adversarial network and improves the precision and generation quality of the entire network through the adversarial learning process.

5. A reflection interference removal system based on reflection region detection and edge attention, implementing the reflection interference removal method based on reflection region detection and edge attention as described in any one of claims 1 to 4, characterized in that, The system includes an image preprocessing module, a reflectance region detection network module, a reflectance feature enhancement network module, a generative adversarial network module including a generator network module, and a discriminator network module; The image preprocessing module is used to normalize the size of images after they are input into the network. The reflection region detection network module, connected to the image preprocessing module, is used to obtain multi-scale Gaussian-Laplacian features of the image in the reflection region detection network. The reflection feature enhancement network module, connected to the reflection region detection network module, is used to further refine the reflection features through the reflection feature enhancement network that incorporates channel attention, and outputs the probability map of the reflection region and the prediction map of the reflection image to the generative adversarial network. The generative adversarial network module includes a generator network module and a discriminator network module, which guide image generation by suppressing the reflection-dominant region and enhancing the transmission-dominant region through the joint action of the generator network and the discriminator network.

6. A computer device comprising a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the de-reflection interference method based on reflection region detection and edge attention as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the de-reflection interference method based on reflection region detection and edge attention as described in any one of claims 1 to 4.

8. A certificate image data processing terminal, the terminal being used to implement the certificate image acquisition system as described in claim 5, which is based on the reflection region detection and edge attention method for de-reflection interference.

Images