A bar code enhanced image processing method and system based on a generative adversarial network

Abstract

The application provides a bar code enhanced image processing method and system based on a generative adversarial network. The method comprises the following steps: obtaining a training set and an initial model; the training set comprises a plurality of training data, each training data comprising a sample matrix bar code image and corresponding label data; the initial model is an adversarial network constructed by a generator model and a discriminator model; a bar code enhancement model is obtained by training the model using the training set according to the initial model and a target loss function; the target loss function comprises a generator loss function using a weighted compound loss technique and a discriminator loss function using a binary cross-entropy loss technique; a data matrix bar code image in a real environment is collected by an industrial camera and input into the bar code enhancement model to obtain a standard binary bar code. The application improves the bar code decoding success rate by constructing an adversarial training architecture composed of a generator and a discriminator.

Description

[0001] This application claims partial priority to Chinese patent application No. 202510337412.2, filed on March 21, 2025, the contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing technology, and in particular to a barcode enhancement image processing method and system based on generative adversarial networks. Background Technology

[0003] In smart manufacturing scenarios, barcodes serve as the core carrier for product traceability. Taking DM barcodes as an example, in SMT (Surface Mount Technology) production lines, which involve multiple processes such as high-speed placement, reflow soldering, AOI (Automated Optical Inspection), and SPI (Solder Solder Paste Inspection), each PCB board, electronic component, or assembly is usually marked with a DM barcode to ensure accurate production data management. Its identification reliability plays a key role in production traceability, quality control, and automated management.

[0004] However, barcodes in industrial environments often face interference from multiple factors: blurred module edges due to printing defects, structural damage caused by mechanical wear, contrast reduction due to uneven lighting, and motion blur caused by camera movement. Traditional solutions mainly rely on manually designed feature extraction methods such as threshold segmentation and morphological operations, but their generalization ability is greatly limited when facing complex scenarios, resulting in unsatisfactory DM barcode decoding success rates.

[0005] Therefore, there is an urgent need to provide a barcode enhancement image processing method and system based on generative adversarial networks to improve the decoding success rate. Summary of the Invention

[0006] To address the problem of unsatisfactory barcode decoding success rate in existing technologies, this application provides a barcode enhancement image processing method and system based on generative adversarial networks.

[0007] The objective of this application is achieved through the following technical solution:

[0008] In a first aspect, this application provides a barcode enhancement image processing method based on generative adversarial networks, the method comprising:

[0009] Obtain a training set and an initial model; the training set includes multiple training data, each training data including a sample matrix barcode image and its corresponding labeled data; the initial model is an adversarial network constructed from a generator model and a discriminator model.

[0010] Based on the initial model and the target loss function, the model is trained using the training set to obtain the barcode enhancement model; the target loss function includes a generator loss function using weighted composite loss technology and a discriminator loss function using binary cross-entropy loss technology.

[0011] A data matrix barcode image is captured using an industrial camera in a real environment and input into the barcode enhancement model to obtain a standard binarized barcode.

[0012] The generator loss function consists of an adversarial loss function, an image inpainting loss function, a feature loss function, and a structural similarity loss function, which are weighted and summed using preset weights.

[0013] The beneficial effects of this technical solution are as follows: Compared to traditional barcode image processing methods, which often require extensive manual intervention for image preprocessing and parameter adjustment to adapt to different barcode quality and environmental conditions, the generative adversarial network (GAN)-based method provided in this solution automates barcode image enhancement processing. This reduces the need for manual complex image processing parameter settings and extensive post-processing corrections, saving manpower and time costs and improving processing efficiency, making it suitable for the barcode data processing needs of large-scale industrial applications. Furthermore, by leveraging the generative capabilities of GANs, low-quality barcode images captured under various complex conditions in industrial environments are converted into binary barcode images conforming to the ECC200 standard. This effectively repairs defects and noise in the barcode images, enhances the edge and module features of the barcode, making it easier for barcode recognition devices to accurately read and decode, thereby significantly improving the barcode recognition success rate, reducing recognition failures due to barcode quality issues, and ensuring the smooth operation of barcode applications in the production process. On the other hand, by adopting an end-to-end learning architecture, low-quality barcode images are directly input into the model. After one forward propagation calculation, a standard-compliant binary barcode image can be output. Compared with traditional multi-stage barcode image processing methods, this avoids cumbersome preprocessing, feature extraction, post-processing, and other steps, reduces data transmission and conversion time between different processing stages, and improves the efficiency and real-time performance of the entire barcode image processing workflow. This meets the requirements for rapid barcode recognition and processing on high-speed production lines, thereby improving overall production efficiency. By comprehensively considering adversarial loss, image inpainting loss, feature loss, and structural similarity loss, different loss functions constrain and optimize the generator from different perspectives, enabling the model to learn more comprehensive and richer image feature representations. Adversarial loss helps the model learn data distribution and improves its adaptability to different barcode image styles and types; image inpainting loss ensures the model's accuracy in pixel-level restoration, adapting to barcode images under various imaging conditions; feature loss enables the model to understand the essential features of barcode images at the semantic level, handling input images of varying complexity and quality; structural similarity loss strengthens the model's attention to barcode structural details, ensuring the standardization and integrity of the barcode during generation. This comprehensive optimization approach enhances the model's generalization ability, enabling it to consistently output high-quality enhancement results when faced with different barcode images in various industrial environments. The weighted summation method with preset weights provides flexibility for model adjustment and optimization. The weights of different loss functions can be adjusted according to actual application needs and data characteristics to highlight or balance performance in specific areas. For example, for barcodes, the weight of structural similarity loss can be appropriately increased; when a focus on the overall realism of the generated image is needed, the weight of adversarial loss can be increased.The aforementioned flexibility allows the model to better adapt to different practical needs and industrial environments. The performance of the generator model can be optimized and customized through simple weight adjustments, without the need for large-scale modifications and retraining of the model structure.

[0014] Secondly, this application also provides a barcode enhancement image processing method based on generative adversarial networks. This method is used to perform barcode identification region extraction, geometric regularization transformation, and structural defect completion on barcode images collected in an industrial environment, including the following steps:

[0015] Construct a training set; the training set contains multiple matrix barcode image data, including data on deformation, occlusion, and printing defects, collected by industrial cameras in a real production line environment, and their corresponding annotation data; the annotation data is the corresponding matrix barcode image data in which the barcode area position is manually marked, and a standard format barcode image is generated according to the encoding protocol applicable to the barcode, which is geometrically aligned with the barcode protocol and corresponds one-to-one with the spatial position of the area.

[0016] An adversarial network model, consisting of a generator and a discriminator, is used as the initial model. The generator is built on the UNet architecture and optimized through a weighted combination of adversarial loss, image reconstruction loss, structural similarity loss, and feature consistency loss. It is used to automatically extract barcode identification regions from 2D barcode images in real industrial scenarios and to achieve geometric regularization of the barcode structure and structural completion of missing regions. The discriminator adopts the ResNet architecture and uses a binary cross-entropy loss optimizer to determine the structural similarity between the generator's output image and the standard barcode image generated according to the barcode encoding protocol.

[0017] The initial model is trained based on a composite loss function to obtain an enhanced model with barcode extraction, geometric regularization and defect completion capabilities;

[0018] Data matrix barcode images are captured using industrial cameras in real industrial environments, and then input into a trained augmentation model to output standardized barcode images with one or more of the following features:

[0019] Filter background interference areas; perform geometric regularization processing on structural distortions in the barcode to restore each module to a square unit that conforms to the standard matrix arrangement; for barcode module missing due to one or more factors such as occlusion, dirt, scratches, or ink loss, perform structural consistency completion based on contextual semantic information; generate results that meet the structural rules of the corresponding barcode encoding protocol, and output a standard binarized image with clear boundaries and regular geometric structure.

[0020] Thirdly, this application also provides a barcode enhancement image processing system based on generative adversarial networks, the system comprising:

[0021] The information acquisition module is used to acquire the training set and the initial model; the training set includes multiple training data, each training data includes a sample matrix barcode image and its corresponding annotation data; the initial model is an adversarial network constructed from a generator model and a discriminator model.

[0022] The model training module is used to train the model using the training set based on the initial model and the target loss function to obtain the barcode enhancement model; the target loss function includes a generator loss function using weighted composite loss technology and a discriminator loss function using binary cross-entropy loss technology.

[0023] The barcode acquisition module is used to capture data matrix barcode images in a real environment using an industrial camera and input them into the barcode enhancement model to obtain a standard binary barcode. Attached Figure Description

[0024] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0025] Figure 1 This is a flowchart illustrating a barcode enhancement image processing method provided in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of the structure of a generative adversarial network provided in an embodiment of this application;

[0027] Figure 3 This is a schematic diagram of a barcode enhancement image processing system provided in an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the QR code provided in the embodiments of this application;

[0030] Figure 6 These are some examples of image enhancement data provided in the embodiments of this application. Detailed Implementation

[0031] The present application will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments. The implementation process of the present application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific implementation procedures, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application and not for limiting the scope of protection of the present application.

[0032] The following is a brief description of the technical field and related terms of the embodiments of this application, so as to facilitate understanding by those skilled in the art.

[0033] Generative Adversarial Networks (GANs) are generative models that learn by having two neural networks play against each other.

[0034] Data Matrix (DM) barcode images are used because DM barcodes have the smallest size of all barcodes currently available, making them suitable for identifying small parts. They can be directly printed onto small parts and are widely used in small items such as circuits and pharmaceuticals, as well as in assembly line production processes in manufacturing. The original barcode image mentioned in this application is a DM barcode image.

[0035] In recent years, deep learning technology has developed rapidly, and significant progress has been made in the field of image super-resolution. Deep learning-based image super-resolution methods have been gradually extended to barcode recognition tasks. Super-resolution techniques such as SRCNN and EDSR can effectively improve the clarity of low-resolution images, thereby enhancing the readability of blurred barcodes. However, these improvements lack consideration for field applications. On the one hand, super-resolution techniques for blurred barcodes mainly rely on simulation data for training, but due to the differences between simulation data and actual blurred barcodes, the effect of improving clarity on some blurred images is not ideal. On the other hand, in actual industrial environments, factors affecting barcode recognition are not limited to blurriness, but also include deformation, occlusion, and lighting differences. Once these problems exist, relying solely on super-resolution techniques to improve decoding success rates is not ideal, therefore further optimization and improvement are still needed.

[0036] In other words, related technologies focus more on super-resolution methods to improve the clarity of low-resolution images. However, the technical solution provided in this application includes the following barcode enhancement process: data preprocessing – barcode extraction (outputting a binarized barcode image) – post-processing (binarized barcode completion, shape reshaping). The technical solution provided in this application is not simply a super-resolution method focused on improving image clarity, but a complete barcode enhancement processing scheme. The resulting binarized barcode image can be used for direct recognition. Through generative adversarial networks and specific data augmentation strategies, it achieves comprehensive enhancement of low-quality barcode images, making them conform to the corresponding barcode protocol (e.g., ECC200) standard and improving the barcode recognition success rate.

[0037] In summary, to address the recognition failure problem caused by degraded barcode image quality in industrial environments, this application provides a barcode enhancement image processing method and system based on generative adversarial networks (GANs). By constructing an adversarial training architecture consisting of a generator and a discriminator, the generator can autonomously learn and accurately extract real barcode regions in complex backgrounds, automatically perform geometric regularization of the barcode, and finally output a standard binarized barcode image conforming to the corresponding barcode protocol specification format. Simultaneously, the discriminator ensures that the generated result's geometric structure better conforms to its barcode protocol standard. The generator adopts a UNet structure, and the discriminator adopts a ResNet18 structure. Combined with a loss function, this ensures that the generated barcode image is aligned with the barcode protocol specification format at multiple levels, including pixel level, semantic level, and structural level.

[0038] The technical solutions of the embodiments of this application and how the technical solutions of the embodiments of this application solve the above-mentioned technical problems will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the various embodiments or technical features described below can be arbitrarily combined to form new embodiments, and the same or similar concepts or processes may not be described again in some embodiments. Obviously, the described embodiments are some embodiments of the embodiments of this application, but not all embodiments.

[0039] First method embodiment.

[0040] See Figure 1 , Figure 1 This is a flowchart illustrating a barcode enhancement image processing method provided in an embodiment of this application. This embodiment provides a barcode enhancement image processing method based on a generative adversarial network, the method comprising:

[0041] S101, Obtain the training set and initial model; the training set includes multiple training data, each training data includes a sample matrix barcode image and its corresponding annotation data; the initial model is an adversarial network constructed from a generator model and a discriminator model;

[0042] It is worth noting that the datasets in related technologies primarily use blurred images for training super-resolution models. These models then recover the images, making the blurred barcodes clearer. Even after super-resolution, the barcode still requires further processing, including barcode extraction (binarization), before it can be read. The training set used for model training in this application... Each training data in the training set is composed of samples This matrix consists of barcode images and their corresponding labeled data. The sample matrix barcode images respectively cover those appearing in industrial environments. Various situations, such as different printing methods like thermal, inkjet, and laser engraving, as well as different defects such as normal, deformed, obscured, and blurred. The barcode samples were collected. The labeled data, obtained by annotating the sample barcode images, is used to guide model training and help... The model enables automatic extraction of various barcode regions under diverse background interference conditions. Accurately complete the barcode area With geometric regularity.

[0043] The initial model is an adversarial network consisting of a generator model and a discriminator model. The generator model aims to convert the input low-quality barcode image into a binary barcode image conforming to the ECC200 standard; the discriminator model is used to distinguish the barcode image generated by the generator model from the real standard barcode image, providing feedback to the generator model and prompting it to continuously optimize the generation results.

[0044] S102, Based on the initial model and the target loss function, the model is trained using the training set to obtain the barcode enhancement model; the target loss function includes a generator loss function using weighted composite loss technology and a discriminator loss function using binary cross-entropy loss technology.

[0045] The target loss function guides the model training direction. The generator loss function employs a weighted composite loss technique, comprehensively considering adversarial loss, image inpainting loss, feature loss, and structural similarity loss. Adversarial loss ensures the generated image closely approximates the distribution of the real barcode image, giving it high realism. Image inpainting loss calculates the L1 distance between the generated and target images, ensuring pixel-level reconstruction accuracy. Feature loss improves the semantic consistency of the generated image by comparing the differences in features from the discriminator's intermediate layers. Structural similarity loss optimizes the boundary clarity of the barcode module, making it easier to decode. The discriminator loss function uses binary cross-entropy loss to measure the discriminator's classification accuracy between real and generated barcode images, helping the discriminator model better learn the features that distinguish between the two.

[0046] During training, the training set prepared by S101 is used, and the sample matrix barcode images are input into the initial model. The generator model accurately extracts the barcode region and completes the geometric structure based on the input barcode image, which contains background interference, deformation, and occlusion. The discriminator module simultaneously receives the generated barcode image and the binarized barcode image of the corresponding barcode protocol standard, compares and judges them, and outputs the evaluation result of the generated image. The loss values ​​of the generator and discriminator are calculated according to the target loss function. The parameters of the model are adjusted by optimization algorithms (such as Adam, SGD, etc.), and the training process is continuously iterated. This enables the generator to generate enhanced images that are increasingly closer to the real standard barcode image, while the judgment ability of the discriminator module is also improved. Finally, a trained barcode enhancement model that meets the training standard is obtained.

[0047] S103, an industrial camera is used to capture a data matrix barcode image in a real environment and input into the barcode enhancement model to obtain a standard binary barcode. Obtaining a standard binary barcode means: automatically extracting the barcode area and performing geometric regularization and morphological reshaping on the extraction result to generate a binary barcode image with a standardized format that is geometrically aligned with its barcode protocol.

[0048] In practical applications, industrial cameras are used to acquire data matrix barcode images in real working environments. Data matrix barcode images can be affected by various practical factors such as uneven lighting, camera movement, and barcode contamination, leading to a decrease in image quality and making it difficult to directly and accurately identify and decode them.

[0049] See Figure 2 , Figure 2 This is a schematic diagram of a generative adversarial network (GAN) provided in this application. Input barcode data (sample matrix barcode images from the training data) to the generator (model) yields generated data (enhanced barcode images). The labeled data corresponding to the sample matrix barcode images are used as label data. The generated data and label data are then input to the discriminator (model) for model training.

[0050] The collected barcode image matrix from a real-world environment is input into a pre-trained barcode enhancement model. The model automatically extracts the barcode region and regularizes the geometry of the extracted result, aligning its geometric structure with the standard format of the barcode protocol. The enhanced barcode image can be considered to have higher quality and readability, better meeting the requirements of subsequent barcode recognition and decoding systems, and improving the efficiency and reliability of the entire barcode application process.

[0051] Therefore, compared to traditional barcode image processing methods, which often require significant manual intervention for image preprocessing and parameter adjustment to adapt to varying barcode quality and environmental conditions, this technical solution offers a generative adversarial network (GAN)-based approach. On one hand, it automates barcode image extraction and enhancement, reducing manual intervention in complex image processing parameter settings and extensive post-processing corrections. This saves manpower and time while improving processing efficiency, making it suitable for the barcode data processing needs of large-scale industrial applications. On the other hand, by leveraging the generative capabilities of GANs, it automatically extracts barcode regions from barcode images captured under various complex conditions in industrial environments, including those with background interference, deformation, and occlusion. It then regularizes the geometry of the extracted results, effectively completing the barcode extraction and geometric regularization tasks. This significantly improves barcode recognition rates, reduces recognition failures due to barcode quality issues, and ensures the smooth operation of barcode applications in production processes. On the other hand, by adopting an end-to-end learning architecture, barcode images containing background interference, deformation, and occlusion are directly input into the model. After one forward propagation calculation, a standard-compliant binary barcode image can be output. Compared with traditional multi-stage barcode image processing methods, this avoids cumbersome preprocessing, feature extraction, post-processing, and other steps, reduces the data transmission and conversion time between different processing stages, and improves the efficiency and real-time performance of the entire barcode image processing process. It can meet the requirements of rapid barcode recognition and processing on high-speed production lines and improve overall production efficiency.

[0052] The training set is used to train the model. The training termination condition for obtaining the barcode enhancement model is, for example, when the performance of the discriminator model and the generator model reaches a balance (e.g., the accuracy of the discriminator model is close to 90%), the training ends; or the number of training epochs reaches a preset threshold (e.g., 10 times, 100 times, etc.); or the training ends when all data in the training set has been completed once (1 epoch) or multiple times (e.g., 3 epochs, 10 epochs, etc.).

[0053] As an example, the training framework used during model training is PyTorch, and the generator learning rate is l. D1 =1×10 -4 The batch size is 64, the optimizer uses Adam, and the discriminator learning rate is l. D2 =1×10 -4 The batch size is 64, the optimizer uses SGD, and an alternating training strategy is adopted. First, the discriminator is trained, and then the generator is trained to continuously improve the capabilities of both, so that the generated DM barcode image is close to the ECC200 standard binarized barcode.

[0054] In some embodiments, the method for obtaining the sample matrix barcode image includes:

[0055] The original barcode images of thermal, inkjet, and laser engraving printing methods are acquired in an industrial environment using an industrial camera. Data augmentation strategies are then applied to these images, which are used as sample matrix barcode images. Each sample matrix barcode image contains regular barcode images and barcode images with anomalous features. The anomalous features of the barcode images include one or more combinations of the following: shape distortion, foreign object obstruction, dirt, uneven brightness, scratches, and blurring.

[0056] Therefore, during the model training process, barcode images covering common printing methods and abnormal features were used, starting from specific application scenarios. This enabled the trained barcode enhancement model to have strong generalization ability. Thus, regardless of whether the barcode images were taken on different production lines or under different lighting, temperature, humidity and other environmental conditions, high-quality enhanced barcode images could be output stably, ensuring the reliability of barcode processing.

[0057] In practical applications, the data augmentation strategies described above can be one or a combination of online and offline data augmentation strategies. When an online data augmentation strategy is chosen, the data loader dynamically generates augmented samples during each training iteration, instead of using pre-processed data, reducing storage overhead. When an offline data augmentation strategy is chosen, training efficiency can be improved. When both are combined, offline augmentation can generate a large number of fixed augmented samples, expanding the size of the dataset; online augmentation can introduce randomness during training, further enhancing data diversity.

[0058] As an example, offline enhancement is first performed on the original barcode image, including adding Gaussian noise, simulating blur effects, and generating images under different lighting conditions. The enhanced images are then saved to disk as an expanded training set. During training, the offline-enhanced images are further subjected to operations such as random occlusion and random brightness adjustments. These operations are generated in real time during each training session, reducing storage requirements.

[0059] As an example, six sets of images (a, b, c, d, e, f) are provided. Set a consists of images generated by adding noise to the original barcode image; set b consists of images generated by dilating the original barcode image; set c consists of images generated by eroding the original barcode image; set d consists of images generated by blurring the original barcode image; set e consists of images generated by overlaying occlusions on the original barcode image; and set f consists of images generated by randomly adjusting the brightness of the original barcode image. Using these images as the original barcode images, this application can enhance the original barcode images of the above types by adding noise, dilating, eroding, blurring, overlaying occlusions, and randomly adjusting brightness, and obtain the corresponding enhanced images.

[0060] In some embodiments, the data augmentation strategy includes applying at least one interference operation to the sample matrix barcode image, the interference operation including noise injection, morphological defect simulation, blurring, occlusion overlay, and illumination adjustment;

[0061] When the interference operation is noise injection, Gaussian noise and salt-and-pepper noise are added to the original barcode image to simulate imaging sensor noise and environmental interference. Salt-and-pepper noise refers to white or black pixels appearing at random locations in the image, while Gaussian noise refers to random noise that follows a Gaussian distribution.

[0062] When the interference operation is used to simulate morphological defects, it simulates barcode redundancy or missing defects in barcode printing through expansion or erosion operations.

[0063] When simulating blurred scenes, interference operations are performed by using a blur kernel to simulate the blurring caused by device jitter or rapid movement during barcode scanning; for example, a Gaussian blur kernel can be used to simulate image blurring caused by camera focusing errors or optical problems, or a mean blur kernel can be used to blur local areas in an image.

[0064] When the interference operation is to simulate occlusion and dirt, objects of different shapes, sizes and transparency are randomly superimposed on the barcode image to simulate the loss of some information in the barcode due to stains, scratches or external objects.

[0065] When the interference operation is a simulation of lighting conditions, the changes in barcode brightness are simulated by adjusting the overall brightness value of the barcode image, the contrast is adjusted by histogram stretching or local contrast enhancement, and the brightness distribution of the image is adjusted by gamma correction through nonlinear transformation.

[0066] To address noise injection, Gaussian noise and salt-and-pepper noise are added to the original barcode image to simulate imaging sensor noise and environmental interference. Gaussian noise increases random perturbation in the image by introducing a normally distributed random bias into each pixel value; salt-and-pepper noise simulates severe errors caused by sensor malfunction or external factors by randomly selecting a subset of pixels in the barcode image and setting their values ​​to maximum or minimum. During noise generation, limiting the amplitude and distribution ratio of the noise improves the model's ability to recognize unclear or interfered barcode images.

[0067] To simulate morphological defects, dilation or erosion operations are used to simulate quality defects in barcode printing caused by ink diffusion, insufficient ink volume, or varying degrees of laser engraving, resulting in redundancy or missing bars. Dilation can be considered to thicken barcode lines, simulating overprinting; erosion, on the other hand, thins barcode lines, simulating underprinting. In practical applications, by selecting the shape and size of structural elements and controlling the number of dilation or erosion operations, changes in barcode morphology can be effectively simulated. Furthermore, combining opening (erosion followed by dilation) and closing (dilation followed by erosion) operations can remove redundant noise and fill small holes in the barcode, thereby enhancing the model's robustness to non-standard barcodes.

[0068] For simulating blurred scenes, a blur kernel is used to simulate the motion blur effect of barcodes caused by device jitter or rapid movement during scanning. By controlling the direction and intensity of the blur, the blur effect under different motion states in real-world scenarios can be reproduced. For example, Gaussian blur is used to simulate image blur caused by camera focusing errors or optical problems, which can reduce the edge sharpness of the barcode and improve the model's ability to enhance low-quality barcodes. Mean blur is used to process local areas in the image, further enhancing the model's robustness and enabling it to better adapt to blurred barcode images in actual production, thus improving its enhancement capabilities. The blur kernel is a two-dimensional matrix used for convolution operations, achieving the blur effect through a weighted summation with the neighborhood of image pixels.

[0069] To simulate occlusion and dirt, objects of different shapes, sizes, and transparency are randomly superimposed on the barcode image to simulate the loss of partial barcode information due to stains, scratches, or external objects. The shape and position of the occluders are randomly generated, while constraints are imposed based on the structural characteristics of the barcode to avoid excessive occlusion of critical information areas. Adjusting the transparency allows for varying degrees of occlusion, simulating the intensity of dirt under different environments. This improves the model's recovery capability in cases of missing or damaged barcodes, enabling it to maintain a high degree of morphological restoration even with information loss.

[0070] To simulate lighting conditions, the model's brightness variation was simulated by adjusting the overall brightness value of the barcode image, enhancing its enhancement capabilities under different lighting intensities. Contrast adjustment was performed using histogram stretching or local contrast enhancement (CLAHE) to improve barcode recognition performance in low-contrast environments. Gamma correction using nonlinear transformation adjusted the image's brightness distribution, enhancing detail in dark or bright areas and improving barcode enhancement performance under extreme lighting conditions.

[0071] Therefore, by simulating various adverse conditions that may occur in an industrial environment, the model's adaptability and robustness to barcode images of different qualities and environments are enhanced, improving the model's ability to process various complex barcode images in practical applications. The diverse training samples generated by data augmentation strategies help the model better learn the essential features of barcode images, rather than being limited to specific imaging conditions or defect types. This results in a more generalized model that can be applied to barcode image augmentation tasks in different industrial scenarios. In real industrial environments, collecting high-quality barcode image data covering various defects and interference conditions often requires significant time and resources. Data augmentation strategies can generate a large amount of simulated data with different characteristics on existing basic datasets, effectively expanding the dataset size, reducing reliance on large-scale real data collection, thereby lowering data collection costs, improving R&D efficiency, and accelerating the model training and optimization process.

[0072] In some embodiments, the method for obtaining the labeled data includes:

[0073] In response to annotation instructions for the three vertices of the barcode region in the sample matrix barcode image, the position and orientation of the barcode are determined; a binary barcode conforming to the ECC200 standard is generated within the barcode region based on the decoded plaintext information or interactive input information and used as annotation data.

[0074] In practical applications, users use annotation tools or software to sequentially mark the positions of three adjacent vertices within a barcode region of a sample matrix barcode image. The coordinates of these three vertices uniquely determine the barcode's position and orientation within the image, providing precise geometric parameters for subsequent barcode region localization and processing. Based on the marked vertex coordinates, geometric calculation methods, such as the minimum bounding rectangle algorithm for triangles, are used to calculate the precise boundary and orientation information of the barcode region, separating the barcode from the complex background of the image.

[0075] The acquired sample matrix barcode images can be decoded using decoding algorithms to obtain the stored plaintext information. Alternatively, users can manually input plaintext information related to the barcode through interactive input.

[0076] Within the defined barcode area, based on the acquired plaintext information, and in accordance with the ECC200 standard's encoding rules and image generation specifications, a corresponding binarized barcode image is generated as annotation data.

[0077] Therefore, by generating binary barcodes conforming to the ECC200 standard based on the decoded plaintext information or user interactive input information, the standard consistency between the labeled data and the barcodes in actual industrial applications is ensured. The above standard-based labeling method enables the model to learn accurate barcode features and specifications during training, which helps to reduce the risk of model performance degradation caused by non-standard labeled data.

[0078] The method for obtaining the labeled data may also include: inputting the sample matrix barcode image into the region positioning model to obtain positioning data for determining the barcode region; and extracting the binary barcode conforming to the ECC200 standard from the barcode region using a barcode decoding algorithm and using it as labeled data.

[0079] The sample matrix barcode image is input into a region localization model, which, for example, uses a convolutional neural network (CNN), to detect the position of the barcode in the image and generate barcode region localization data. This barcode region localization data can be considered to include the barcode's position and size information. Based on this localization data, barcode region images are extracted from the sample matrix barcode image. Then, a barcode decoding algorithm is applied to decode the extracted barcode region images, generating a binary barcode image conforming to the ECC200 standard. This binary barcode image conforming to the ECC200 standard is output as annotation data. The barcode decoding algorithm, for example, utilizes the libdmtx barcode decoding library.

[0080] As an example, the training process of the area localization model includes:

[0081] A region localization training set is obtained, comprising multiple training data sets, each of which includes a sample matrix barcode image and labeled localization data of the sample matrix barcode image. For each training data set in the region localization training set, the following processing is performed: the sample matrix barcode image in the training data is input into a preset deep learning model to obtain predicted data including localization data; the model parameters of the deep learning model are updated based on the predicted data and the labeled data; a preset training termination condition is checked; if yes, the trained deep learning model is used as the region localization model; if no, the deep learning model is trained again using the next training data set.

[0082] In some embodiments, the generator model adopts a UNet architecture, and the generator model includes:

[0083] The encoder, consisting of multiple convolutional and max-pooling layers, is used to progressively refine and abstract the features of the input image by stacking them layer by layer.

[0084] The decoder is used to restore the original resolution of features using deconvolution and upsampling techniques;

[0085] Skip connections are used to build direct paths between the encoder and decoder, allowing low-level details to be incorporated into high-level features.

[0086] The final output layer is used to generate a normalized grayscale barcode image through the sigmoid activation function.

[0087] The encoder, the first part of the generator model, contains multiple convolutional layers and max-pooling layers. The convolutional layers perform feature extraction and transformation on the input image, extracting local features such as edges and textures by sliding convolution kernels across the image. Max-pooling layers downsample the feature maps output by the convolutional layers, reducing their size, computational complexity, and enhancing the model's invariance to image translation and other changes. In the encoder, through layer-by-layer stacking, the feature map size gradually decreases while the number of feature channels gradually increases with network depth. This allows for the progressive refinement and abstraction of input image features, extracting high-level complex semantic features from low-level, simple features.

[0088] The decoder is the second part of the generator model, used to progressively restore the high-level features extracted by the encoder back to the original resolution. It employs deconvolution (i.e., transposed convolution) and upsampling techniques. Deconvolution layers enlarge the feature map size, while upsampling further improves the resolution, restoring it to a size similar to or the same as the input image. In the decoder, as the network depth decreases, the feature map size gradually increases, while the number of feature channels gradually decreases, ultimately outputting a feature map of the same size as the input image.

[0089] Skip connections establish a direct path between the encoder and decoder. Specifically, a feature map at a certain level in the encoder is directly connected to a feature map at the corresponding level in the decoder, allowing low-level details to be integrated into high-level features. In this way, during the decoder's feature map reconstruction process, not only can high-level semantic information be utilized, but low-level details such as image edges and textures can also be preserved, thus contributing to the generation of images that are geometrically aligned with the standard format of their barcode protocol.

[0090] The final output layer, located at the end of the decoder, performs final processing on the feature map output by the decoder to generate a qualified output image. The final output layer uses the sigmoid activation function to map each pixel value of the feature map to a range of 0 to 1, generating a normalized grayscale barcode image. The output of the sigmoid function represents the probability that a pixel belongs to a barcode module. By setting a threshold (e.g., 0.5), the grayscale image can be converted into a binary barcode image, where pixel values ​​greater than the threshold represent barcode modules, and values ​​less than the threshold represent the background.

[0091] Therefore, compared with traditional models that rely solely on high-level semantic features for generation, the technical solution in this embodiment, due to the skip connections in the UNet structure, allows low-level detailed features to be directly passed to the decoder. This results in better preservation of barcode edges, lines, and other detailed information when generating the final barcode image. In other words, the UNet structure can generate clearer and more accurate barcode images, reducing decoding errors caused by loss of detail. Simultaneously, it can more accurately extract barcode regions and regularize the geometry of the extracted results, aligning the resulting image's geometry with the standard format of the barcode protocol, greatly improving the decoding success rate.

[0092] In some embodiments, the discriminator model adopts a ResNet18 architecture, and the discriminator model includes:

[0093] Multiple residual modules are used to perform feature extraction operations through convolutional kernels and to perform data normalization processing through batch normalization technology;

[0094] The global average pooling layer is used to perform global average pooling on the extracted features by skipping the residual module.

[0095] The output layer connects to the output of the global average pooling layer and outputs a predicted value representing the probability that the input image is a real barcode image through an activation function.

[0096] The discriminator model comprises multiple residual modules. Each residual module performs feature extraction operations using convolutional kernels and undergoes data normalization through batch normalization. Convolutional operations are used to extract local features from the input image, such as edges and textures, while batch normalization standardizes the output of the convolutional layers, giving the data zero mean and unit variance, thus accelerating the model's convergence and improving its stability.

[0097] Following the residual module, the features are connected via skip connections, and a global average pooling operation is performed on the extracted features. The global average pooling layer globally averages the feature maps of each feature channel, compressing the two-dimensional feature map of each feature channel into a single value that represents the average response of that feature channel across the entire image.

[0098] The output layer is connected to the output of the global average pooling layer. It outputs a predicted value through an activation function (sigmoid activation function). This predicted value represents the probability that the input image is a real barcode image. The sigmoid function restricts the output value to between 0 and 1, which can be interpreted as the probability that the input image is a real barcode image. For example, an output value close to 1 indicates that the model considers the input image to be a real barcode image with a high probability, while a value close to 0 indicates that the discriminator model considers the input image (from the generator model) to be a generated barcode image with a high probability.

[0099] This can be understood as a predicted value representing the degree to which the image geometry aligns with the barcode protocol specification. The sigmoid function interprets the output value, which is limited to between 0 and 1, as the degree to which the image geometry aligns with the barcode protocol specification. For example, an output value close to 1 indicates that the model believes the input image geometry aligns highly with the barcode protocol specification, while a value close to 0 indicates that the discriminator model (or generator model) believes the input image geometry aligns poorly with the barcode protocol specification.

[0100] Therefore, by using multiple residual modules in the ResNet18 structure, high-level semantic features of the input image can be extracted layer by layer, gradually abstracting from low-level features such as edges and textures to high-level barcode structure and pattern features. This layer-by-layer feature extraction and abstraction capability enables the discriminator model to more accurately capture the differences between real barcode images and images conforming to barcode protocol specifications, thereby improving its ability to distinguish the authenticity of input images and providing more effective feedback to the generator model. The application of global average pooling layers not only reduces the number of model parameters and lowers model complexity but also enhances the model's adaptability to different input images.

[0101] In summary, this application constructs an adversarial generative network based on UNet and ResNet18, which not only transforms blurry RGB barcodes into clear RGB barcodes, but also automatically and accurately extracts barcode regions and completes the geometric structure under scenarios including background interference, deformation, occlusion, and blurring.

[0102] Even when models are used in related technologies, they are only used to recover the image super-resolution model, making blurry barcodes clearer. After super-resolution, the barcode still needs to go through processes such as barcode extraction (binarization) to be read. In contrast, the technical solution of this application involves a robust reconstruction of the barcode's logical structure under the condition that the image is blurred or contaminated. The aim is to extract the barcode region stably and accurately and align its geometric structure with the barcode protocol to reduce the difficulty of reading the barcode, rather than the traditional image sharpening or super-resolution processing.

[0103] Therefore, this application utilizes a trained generator to automatically extract various types of barcodes and performs shape regularization and defect filling on the extracted barcodes. Combined with refined post-processing, it effectively solves the problem of low recognition rate caused by poor quality industrial barcodes. Experiments demonstrate that this method significantly outperforms related technologies in terms of quantitative indicators, decoding performance, and real-time performance, providing a reliable barcode processing solution for industrial automation.

[0104] As an example, the discriminator's multiple residual modules perform feature extraction using 3×3 convolutional kernels, followed by data normalization using Batch Normalization. Skip connections address the vanishing or exploding gradient problem, thereby enhancing the stability of the model during training. A global average pooling layer is set, effectively reducing model parameters while enhancing the model's generalization ability. The output sigmoid prediction value is used to determine the degree to which the geometric structure of the input image aligns with the barcode protocol specifications.

[0105] In some embodiments, the generator loss function consists of an adversarial loss function, an image inpainting loss function, a feature loss function, and a structural similarity loss function, and is weighted and summed using preset weights.

[0106] Among them, the weight parameters were determined experimentally. The adversarial loss function corresponds to the adversarial loss weight, which is used to maintain the dynamic balance between the generator and the discriminator. The pixel-level restoration loss weight corresponding to the image restoration loss function has the greatest impact on the generation quality and can be given the highest weight to improve the generation quality. The feature loss weight corresponding to the feature loss function is used to ensure semantic consistency. The structural loss weight corresponding to the structural similarity loss function is used to affect the boundary quality and can be given a relatively high weight.

[0107] By comprehensively considering adversarial loss, image inpainting loss, feature loss, and structural similarity loss, different loss functions constrain and optimize the generator from different perspectives, enabling the model to learn more comprehensive and richer image feature representations. Adversarial loss helps the model learn data distributions, improving its adaptability to different barcode image styles and types; image inpainting loss ensures the model's accuracy in pixel-level restoration, adapting to barcode images under various imaging conditions; feature loss enables the model to understand the essential features of barcode images at the semantic level, handling input images of varying complexity and quality; structural similarity loss strengthens the model's attention to barcode structural details, ensuring the standardization and integrity of the barcode during generation. This comprehensive optimization approach enhances the model's generalization ability, enabling it to consistently output high-quality enhancement results when faced with different barcode images in various industrial environments. The weighted summation method with preset weights provides flexibility for model adjustment and optimization, allowing the weights of different loss functions to be adjusted according to actual application needs and data characteristics to highlight or balance performance in certain specific aspects. For example, in applications where barcode boundary clarity is extremely important, the weight of structural similarity loss can be appropriately increased; conversely, when the overall realism of the generated image is paramount, the weight of adversarial loss can be increased. This flexibility allows the model to better adapt to different practical needs and industrial environments. Generator model performance can be optimized and customized through simple weight adjustments, without requiring large-scale modifications and retraining of the model structure.

[0108] The technical solution provided in this embodiment achieves automatic barcode region extraction with strong background adaptability. Even under complex backgrounds or interference conditions, it can accurately locate the barcode region with stable and reliable extraction results. Compared to traditional algorithms that rely on manual image processing parameter settings, this method eliminates cumbersome manual intervention, overcoming the problems of poor parameter generalization and inability to adapt to diverse scenarios found in conventional algorithms. It is more suitable for complex environments in practical industrial applications. Furthermore, addressing the issue of significant differences in shape between the original barcode and the standard protocol due to factors such as angle distortion, occlusion, or printing defects during acquisition, which affects decoding accuracy, this method can reshape and complete the extracted barcode region, aligning its geometric structure with the standard format of the corresponding barcode protocol, thereby significantly improving the reading success rate. Compared to traditional algorithms that mainly rely on image fitting or rule adjustment to adapt to such deformed scenarios, this method exhibits stronger versatility and recognition stability under different application conditions, with wider adaptability and higher robustness. On the other hand, an end-to-end learning architecture is adopted, directly inputting industrial barcode images into the model. After one forward propagation calculation, a binarized barcode image with a geometric structure aligned to the standard format of the corresponding barcode protocol can be output. Adversarial loss enhances the overall plausibility of the generator's generated images, ensuring that it can still reproduce the barcode geometry that meets protocol requirements under different backgrounds or occlusion conditions, rather than merely improving the image's "realism." Image inpainting loss ensures that the model outputs geometric and structural specifications that conform to the target barcode protocol at the pixel level. Feature loss, by comparing the feature representations of the discriminator's intermediate layers, ensures that the barcode regions extracted from the generated images have the same semantic structure and logical arrangement as standard barcodes. Structural similarity loss focuses on the clarity and symmetry of key structures such as internal modules, borders, and positioning patterns of the barcode, making the model output more regular in local structure, facilitating subsequent protocol-level recognition and parsing. This comprehensive optimization method enhances the model's generalization ability, enabling it to stably extract barcode regions automatically and regularize the geometry of the extracted results when faced with different barcode images in various industrial environments, ensuring that the resulting image's geometric structure aligns with the standard format of its barcode protocol. The pre-defined weighted summation method provides flexibility for model adjustment and optimization. The weights of different loss functions can be adjusted based on actual application needs and data characteristics to highlight or balance performance in specific aspects. For example, in applications requiring extremely high clarity of barcode boundaries and structural accuracy, the weight of structural similarity loss can be appropriately increased.

[0109] As an example, adversarial loss function By leveraging the adversarial training mechanism of GANs, the realism of the generated images can be improved, making them as close as possible to real barcode images, as defined below:

[0110]

[0111] x is a random variable that follows the distribution of the true data. D(G(x)) is the output probability of the discriminator for the generated image G(z), representing the probability that the discriminator considers the generated image to be a real barcode (range [0,1]), and logD(G(x)) is the probability that the generator wants to maximize the discriminator's judgment probability for the generated image;

[0112] Image restoration loss Calculate the L1 distance between the generated image and the target image to ensure pixel-level reconstruction accuracy, defined as follows:

[0113]

[0114] It is a binary barcode image that conforms to the ECC200 standard (which may be the corresponding barcode protocol).

[0115] Feature loss function The semantic consistency of the generated image is improved by calculating the differences in features of the intermediate layer of the discriminator, as defined below:

[0116]

[0117] D (i) For the intermediate feature extraction part of the i-th layer of the discriminator, G(x) gt ) represents the generator G on the image x gt The processing result, G(x) gt ) is the generator G that will generate x gt Output as input.

[0118] Structural similarity loss function By optimizing the boundary clarity of the barcode module to make it easier to decode, the following definition is made:

[0119]

[0120] SSIM stands for Structural Similarity Index.

[0121] The final weighted loss of the generator loss function The definition is as follows:

[0122]

[0123] λ adv , λ rec , λ feat and λ ssimEach parameter has its own weight. The weight parameters were determined experimentally. The adversarial loss weight was set to 1.0 to maintain the dynamic balance between the generator and the discriminator; the pixel-level repair loss weight was set to 10.0 because it has the greatest impact on the generation quality, so it was given the highest weight to improve the generation quality (the degree of alignment of the image geometric structure with the barcode protocol standard); the feature loss weight was set to a moderate 1.0 to ensure semantic consistency; and the structural loss weight was set to 5.0 because it affects the boundary quality, so it was given a relatively high weight.

[0124] As an example, the discriminator loss function uses the binary cross-entropy loss function to calculate the loss of the ECC200 standard binarized barcode label image and the generated standard barcode image. Label smoothing technology is applied to smooth the label y in the real image. real The value was adjusted from 1.0 to 0.9, and the generated image label y was... fake The value was adjusted from 0 to 0.1 to reduce the discriminator's overconfidence and mitigate pattern collapse. The total loss function of the discriminator was also adjusted. The definition is as follows:

[0125]

[0126] y represents the adversarial loss value of the discriminator, indicating the classification error of the discriminator between the real and generated images. real For the labels of real images, y fake The labels for the generated images are denoted as . D(x) is the output of the discriminator for the real image x, representing the probability that the discriminator considers x to be a real image. D(G(z)) is the output of the discriminator for the generated image G(z), representing the probability that the discriminator considers G(z) to be a real image.

[0127] In some embodiments, the method further includes: post-processing the standard binarized barcode to obtain a target binarized barcode; the post-processing includes the following steps:

[0128] For the standard binary barcode, an adaptive threshold binarization method is used. This method calculates thresholds that adapt to different brightness levels in different areas based on a local window, separating the barcode module from the background. The corresponding formula is as follows:

[0129]

[0130] Among them, G bin (x,y) represents the binarized pixel value, G(x,y) represents the pixel value of the generated barcode image, and T(x,y) represents the dynamic threshold, calculated using an adaptive threshold binarization method, used to distinguish the barcode module from the background. The threshold is calculated based on the pixel value distribution within a local window to adapt to brightness variations in different areas of the image.

[0131] As an example, the adaptive thresholding binarization method determines the threshold T for each pixel by defining a local window around it and calculating the statistical properties (such as mean, median, etc.) of the pixel values ​​within that window. This method automatically adapts to brightness variations in different regions of the image, ensuring accurate separation of the barcode module from the background even under uneven lighting conditions. During calculation, for each pixel (x, y) in the image, its grayscale value G(x, y) is compared with the dynamic threshold T(x, y).

[0132] Morphological opening operations are employed to eliminate isolated noise points and connect fracture boundaries to obtain the target binarized barcode. For example, a 3×3 kernel is used for morphological opening operations (erosion followed by dilation) to eliminate noise generated during the generation process and connect fracture boundaries to optimize barcode quality and improve readability.

[0133] Therefore, by using adaptive threshold binarization and morphological opening post-processing steps, the quality of the standard binarized barcode image output by the generator is further improved, making it more compliant with the ECC200 standard and further reducing the decoding failure rate.

[0134] To verify the validity of this application, the technical solution proposed in this application (steps S101 to S103) is compared with traditional image processing methods using the same dataset. Specifically, performance evaluation is conducted on the same dataset and in typical industrial scenarios, comprehensively demonstrating the practicality and superiority of this technology in handling complex industrial scenarios from multiple dimensions such as barcode region extraction, structural regularization, and defect completion.

[0135] Traditional image processing methods rely on fixed image processing rules and expert experience. As a comparative example, the steps of traditional image processing methods are as follows:

[0136] Image preprocessing involves converting color images to grayscale to reduce data volume and improve processing speed; and converting grayscale images to black and white to make it easier to identify barcode bars and spaces. Filtering methods are used to remove noise from the image, improving image quality.

[0137] Barcode region detection involves extracting image structure using edge detection operators (such as Sobel and Canny) and detecting edges in the image to determine the barcode boundaries. Morphological operations (such as dilation and erosion) are used to enhance the features of the barcode regions. The image is segmented into multiple regions, and the regions containing the barcode are identified.

[0138] Decoding involves measuring the width of the bars and spaces in the barcode and determining the barcode encoding based on this width information. The detected barcode pattern is matched against a predefined barcode template to determine the barcode type and encoding. Key barcode features are extracted, such as the order of bars and spaces, start and stop characters, etc. The identified barcode pattern is converted into the corresponding numeric or character encoding. As an example, for detected candidate areas, the open-source barcode decoding library libdmtx is used to decode the ECC200 standard DataMatrix barcode. libdmtx can automatically identify the barcode positioning pattern, perform error correction and content restoration, and finally output the corresponding encoded information.

[0139] However, the aforementioned image processing methods require manual design of appropriate morphological operations and parameters based on the specific characteristics of the barcode image and the application scenario, demanding a high level of professional knowledge and experience. In other words, while this process is effective for standardized, high-contrast images, it relies on preset rules and fixed morphological assumptions, requiring a high level of expertise from the operator. In complex industrial environments, such as those with metallic background interference, damaged barcodes, unclear printing, or strong light reflection, the stability and adaptability of this method significantly decrease, making it difficult to guarantee robustness and accuracy. For the complex barcode image defects or diverse application scenarios mentioned in this application, compared to the technical solutions mentioned in this application, there are limitations in handling complex scenarios and adaptability.

[0140] This application can process DM barcode images with abnormalities (low quality) in various real-world industrial scenarios. For example, it can process DM barcode images with scratches or irregular etching (such as several irregularly shaped and small areas in the clock pattern area), barcodes with irregular printing or overexposed light sources, barcodes with scratches and barcode areas similar to background features, dot-type barcodes with scratches and barcode areas similar to background features, barcodes with broken alignment patterns (such as the lower left corner area), dirty barcodes, barcodes with significant differences in shape from ECC200 standard barcodes, barcodes with unclear silkscreen printing and a certain degree of blurriness, and barcodes with added noise.

[0141] The method described in this application is based on a generator-based approach for automatic barcode extraction, shape regularization, and defect completion. This application utilizes a Generative Adversarial Network (GAN) as its core. Through a pre-trained generator model, it automatically performs the three-stage tasks of "barcode recognition region extraction—geometric regularization—structural completion" without the need for manual design of morphological rules. The overall process is as follows:

[0142] Barcode structure modeling and enhancement: The original low-quality DM barcode image is input and processed by a pre-trained generator network. Through deep feature learning, the generator significantly improves the structural consistency and decodeability of the barcode image, adapting to complex industrial image interference scenarios without requiring pre-defined morphological rules. Barcode decoding and recognition: The enhanced image is input to a unified decoder (also using libdmtx) for decoding to ensure the consistency and fairness of the comparative experiments, further verifying the actual promoting effect of the image enhancement process on the final recognition performance.

[0143] See Figure 5 This demonstrates the enhancement effect and adaptability of this application in typical industrial scenarios. Each set of images in the figure includes the original abnormal barcode image on the left and the image enhanced by the method of this application on the right, totaling nine typical interference types. In specific applications (see...),... Figure 5 The categories include: Group a, with scratches and irregular etching (e.g., abnormal shape or reduced area of ​​clock pattern blocks); Group b, images with poor printing quality and affected by overexposure; Group c, barcode areas with similar background features and superimposed scratch interference; Group d, dotted codes blending with metallic textured backgrounds, with unclear boundaries; Group e, alignment patterns with breaks and missing parts; Group f, dirty barcodes with severe surface contamination; Group g, abnormal barcodes with structures significantly deviating from the ECC200 standard; Group h, blurry silkscreen printing and overall low clarity issues; and Group i, strong noise interference affecting the integrity of the barcode structure.

[0144] As shown in the figure, the method of this application can stably extract barcode regions under various abnormal industrial conditions, significantly improve structural coherence, restore the shape of missing modules, enhance boundary clarity, and effectively improve barcode recognizability. Compared with traditional image processing methods, it has significant advantages in robustness, adaptability, and automation, and is particularly suitable for challenging industrial barcode recognition scenarios such as intelligent manufacturing, electronic assembly (e.g., SMT), and heavy industry inspection.

[0145] Comparative experiments were conducted on a self-built real industrial barcode dataset (containing 5000 images of abnormal DM barcode images from various real industrial scenarios, with 70% as the training set, 15% as the validation set, and 15% as the test set). The results are shown in Table 1.

[0146] Table 1: Comparison of Results

[0147] method SSIM Decoding success rate (%) Technical solutions of traditional image processing methods 0.653 89.1 The technical solution proposed in this application 0.889 99.8

[0148] The experimental results are analyzed as follows: In terms of quantitative indicators, this application significantly outperforms the comparative method (traditional image processing method) in SSIM (0.889), demonstrating its advantages in pixel-level reconstruction and structure preservation. Regarding decoding performance, the method provided in this application achieves a decoding success rate of 99.8%, an improvement of 10.7 percentage points compared to the comparative method, indicating that the generated image better conforms to the ECC200 standard specifications.

[0149] Meanwhile, the experiment evaluated the results from two dimensions: quantitative image quality metrics and decoding performance metrics. The analysis results are as follows:

[0150] In terms of quantitative metrics, this application achieved a high score of 0.889 on the Structural Similarity Index (SSIM), significantly outperforming traditional image processing methods. As an important indicator of image structural fidelity, this result demonstrates that our method has significant advantages in maintaining barcode structural coherence and texture consistency, enabling more realistic reconstruction of damaged areas while preserving the geometric features of the original image.

[0151] In terms of decoding performance: Under consistent evaluation conditions using the libdmtx decoder, the decoding success rate of the method in this application reaches 99.8%, which is 10.7 percentage points higher than the traditional comparison method. This result fully demonstrates that the image generated by this method better conforms to the module arrangement specifications and structural integrity requirements of the ECC200 standard, and can effectively improve the readability and reliability of barcodes in industrial applications.

[0152] In summary, the advantages of the method provided in this embodiment are specifically reflected as follows:

[0153] 1. The barcode extraction is more accurate. The generative adversarial network used in the technical solution has a powerful target structure learning capability, which can accurately extract the barcode identification area under complex background interference, low contrast, or occlusion conditions. Compared with traditional methods that rely on edge detection and morphological rules, this method gets rid of the dependence on the preset structural features of the image and achieves adaptive separation of the background and barcode area, thereby significantly reducing false detections and false negatives.

[0154] 2. Enhanced structural regularity: The decodeability of barcodes highly depends on the geometric regularity between modules. Traditional methods struggle to restore regularity when dealing with structural offsets caused by printing distortion or angular aberrations. This application utilizes an internal geometric modeling mechanism within the generator to uniformly regularize key geometric forms in the barcode, such as clock patterns, alignment patterns, and module grid structures, enhancing structural symmetry and grid alignment effects, and significantly improving the decoder's fault tolerance.

[0155] 3. The missing data completion method is more context-consistent. For barcode images with missing areas such as scratches, occlusions, or contamination, this method can intelligently reason and complete the missing structure based on global and local contextual information of the image. Compared to the irreversible image damage in traditional methods, this application can utilize the barcode semantics and structural priors learned by deep neural networks to achieve reasonable judgment and repair of the module shape, quantity, and distribution, ensuring that the generated result has structural integrity and conforms to the coding standard.

[0156] Therefore, this application not only outperforms traditional image processing methods in quantitative metrics and decoding performance, but also demonstrates significant technical advantages of deep learning methods in key tasks such as barcode region extraction, structural regularization, and defect completion. This method is highly adaptable and robust, making it particularly suitable for the complex and ever-changing visual environments in industrial production, and possesses broad engineering application prospects.

[0157] In practical applications, the system deployed on the enterprise side using the above methods is configured with an Intel i7 8700k processor and an NVIDIA 1660ti graphics card. The actual production line deployment and testing process is as follows:

[0158] Real-time data acquisition steps: Barcode images are captured at 9fps using a 12-megapixel camera and transmitted to the processing terminal via gigabit Ethernet.

[0159] Deployment and testing steps: Input the collected barcode data into the trained generator model. Batch processing is supported, with an average processing time of ≤10ms per barcode.

[0160] Results feedback process. The decoding engine (OpencvEvision / ZBar) decodes the enhanced image. If decoding fails, an alarm is triggered and the original data is saved for model iteration and optimization.

[0161] After running the above process continuously for 30 days, the results before and after the import are compared as follows:

[0162] The average decoding rate increased from 90.1% before deployment to 98.9% (30-day comparison data); the system latency end-to-end processing latency is ≤20ms, meeting the real-time requirements of the production line.

[0163] This application constructs an adversarial generative network based on U-Net and ResNet18, not to convert fuzzy RGB barcodes into clear RGB barcodes, but to enable barcode extraction under the same conditions (e.g., fuzzy RGB barcodes).

[0164] Even when models are used in related technologies, they are only used to recover the super-resolution model of the image, making the blurry barcode clearer. After super-resolution, the barcode still needs to go through processes such as barcode extraction (binarization) to be read. In contrast, the technical solution of this application involves a robust reconstruction of the barcode's logical structure under the condition that the image is blurred or contaminated. The aim is to output a standard binary barcode with a structure closer to the coding specification (e.g., ECC200) to reduce the difficulty of reading the barcode, rather than image sharpening or super-resolution processing in the traditional sense.

[0165] Therefore, this application utilizes a trained generator to automatically extract various types of barcodes and performs shape regularization and defect filling on the extracted barcodes. Combined with refined post-processing, it effectively solves the problem of low recognition rate caused by poor quality industrial DM barcodes. Experiments demonstrate that this method significantly outperforms related technologies in terms of quantitative indicators, decoding performance, and real-time performance, providing a reliable barcode processing solution for industrial automation.

[0166] Second method embodiment.

[0167] This application provides a barcode enhancement image processing method based on generative adversarial networks. It automatically extracts the barcode region from a real barcode and regularizes the geometry of the extracted result, aligning the resulting image's geometric structure with the standard format of the barcode protocol to enhance decoding success rate. Its specific implementation and the achieved technical effects are consistent with those described in the first method embodiment above, and some details will not be repeated.

[0168] The barcode enhancement image processing method is used to perform barcode identification region extraction, geometric regularization transformation, and structural defect completion on barcode images collected in industrial environments. It includes the following steps:

[0169] P101, Construct a training set; the training set contains multiple matrix barcode image data, including data on deformation, occlusion, and printing defects, collected by industrial cameras in a real production line environment, and their corresponding annotation data; the annotation data is the corresponding matrix barcode image data, in which the barcode area position is manually annotated, and a standard geometric barcode image corresponding one-to-one with the spatial position of the area is generated according to the encoding protocol applicable to the barcode.

[0170] Specifically, each training sample in the training set consists of a "matrix barcode image collected in an industrial setting" and its corresponding "structured annotation data". The sample images comprehensively cover a variety of complex working conditions and manufacturing methods, including thermal printing, inkjet printing, laser etching, laser engraving and other processes, as well as barcode images with typical defects such as rotation, occlusion, deformation and missing parts.

[0171] Unlike traditional binary classification or segmentation pixel value annotation, the labeled data adopts a structure alignment strategy, annotating the outline of the barcode encoding area, module position and possible missing areas at the pixel level or module level, which significantly improves the training supervision accuracy and supports the model to learn more refined region localization and shape reconstruction capabilities.

[0172] P102 uses an adversarial network model constructed with a generator and a discriminator as the initial model. The generator is built based on the UNet structure and optimized by a weighted combination of adversarial loss, image reconstruction loss, structural similarity loss, and feature consistency loss. It is used to automatically extract barcode identification regions from two-dimensional barcode images in real industrial scenarios and to achieve geometric regularization of the barcode structure and structural completion of missing regions. The discriminator adopts the ResNet architecture and uses the discrimination ability of the binary cross-entropy loss optimizer to determine the structural similarity between the generator output image and the standard barcode image generated according to the barcode encoding protocol in terms of geometric shape.

[0173] Specifically, the initial model is constructed as a Generative Adversarial Network (GAN), comprising: a generator (model): based on a modified UNet encoder-decoder structure, featuring multi-scale receptive fields and feature fusion mechanisms, designed to automatically extract the identifier region from low-quality or defective input barcode images and map the extracted barcodes into standard binary images consistent with their protocol, with regular shapes and complete information, for subsequent direct decoding. A discriminator (model): employing a lightweight residual structure (such as ResNet18), unlike image discriminators in general deblurring tasks, this discriminator focuses on the correctness of the protocol structure rather than visual realism. By learning the geometric properties and structure of standard barcodes, it significantly improves the accuracy of judging the "decodeability" and "protocol legitimacy" of barcode image generation.

[0174] Unlike existing weakly supervised methods that rely on image sharpening or region cropping, the model constructed in this step has end-to-end processing capabilities, a more structure-aware annotation strategy, and improves model learning efficiency and result stability. It does not rely on subsequent tuning and post-processing modules, significantly improving industrial deployment efficiency.

[0175] The goal of the generator model is to extract the input barcode image containing background interference, deformation, occlusion, etc., and complete the geometric structure; the discriminator model is used to distinguish the barcode image generated by the generator model from the binarized barcode image of the corresponding barcode protocol standard, providing feedback to the generator model and prompting the generator model to continuously optimize the generation results.

[0176] P103, the initial model is trained based on the composite loss function to obtain an enhanced model with barcode extraction, geometric regularization and defect completion capabilities;

[0177] In this step, a multi-branch composite loss function is designed and combined with an adversarial training mechanism to guide the generator to gradually learn the joint optimization capabilities of barcode region extraction, module shape restoration, and intelligent completion of missing regions, forming a barcode enhancement model with high robustness and structure awareness. The composite loss function includes a generator loss function using weighted composite loss technology and a discriminator loss function using binary cross-entropy loss technology.

[0178] The generator loss function (weighted composite structure) includes:

[0179] Adversarial loss: By training the generator and discriminator in a game-like manner, the degree of fit of the output barcode image to its corresponding protocol structure is improved, and the geometric consistency is enhanced;

[0180] Image reconstruction loss: Constrains the consistency between the generated image and the labeled image at the pixel level to ensure the completeness of the incomplete image and the accuracy of the structural restoration;

[0181] Feature loss: Utilize pre-trained models to extract multi-scale semantic features, improve the perceptual consistency between generated and labeled images in the high-level feature space, and ensure semantic alignment of content;

[0182] Structural similarity loss: Focuses on matching local structure and contrast to ensure that the generated barcode area has a clear grid structure and module edges, thereby improving the quality of structural reproduction.

[0183] Loss weighting mechanism: Each loss item is weighted and combined according to empirical weights to achieve the joint optimality of image quality, structural sharpness, and protocol consistency.

[0184] The discriminator loss function uses the discriminative ability of the binary cross-entropy loss optimizer to determine the structural similarity between the generator output image and the standard barcode image generated according to the barcode encoding protocol (such as ECC200, ISO / IEC18004, etc.), thereby effectively guiding the generator to learn and generate barcode images with reasonable structure and easy recognition.

[0185] The generator loss function employs a weighted composite loss technique, comprehensively considering multiple aspects such as adversarial loss, image inpainting loss, feature loss, and structural similarity loss. It focuses on constraining the model's performance in aligning the extracted barcode region with the geometric shape of the barcode protocol. Adversarial loss enhances the overall plausibility of the generated image, ensuring it can reconstruct the barcode geometry that meets protocol requirements under different backgrounds or occlusion conditions, rather than merely improving the image's "realism." Image inpainting loss calculates the L1 distance between the generated and target images, ensuring the output conforms to the geometric and structural specifications of the target barcode protocol. Feature loss compares the feature representations of the discriminator's intermediate layers, ensuring the extracted barcode region in the generated image has a semantic structure and logical arrangement consistent with the standard barcode. Structural similarity loss focuses on the clarity and symmetry of key structures such as internal modules, borders, and positioning patterns of the barcode, making the model output more regular in local structure and easier for subsequent protocol-level recognition and parsing. The discriminator loss function uses a binary cross-entropy loss technique to measure the discriminator's classification accuracy for standard barcode images and generated barcode images, helping the discriminator model better learn the features that distinguish between the two.

[0186] During training, the training set constructed using P101 is input into the initial model. The generator extracts the barcode identification region, performs shape regularization, and completes missing parts based on the input image. The discriminator distinguishes the generator's output from standard barcode images generated according to barcode encoding protocols (such as ECC200, ISO / IEC18004, etc.). The two are jointly optimized, and the parameters are iteratively updated using optimization algorithms such as Adam, ultimately converging to obtain an enhanced model with barcode extraction, shape regularization, and completion capabilities.

[0187] Compared to traditional image super-resolution or deblurring methods, this method does not focus on improving the quality of the original image. Instead, it aims to accurately extract the barcode marking region and perform shape regularization and defect filling on the extracted barcode region to ensure that the output image conforms to the actual barcode protocol (such as ECC200). It does not rely on manual parameter tuning or post-processing (such as threshold selection, dilation and erosion), simplifying the deployment process and adapting to various industrial environments. It enhances robustness in defective scenarios, especially in the presence of severely occluded, deformed, or defective areas, where it can still recover identifiable images, whereas traditional methods have extremely low recognition rates in such scenarios.

[0188] P104 uses an industrial camera to capture data matrix barcode images in a real industrial environment, inputs them into a trained augmentation model, and outputs a standardized barcode image with one or more of the following features:

[0189] Filter background interference areas; perform geometric regularization processing on structural distortions in the barcode to restore each module to a square unit that conforms to the standard matrix arrangement; for barcode module missing due to one or more factors such as occlusion, dirt, scratches, or ink loss, perform structural consistency completion based on contextual semantic information; generate results that meet the structural rules of the corresponding barcode encoding protocol, and output a standard binarized image with clear boundaries and regular geometric structure.

[0190] In this step, an industrial camera is used to capture barcode images in a real working environment. The captured images may have low image quality, blurred module boundaries, and low contrast due to problems such as uneven lighting, stains, metal textures, occlusion, and distortion, which seriously affect the subsequent barcode recognition and decoding results.

[0191] To overcome the drawbacks of traditional methods that require extensive manual parameter tuning and tedious image preprocessing, this technical solution employs a trained (barcode) enhancement model to accurately extract the barcode identifier region from the aforementioned low-quality barcode images. The extracted identifier region is then structurally regularized and its defects are filled in. This model can adapt to various barcode types (such as DataMatrix, QR, etc.) and restore their module structure and spatial arrangement according to their respective encoding protocols, ultimately outputting a standard binarized image that conforms to the barcode's own encoding rules.

[0192] The generated enhanced images have the characteristics of high contrast, clear structure, and accurate defect repair. They can be directly input into general or special barcode decoders for recognition and parsing without the need for image preprocessing steps such as dilation erosion and rotation correction, which significantly improves the recognition success rate and processing efficiency.

[0193] Therefore, the technical solution provided in this embodiment supports image enhancement and normalization for multiple barcode protocols. During training, through mixed learning of multiple types of samples, the model acquires the ability to understand and recover cross-protocol structures, no longer limited to a single standard (such as ECC200), but adaptable to barcode encoding schemes with different module sizes, alignment methods, and error correction code arrangements. On the other hand, a standard image is output in a single forward inference, eliminating cumbersome post-processing. Compared to traditional barcode image processing methods that require multiple operations (image enhancement → denoising → geometric correction → binarization → cropping), this method integrates all functions into a single model, achieving an integrated processing flow through end-to-end learning, reducing system complexity, and improving the real-time performance of the decoding link. Furthermore, this method is adaptable to complex industrial conditions, requiring no manual parameter tuning. The model training process incorporates data from various distortion scenarios such as contamination, occlusion, and deformation, enhancing the model to possess excellent structural completion and defect completion capabilities. No environmental or parameter fine-tuning is required during the deployment phase, truly achieving "plug and play." On the other hand, it can improve the robustness of barcode recognition and reduce recognition failures on the production line. Through deep learning-driven image understanding and generation, the enhanced barcode image extraction capability, as well as the clear boundary and semantic consistency of the results, significantly improves the recognition success rate under extreme conditions such as edge defects and severe contamination, effectively reducing problems such as missorting and false work orders caused by recognition failures on the production line.

[0194] To improve training efficiency and model generalization ability, mainstream deep learning frameworks such as PyTorch are typically used during training. The following parameter configuration and training strategy are provided as an example:

[0195] Generator learning rate l D1 =1×10 -4 The batch size is 64, the optimizer uses Adam, and the discriminator learning rate is l. D2 =1×10 -4 The batch size is 64. The optimizer uses SGD and adopts an alternating training strategy. That is, in each iteration cycle, the discriminator is first trained to better evaluate the authenticity and decodeability of the generated barcode image from the perspective of its corresponding protocol structure. Then the generator is trained to improve its ability to extract barcode identification areas and perform shape regularization and missing part completion on the extracted areas.

[0196] Throughout the training process, the parameters of the generator and discriminator are continuously updated so that the game process continuously approximates the distribution of the image corresponding to the barcode protocol, and the final output enhanced image can conform to the encoding protocol and structural rules corresponding to its barcode type.

[0197] Through the above training strategy, the resulting augmented model can automatically normalize and restore the structure of barcode images of different categories and defect levels in various industrial scenarios, and output high-quality binarized images that can be accurately decoded by standard barcode recognition engines, demonstrating good versatility and practicality for deployment.

[0198] In some embodiments, the method for obtaining the sample matrix barcode image includes:

[0199] We use industrial cameras to capture raw barcode images printed using common methods such as thermal printing, inkjet printing, and laser engraving in real industrial environments. We then design a special data augmentation strategy based on the barcode's structural features to perform structure-aware enhancement processing, generating training sample images with diverse defect forms, which serve as input for model training.

[0200] The enhancement strategy proposed in this application is not a simple application of traditional image enhancement or image deblurring methods, but is tailored to the structural characteristics of barcode images and common failure modes in industrial production, thus exhibiting stronger specificity and application adaptability. In particular, the encoded information in barcode images is highly structured and has limited fault tolerance space. Therefore, the enhancement strategy of this application is designed around the "structural perturbation and morphological restoration capability of the barcode region," reflecting a fundamental difference from general image enhancement models.

[0201] During the enhancement process, the barcode image samples include not only regular clear images but also image combinations covering one or more of the following defect features: deformation distortion, foreign object occlusion, contamination, uneven brightness, and mechanical scratches. By summarizing and modeling real failure types, the proposed enhancement strategy generates a representative training image group based on the image acquisition samples, effectively improving the model's adaptability and enhancement effect on barcode images under different industrial scenarios, equipment platforms, and acquisition conditions.

[0202] The augmentation strategy may include one or a combination of online and offline data augmentation methods. Online augmentation generates images in real time during model training, offering advantages such as high flexibility and minimal storage requirements; offline augmentation generates and saves augmented images before training, significantly improving training loading efficiency. Combining both approaches balances data diversity and training efficiency, enhancing the overall generalization ability of the model.

[0203] As an example, the offline enhancement stage expands the diversity of training samples by adding Gaussian noise and generating images under different lighting conditions. Online enhancement is then performed during training, such as random occlusion and brightness perturbation, thus forming an enhancement mechanism that combines "structure awareness + multi-source interference + industrial distortion simulation".

[0204] Figure 6 Examples of image enhancement data provided in embodiments of this application are shown. Figure 6 In the above, for the original data (images), Figure a shows the noise injection result; Figure b shows the structural redundancy in the dilation simulation; Figure c shows the structural loss in the erosion simulation; Figure d shows the visual contrast anomaly image under brightness perturbation; Figure e shows the local occlusion simulation; in addition, various interference operations have a clear barcode adaptation design:

[0205] 1. Noise injection to simulate interference during industrial camera imaging. Unlike general image denoising training, this method limits the noise intensity and distribution area, highlighting the boundary sensitivity of the barcode area and helping the model identify structural features in the interference.

[0206] 2. Morphological perturbation (expansion / erosion) simulates problems such as overprinting, underprinting, and uneven etching, emphasizing the ability to preserve structural details, and has a completely different design starting point from image deblurring models.

[0207] 3. Partial occlusion simulation: Simulates occlusion factors such as scratches, oil stains, and metallic reflections to enhance the model's ability to complete the structure of damaged barcodes.

[0208] 4. Illumination / Contrast Perturbation: To address the frequently changing lighting conditions in industrial scenarios, the overall brightness and local contrast are controlled, making the model more robust to dynamic lighting environments.

[0209] In summary, the enhancement strategy emphasizes the synergistic mechanism of "structure preservation" and "defect modeling," which is significantly different from general image enhancement techniques. It improves the extraction effect of barcode identification areas while ensuring the integrity of barcode structure and semantic information.

[0210] The training set constructed using the above strategy not only covers a rich variety of industrial defect patterns but also greatly expands the data distribution space that the model can learn from, thereby significantly improving the generalization ability and practicality of the barcode enhancement model. This strategy reduces the cost of collecting real defect samples and provides a data foundation and methodological support for the efficient deployment of enhancement models in real-world industrial scenarios.

[0211] In some embodiments, the method for obtaining the labeled data includes:

[0212] In response to annotation instructions for the three vertices of the barcode region in the sample matrix barcode image, the position and orientation of the barcode are determined; a binary barcode in a geometrically aligned format according to its barcode protocol is generated within the barcode region based on the decoded plaintext information or interactive input information and used as annotation data; or, the sample matrix barcode image is input into the region positioning model to obtain positioning data for determining the barcode region; and a binary barcode in a geometrically aligned format according to its barcode protocol is extracted from the barcode region using a barcode decoding algorithm and used as annotation data.

[0213] In practical applications, users use annotation tools to sequentially select three adjacent vertices of the barcode region in a sample image. The coordinates of these three vertices are sufficient to uniquely determine the barcode's position, size, and rotation angle in the image, providing a high-precision positioning basis for subsequent geometric normalization and structural reconstruction of the barcode region. Based on these three points, geometric algorithms (such as rotated rectangle fitting or the minimum bounding rectangle method) can be used to accurately calculate the boundary of the barcode region, achieving accurate separation and geometric regularization of the barcode region in complex backgrounds.

[0214] Subsequently, the extracted barcode area image can be parsed using a decoding algorithm to obtain its plaintext information; alternatively, the user can directly input the corresponding encoded data. Based on this plaintext content, a barcode generation algorithm is invoked to encode and render the image according to the corresponding barcode encoding protocol (such as DataMatrix, QR, PDF417, etc.), generating a standard-structured binary barcode image. This image strictly conforms to the structural specifications and pattern layout of the barcode protocol, while maintaining geometric alignment with the barcode area in the original image, serving as a structural alignment annotation map required for model training. This annotation method has the following advantages:

[0215] 1. Structural standardization: The generated annotation maps strictly follow the encoding and layout specifications of the barcode protocol, avoiding structural errors caused by manual annotation, deformation, and damage, thus improving the standardization and usability of the annotation data.

[0216] 2. Positional consistency: The barcode region is accurately geometrically mapped using three vertices, ensuring that the generated standard barcode image can completely correspond to the original barcode region during training, thus improving the model's learning effect on "position-structure" consistency.

[0217] 3. It has strong geometric adaptability and is compatible with barcode images of arbitrary rotation angles and scales. It achieves accurate registration of the standard barcode with the original image area through affine or perspective transformation mechanisms, making it suitable for barcode training needs in complex scenarios.

[0218] 4. Multi-source information fusion supports automatic decoding to obtain encoded content, and also allows for supplementary input through manual interaction, effectively improving the flexibility and adaptability of the annotation process and solving the problem of being unable to generate tags due to decoding failure.

[0219] Furthermore, the method for acquiring the labeled data may also include an automated process: inputting the sample matrix barcode image into the barcode identifier region extraction, geometric shape regularization, and defect completion model trained in this application to enhance and locate the barcode region; subsequently, using a barcode decoding algorithm to extract the plaintext content; and based on the decoding result, generating a standard barcode image with standardized structure and uniform resolution as labeled data strictly aligned with the barcode region in the original image. The decoding algorithm can be implemented using a general-purpose barcode decoding library such as libdmtx.

[0220] This automated annotation process significantly reduces the cost of manual intervention and avoids structural deviations caused by poor image quality. It provides reliable support for building large-scale, high-quality, and structurally consistent barcode training sets, and is especially suitable for training high-robustness, high-precision barcode enhancement models in industrial vision.

[0221] In some embodiments, the generator model adopts a UNet architecture, and the generator model includes:

[0222] The encoder, consisting of multiple convolutional and max-pooling layers, is used to progressively refine and abstract the features of the input image by stacking them layer by layer.

[0223] The decoder is used to expand deep features to the spatial scale of the original image using deconvolution and upsampling techniques.

[0224] Skip connections are used to build direct paths between the encoder and decoder, allowing low-level details to be incorporated into high-level features.

[0225] The final output layer is used to generate an image in a canonical format that is geometrically aligned with its barcode protocol through a sigmoid activation function.

[0226] As an example, this paper addresses issues such as complex background interference, geometric deformation, and occlusion in barcode images, achieving automatic barcode extraction, geometric regularization, and high-quality completion of missing areas, demonstrating strong versatility and robustness. The generator model (i.e., the generator model) includes the following structural modules:

[0227] The encoder consists of multiple convolutional layers and max pooling layers. It uses a layer-by-layer stacking approach to perform multi-scale abstraction and extraction of low-level texture features and high-level structural semantics of barcode images, thereby improving the robust extraction capability of features under deformed and occluded conditions.

[0228] The decoder uses deconvolution and upsampling operations to gradually restore the spatial structure of the image, and combines deep abstract semantic information to achieve precise extraction of the marked regions of the original image structure;

[0229] Skip Connections directly pass shallow features from the encoder to the corresponding layers of the decoder, allowing detailed information (such as barcode edges, contrast, and module boundaries) to be preserved, effectively avoiding the loss of image structure caused by relying solely on high-level semantics.

[0230] The output layer maps the decoder output to a 0-1 grayscale space using the sigmoid activation function, generating a standardized grayscale barcode image, and supports subsequent thresholding to obtain a clear binary barcode image.

[0231] Compared to traditional methods that only enhance or simply restore barcode images, this embodiment utilizes structural modeling to achieve an integrated processing flow from barcode recognition region extraction and shape regularization to defect completion. The introduction of the skip connection mechanism significantly improves the fidelity of image restoration, ensuring that the generated image possesses complete barcode structural information, thereby enhancing the success rate and accuracy of subsequent decoding.

[0232] In some embodiments, the discriminator model adopts a ResNet18 architecture, and the discriminator model includes:

[0233] Multiple residual modules are used to perform feature extraction operations through convolutional kernels and to perform data normalization processing through batch normalization technology;

[0234] The global average pooling layer is used to perform global average pooling on the extracted features by skipping the residual module.

[0235] The output layer connects to the output of the global average pooling layer and outputs a predicted value representing the degree to which the image geometry aligns with the barcode protocol specifications through an activation function.

[0236] As an example, the discriminator model employs a lightweight ResNet18 architecture to determine the structural similarity in geometry between the generator's output image and a standard barcode image generated according to a barcode encoding protocol (such as ECC200, ISO / IEC18004, etc.), thereby guiding the generator to generate a barcode image that better conforms to its corresponding barcode protocol. This model includes:

[0237] Multiple residual modules: Combining 3×3 convolutional kernels with Batch Normalization technology, the deep semantic structure of barcode images is extracted while maintaining network stability;

[0238] Global average pooling layer: Compresses and summarizes feature maps from all channels, effectively reducing the model parameter size and improving adaptability to different input scales and barcode styles;

[0239] Output layer: The sigmoid activation function is used to output a predicted value between 0 and 1, which reflects the credibility of whether the currently generated barcode image has "the true structure that conforms to the corresponding barcode protocol".

[0240] Through residual structure and skip connection mechanism, the discriminator model has a stronger ability to express deep features, effectively distinguishing real barcode images from fake images generated by the generator, thereby providing the generator with accurate and structure-sensitive feedback signals and improving the generation quality.

[0241] In some embodiments, the generator's loss function consists of multiple functional losses, which are jointly optimized through a weighted approach, specifically including:

[0242] Adversarial loss: By training the generator and discriminator in a game-like manner, the degree of fit of the output barcode image to its corresponding protocol structure is improved, and the geometric consistency is enhanced;

[0243] Image reconstruction loss: Constrains the consistency between the generated image and the labeled image at the pixel level to ensure the completeness of the incomplete image and the accuracy of the structural restoration;

[0244] Feature loss: Utilize pre-trained models to extract multi-scale semantic features, improve the perceptual consistency between generated and labeled images in the high-level feature space, and ensure semantic alignment of content;

[0245] Structural similarity loss: Focuses on matching local structure and contrast to ensure that the generated barcode area has a clear grid structure and module edges, thereby improving the quality of structural reproduction.

[0246] This multi-branch loss design demonstrates the applicant's deep understanding and systematic modeling capabilities for barcode image enhancement tasks. It integrates semantic understanding at the perceptual level with geometric regularity requirements at the structural level, jointly optimizing various performance metrics of the generator. Furthermore, by setting flexible and adjustable loss weights, the model can be customized for different industrial scenarios in practical applications. For example, for severely damaged metal barcode images in SMT production lines, the structural similarity loss weight can be increased; in scenarios where decoding success rate is prioritized, the proportion of adversarial loss can be enhanced, thereby maximizing model efficiency.

[0247] In summary, the UNet-based generative adversarial network architecture proposed in this embodiment, combined with an end-to-end design for automatic extraction of barcode identification regions, shape regularization, and defect completion, significantly improves the model's ability to reconstruct and enhance complex barcode images, and possesses the following significant technical advantages:

[0248] On the one hand, the barcode recognition process modeling merges the three-stage tasks of "recognition region extraction - geometric regularization - structural completion" into a unified network structure, simplifying system deployment and improving efficiency. On the other hand, the robust structural design, through skip connections, residual modules, and structure-aware loss function design, enables the model to maintain structural integrity and decoding accuracy even when facing complex inputs such as occlusion and deformation. Furthermore, its high industrial adaptability and flexibility, along with the weighted combination of multi-branch losses, allows the model to be tuned on demand, making it suitable for enhancement needs under various barcode formats and imaging conditions. Finally, its end-to-end automatic annotation capability, combined with the enhanced image and decoded content, can automatically generate standard images with unified structure and aligned positions, serving as high-quality annotation data and effectively supporting the automatic construction of subsequent large-scale training data.

[0249] System Implementation Example.

[0250] This application provides a barcode enhancement image processing system based on generative adversarial networks. Its specific implementation method and the technical effects achieved are the same as those described in the above method implementation method, and some contents will not be repeated.

[0251] See Figure 3 , Figure 3 This is a schematic diagram of a barcode enhancement image processing system provided in an embodiment of this application. The system includes:

[0252] The information acquisition module is used to acquire the training set and the initial model; the training set includes multiple training data, each training data includes a sample matrix barcode image and its corresponding annotation data; the initial model is an adversarial network constructed from a generator model and a discriminator model.

[0253] The model training module is used to train the model using the training set based on the initial model and the target loss function to obtain the barcode enhancement model; the target loss function includes a generator loss function using weighted composite loss technology and a discriminator loss function using binary cross-entropy loss technology.

[0254] The barcode acquisition module is used to capture data matrix barcode images in a real environment using an industrial camera and input them into the barcode enhancement model to obtain a standard binary barcode.

[0255] In some embodiments, the method for obtaining the sample matrix barcode image includes:

[0256] The original barcode images of thermal, inkjet, and laser engraving printing methods are acquired in an industrial environment using an industrial camera. Data augmentation strategies are then applied to these images, which are used as sample matrix barcode images. Each sample matrix barcode image contains regular barcode images and barcode images with anomalous features. The anomalous features of the barcode images include one or more combinations of the following: shape distortion, foreign object obstruction, dirt, uneven brightness, scratches, and blurring.

[0257] In some embodiments, the data augmentation strategy includes applying at least one interference operation to the sample matrix barcode image, the interference operation including noise injection, morphological defect simulation, blurring, occlusion overlay, and illumination adjustment;

[0258] When the interference operation is noise injection, Gaussian noise and salt-and-pepper noise are added to the original barcode image to simulate imaging sensor noise and environmental interference.

[0259] When the interference operation is used to simulate morphological defects, it simulates barcode redundancy or missing defects in barcode printing through expansion or erosion operations.

[0260] When the interference operation is a fuzzy scene simulation, the fuzziness caused by device jitter or rapid movement during barcode scanning is simulated by using a fuzz kernel;

[0261] When the interference operation is to simulate occlusion and dirt, objects of different shapes, sizes and transparency are randomly superimposed on the barcode image to simulate the loss of some information in the barcode due to stains, scratches or external objects.

[0262] When the interference operation is a simulation of lighting conditions, the changes in barcode brightness are simulated by adjusting the overall brightness value of the barcode image, the contrast is adjusted by histogram stretching or local contrast enhancement, and the brightness distribution of the image is adjusted by gamma correction through nonlinear transformation.

[0263] In some embodiments, the method for obtaining the annotation data includes: determining the position and orientation of the barcode in response to annotation instructions for the three vertices of the barcode region of the sample matrix barcode image; generating a binary barcode conforming to the ECC200 standard within the barcode region based on the decoded plaintext information or interactive input information and using it as annotation data; or, inputting the sample matrix barcode image into a region positioning model to obtain positioning data for determining the barcode region; and extracting a binary barcode conforming to the ECC200 standard from the barcode region using a barcode decoding algorithm and using it as annotation data.

[0264] In some embodiments, the generator model adopts a UNet structure, comprising: an encoder, including multiple convolutional layers and max pooling layers, for progressively refining and abstracting the features of the input image through layer-by-layer stacking; a decoder, for restoring the original resolution of the features using deconvolution and upsampling techniques; skip connections, for constructing direct paths between the encoder and decoder, allowing low-level details to be integrated into high-level features; and a final output layer, for generating a standardized grayscale barcode image using a sigmoid activation function.

[0265] In some embodiments, the discriminator model adopts a ResNet18 architecture, and the discriminator model includes:

[0266] Multiple residual modules are used to perform feature extraction operations through convolutional kernels and to perform data normalization processing through batch normalization technology;

[0267] The global average pooling layer is used to perform global average pooling on the extracted features by skipping the residual module.

[0268] The output layer connects to the output of the global average pooling layer and outputs a predicted value representing the probability that the input image is a real barcode image through an activation function.

[0269] In some embodiments, the generator loss function consists of an adversarial loss function, an image inpainting loss function, a feature loss function, and a structural similarity loss function, and is weighted and summed using preset weights.

[0270] Equipment implementation example.

[0271] This application also provides an electronic device, including one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors according to any of the methods described in the method embodiments.

[0272] See Figure 4 , Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown.

[0273] Electronic devices may include, for example, at least one memory 11, at least one processor 12, and a bus 13 connecting different platform systems.

[0274] The memory 11 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 111 and / or cache memory 112, and may further include read-only memory (ROM) 113.

[0275] The memory 11 also stores a computer program, which can be executed by the processor 12 to enable the processor 12 to implement the steps of any of the above methods.

[0276] The memory 11 may also include a utility 114 having at least one program module 115, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0277] Accordingly, processor 12 can execute the aforementioned computer program, and can also execute utility 114.

[0278] The processor 12 may employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0279] Bus 13 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any bus structure with multiple bus structures.

[0280] The electronic device can also communicate with one or more external devices, such as keyboards, pointing devices, Bluetooth devices, etc., and with one or more devices capable of interacting with the electronic device, and / or with any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed through input / output interface 14. Furthermore, the electronic device can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 15. Network adapter 15 can communicate with other modules of the electronic device via bus 13. It should be understood that, although not shown in the figures, in practical applications, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0281] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are configured to distinguish similar objects and are not necessarily configured to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0282] This application describes the invention from the perspectives of purpose, performance, progress, and novelty, and it meets the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

Claims

1. A barcode enhancement image processing method based on generative adversarial networks, characterized in that, The method is used to perform barcode identification region extraction, geometric regularization transformation, and structural defect completion on barcode images collected in an industrial environment, including the following steps: Construct a training set; the training set contains multiple matrix barcode image data, including data on deformation, occlusion, and printing defects, collected by industrial cameras in a real production line environment, and their corresponding annotation data; the annotation data is the corresponding matrix barcode image data in which the barcode area position is manually marked, and a standard format barcode image is generated according to the encoding protocol applicable to the barcode, which is geometrically aligned with the barcode protocol and corresponds one-to-one with the spatial position of the area. An adversarial network model is constructed using a generator and a discriminator as the initial model. The generator is built based on the UNet structure and optimized through a weighted combination of adversarial loss, image reconstruction loss, structural similarity loss, and feature consistency loss. It is used to automatically extract barcode identification regions from 2D barcode images in real industrial scenarios and to achieve geometric regularization of the barcode structure and structural completion of missing regions, so that the output barcode image is geometrically aligned with the standard format of its barcode protocol. The discriminator adopts the ResNet architecture and uses the discrimination ability of the binary cross-entropy loss optimizer to determine the geometric structural similarity between the generator's output image and the standard barcode image generated according to the corresponding barcode encoding protocol. The initial model is trained based on a composite loss function to obtain an enhanced model with barcode extraction, geometric regularization and defect completion capabilities; Data matrix barcode images are captured using industrial cameras in real industrial environments, and then input into a trained augmentation model to output standardized barcode images with one or more of the following features: Filter background interference areas; perform geometric regularization processing on structural distortions in the barcode to restore each module to a square unit that conforms to the standard matrix arrangement; for barcode module missing due to one or more factors such as occlusion, dirt, scratches, or ink loss, perform structural consistency completion based on contextual semantic information; generate results that meet the structural rules of the corresponding barcode encoding protocol, and output a standard binarized image with clear boundaries and regular geometric structure.

2. The barcode enhancement image processing method according to claim 1, characterized in that, The methods for obtaining the labeled data include: In response to annotation instructions for the three vertices of the barcode region in the sample matrix barcode image, the position and orientation of the barcode are determined; a binary barcode in a geometrically aligned format according to its barcode protocol is generated within the barcode region based on the decoded plaintext information or interactive input information and used as annotation data; or, the sample matrix barcode image is input into the region positioning model to obtain positioning data for determining the barcode region; and a binary barcode in a geometrically aligned format according to its barcode protocol is extracted from the barcode region using a barcode decoding algorithm and used as annotation data.

3. The barcode enhancement image processing method according to claim 1, characterized in that, The generator model adopts the UNet architecture and includes: The encoder, consisting of multiple convolutional and max-pooling layers, is used to progressively refine and abstract the features of the input image by stacking them layer by layer. The decoder is used to expand deep features to the spatial scale of the original image using deconvolution and upsampling techniques. Skip connections are used to build direct paths between the encoder and decoder, allowing low-level details to be incorporated into high-level features. The final output layer is used to generate an image in a canonical format that is geometrically aligned with its barcode protocol through a sigmoid activation function; The discriminator model adopts a ResNet18 architecture, and the discriminator model includes: Multiple residual modules are used to perform feature extraction operations through convolutional kernels and to perform data normalization processing through batch normalization technology; The global average pooling layer is used to perform global average pooling on the extracted features by skipping the residual module. The output layer connects to the output of the global average pooling layer and outputs a predicted value representing the degree to which the image geometry aligns with the barcode protocol specifications through an activation function.

Images