Bar code enhanced image processing method and system based on generative adversarial network

Abstract

The invention provides a bar code enhanced image processing method and system based on a generative adversarial network. The method comprises the following steps: acquiring a training set and an initial model; the training set comprises a plurality of training data, and each training data comprises a sample matrix barcode image and corresponding annotation data; the initial model is an adversarial network constructed and generated by a generator model and a discriminator model; performing model training by using the training set according to the initial model and the target loss function to obtain a bar code enhancement model; the target loss function comprises a generator loss function adopting a weighted composite loss technology and a discriminator loss function adopting a binary cross entropy loss technology; and acquiring a data matrix bar code image in a real environment by using an industrial camera, and inputting the data matrix bar code image into the bar code enhancement model to obtain a standard binary bar code. According to the DM bar code decoding method and device, the DM bar code decoding success rate is improved by constructing the adversarial training architecture composed of the generator and the discriminator.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to a barcode enhancement image processing method and system based on a generative adversarial network. Background Art

[0002] In smart manufacturing scenarios, DM barcodes serve as the core carrier of product traceability. For example, in the SMT (Surface Mount Technology) production line, which involves multiple links such as high-speed patch, reflow soldering, AOI (automatic optical inspection), and SPI (solder paste inspection), each PCB board, electronic component or assembly is usually marked with a DM barcode to ensure accurate production data management. Its recognition reliability plays a key role in production traceability, quality control, and automated management.

[0003] However, barcodes in industrial environments are often interfered by multiple factors: blurred module edges caused by printing defects, structural damage caused by mechanical wear, contrast attenuation caused by uneven illumination, motion blur caused by camera movement, etc. Traditional solutions mainly rely on manually designed feature extraction methods such as threshold segmentation and morphological operations, but their generalization ability is greatly limited in the face of complex scenarios, and the success rate of DM barcode decoding is not ideal.

[0004] Therefore, there is an urgent need to provide a barcode enhancement image processing method and system based on a generative adversarial network to enhance the decoding success rate. Summary of the invention

[0005] In view of the problem that the success rate of DM barcode decoding in the prior art is not ideal, the present application provides a barcode enhancement image processing method and system based on a generative adversarial network.

[0006] The purpose of this application is achieved by the following technical solutions: In a first aspect, the present application provides a barcode enhancement image processing method based on a generative adversarial network, the method comprising: Obtain a training set and an initial model; the training set includes a plurality of training data, each of which includes a sample matrix barcode image and its corresponding annotation data; the initial model is an adversarial network constructed by a generator model and a discriminator model; According to the initial model and the target loss function, the training set is used to perform model training to obtain a barcode enhancement model; the target loss function includes a generator loss function using a weighted composite loss technology and a discriminator loss function using a binary cross entropy loss technology; An industrial camera is used to collect a data matrix barcode image in a real environment and input it into the barcode enhancement model to obtain a standard binary barcode.

[0007] The beneficial effect of this technical solution is that compared with the traditional barcode image processing method, a large amount of manual intervention is often required to perform operations such as image preprocessing and parameter adjustment to adapt to different barcode quality and environmental conditions. The method based on the generative adversarial network provided by this technical solution, on the one hand, realizes the automated barcode image enhancement processing, reduces the manual complex image processing parameter setting and a large amount of post-correction work, saves manpower and time costs and improves processing efficiency, and is suitable for barcode data processing needs in large-scale industrial applications. On the other hand, through the generation capability of the generative adversarial network, the low-quality barcode images taken under various complex conditions in the industrial environment are converted into binary barcode images that meet the ECC200 standard, effectively repairing the defects and noise in the barcode image, enhancing the edge and module features of the barcode, making it easier to be accurately read and decoded by the barcode recognition equipment, thereby significantly improving the recognition success rate of the barcode, reducing the recognition failure caused by barcode quality problems, and ensuring the smooth progress of barcode application in the production process. On the other hand, by adopting an end-to-end learning architecture, low-quality barcode images are directly input into the model. After one forward propagation calculation, a binary barcode image that meets the standards can be output. Compared with the traditional multi-stage barcode image processing method, it avoids multiple steps such as tedious preprocessing, feature extraction, and post-processing, reduces the data transmission and conversion time between different processing links, and improves the efficiency and real-time performance of the entire barcode image processing process. It can meet the requirements of rapid barcode recognition and processing on high-speed production lines and improve overall production efficiency.

[0008] In some possible implementations, the method of acquiring the sample matrix barcode image includes: An industrial camera is used to collect original barcode images printed by thermal, inkjet and laser engraving in an industrial environment, and data enhancement is performed on them through a data enhancement strategy. The enhanced images are used as sample matrix barcode images; each of the matrix barcode images includes a regular barcode image and an abnormal feature barcode image, and the abnormal features of the abnormal feature barcode image include one or more of the following combinations: shape distortion, foreign body occlusion, dirt, uneven brightness, scratches and blur.

[0009] The beneficial effect of this technical solution is that during the model training process, starting from the specific application scenario, barcode images covering common printing methods and abnormal features are used, so that the trained barcode enhancement model has a strong generalization ability. Therefore, whether it is on different production lines or under different lighting, temperature, humidity and other environmental conditions, the barcode images can be stably output with high-quality enhanced barcode images, ensuring the reliability of barcode processing.

[0010] In some possible implementations, the data enhancement strategy includes: applying at least one interference operation to the sample matrix barcode image, the interference operation including noise injection, morphological defect simulation, blur processing, occlusion superposition and illumination adjustment; When the interference operation is noise injection, Gaussian noise and salt and pepper noise are added to the original barcode image to simulate the imaging sensor noise and environmental interference; When the interference operation is a morphological defect simulation, the barcode redundancy or missing defects in barcode printing are simulated by dilation or corrosion operation; When the interference operation is blur scene simulation, blur kernel is used to simulate the blurring of the barcode caused by device shaking or rapid movement during scanning; When the interference operation is occlusion and dirt simulation, occlusions of different shapes, sizes and transparencies are randomly superimposed on the barcode image to simulate the situation where part of the barcode information is lost due to stains, scratches or external objects; When the interference operation is a simulation of illumination conditions, the change of the barcode brightness is simulated by adjusting the overall brightness value of the barcode image, the contrast is adjusted by histogram stretching or local contrast enhancement, and the brightness distribution of the image is adjusted by gamma correction of nonlinear transformation.

[0011] The beneficial effects of this technical solution are: by simulating various adverse conditions that may occur in industrial environments, the adaptability and robustness of the model to barcode images of different qualities and environments are enhanced, and the model's ability to handle various complex barcode images in practical applications is improved. The diversified training samples generated by the data enhancement strategy can help the model better learn the essential characteristics of barcode images, rather than just being limited to specific imaging conditions or defect types, so that the trained model has stronger generalization capabilities and can be applied to barcode image enhancement tasks in different industrial scenarios. In actual industrial environments, collecting high-quality barcode image data covering various defects and interference conditions often requires a lot of time and resource investment. Through data enhancement strategies, a large amount of simulated data with different characteristics can be generated on the existing basic data set, effectively expanding the scale of the data set and reducing dependence on large-scale real data collection, thereby reducing data collection costs, improving R&D efficiency, and accelerating model training and optimization processes.

[0012] In some possible implementations, the method for acquiring the labeled data includes: In response to the labeling instructions for the three vertices of the barcode area of ​​the sample matrix barcode image, the position and orientation of the barcode are determined; a binary barcode that complies with the ECC200 standard is generated in the barcode area according to the decoded plain text information or the interactive input information and used as the labeling data; or, the sample matrix barcode image is input into the area positioning model to obtain the positioning data for determining the barcode area; and a binary barcode that complies with the ECC200 standard is extracted from the barcode area in combination with the barcode decoding algorithm and used as the labeling data.

[0013] The beneficial effect of this technical solution is that a binary barcode that complies with the ECC200 standard is generated as annotation data based on the decoded plaintext information or user interactive input information, ensuring the standard consistency of the annotation data with the barcode in actual industrial applications. The above-mentioned standard-based annotation method enables the model to learn accurate barcode features and specifications during the training process, which helps to reduce the risk of model performance degradation due to non-standard annotation data.

[0014] In some possible implementations, the generator model adopts a UNet structure, and the generator model includes: The encoder includes multiple convolutional layers and maximum pooling layers, which are used to gradually refine and abstract the features of the input image by stacking them layer by layer; A decoder that restores features to their original resolution using deconvolution and upsampling techniques; Skip connections are used to build direct paths between the encoder and decoder, so that low-level details can be integrated into high-level features; The final output layer is used to generate a standardized grayscale barcode image through a sigmoid activation function.

[0015] The beneficial effect of this technical solution is that compared with the traditional model that only relies on high-level semantic features for generation, the technical solution of this embodiment, due to the jump connection in the UNet structure, the low-level detail features can be directly passed to the decoder part, and when generating the final barcode image, the edge, line and other detail information of the barcode can be better retained. In other words, the UNet structure can generate clearer and more accurate barcode images, reducing decoding errors caused by loss of details.

[0016] In some possible implementations, the discriminator model adopts a ResNet18 structure, and the discriminator model includes: Multiple residual modules for feature extraction via convolutional kernels and data normalization via batch normalization techniques; The global average pooling layer is used to connect the residual module through skip connections and perform global average pooling operations on the extracted features; The output layer is used to connect the output end of the global average pooling layer and output the predicted value of the probability of whether the input image is a real barcode image through the activation function.

[0017] The beneficial effect of this technical solution is that the high-level semantic features of the input image can be extracted layer by layer through multiple residual modules in the ResNet18 structure, and the low-level features such as edges and textures can be gradually abstracted to the high-level barcode structure and pattern features. The ability of layer-by-layer feature extraction and abstraction enables the discriminator model to more accurately capture the difference between the real barcode image and the generated barcode image, thereby improving the ability to distinguish the authenticity of the input image and providing more effective feedback for the generator model. The application of the global average pooling layer not only reduces the number of parameters of the model and reduces the complexity of the model, but also enhances the adaptability of the model to different input images.

[0018] In some possible implementations, the generator loss function is composed of an adversarial loss function, an image restoration loss function, a feature loss function, and a structural similarity loss function, and is weighted and summed using preset weights.

[0019] The beneficial effect of this technical solution is that by comprehensively considering adversarial loss, image restoration loss, feature loss and structural similarity loss, different loss functions constrain and optimize the generator from different angles, so that the model can learn a more comprehensive and richer image feature representation. Adversarial loss helps the model learn data distribution and improves its adaptability to different barcode image styles and types; image restoration loss ensures the accuracy of the model in pixel-level restoration and adapts to barcode images under various imaging conditions; feature loss enables the model to understand the essential characteristics of barcode images at the semantic level and can handle input images of different complexity and quality; structural similarity loss strengthens the model's attention to the details of the barcode structure, so that it can maintain the standardization and integrity of the barcode during the generation process. The comprehensive optimization method enhances the generalization ability of the model, so that it can stably output high-quality enhancement results when facing different barcode images in various industrial environments. The weighted summation method of preset weights provides flexibility for the adjustment and optimization of the model. The weights of different loss functions can be adjusted according to actual application requirements and data characteristics to highlight or balance certain specific aspects of performance. For example, in application scenarios that require extremely high barcode boundary clarity, the weight of the structural similarity loss can be appropriately increased; in cases where it is necessary to focus on the overall realism of the generated image, the weight of the adversarial loss can be increased. The above flexibility enables the model to better adapt to different practical needs and industrial environments. The performance of the generator model can be optimized and customized through simple weight adjustments without large-scale modification and retraining of the model structure.

[0020] In some possible implementations, the method further includes: post-processing the standard binary barcode to obtain a target binary barcode; the post-processing includes the following steps: For the standard binary barcode, an adaptive threshold binarization method is used to calculate the threshold adapted to the brightness of different regions according to the local window to separate the barcode module from the background; corresponding to the following formula:

[0021] Among them, Gbin(x,y) is the pixel value after binarization, G(x,y) is the pixel value of the generated barcode image, and T(x,y) is the pixel threshold. The morphological opening operation is used to eliminate isolated noise points and connect broken boundaries to obtain the target binary barcode.

[0022] The beneficial effect of this technical solution is that through the post-processing steps of adaptive threshold binarization and morphological opening operation, the quality of the standard binary barcode image output by the generator is further improved, making it more compliant with the ECC200 standard and further reducing the decoding failure rate.

[0023] In a second aspect, the present application also provides a barcode enhancement image processing system based on a generative adversarial network, the system comprising: An information acquisition module is used to acquire a training set and an initial model; the training set includes a plurality of training data, each of which includes a sample matrix barcode image and its corresponding annotation data; the initial model is an adversarial network constructed by a generator model and a discriminator model; A model training module, used to perform model training using the training set according to the initial model and the target loss function to obtain a barcode enhancement model; the target loss function includes a generator loss function using a weighted composite loss technology and a discriminator loss function using a binary cross entropy loss technology; The barcode acquisition module is used to use an industrial camera to collect a data matrix barcode image in a real environment and input it into the barcode enhancement model to obtain a standard binary barcode.

[0024] In a third aspect, the present application further provides an electronic device comprising one or more processors and a memory; one or more programs are stored in the memory and are configured to be executed by the one or more processors to perform any of the methods described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present application is further described below in conjunction with the accompanying drawings and implementation methods.

[0026] Figure 1 It is a flowchart of a barcode enhancement image processing method provided in an embodiment of the present application; Figure 2It is a schematic diagram of the structure of a generative adversarial network provided in an embodiment of the present application; Figure 3 It is a module schematic diagram of a barcode enhancement image processing system provided in an embodiment of the present application; Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] Below, in conjunction with the accompanying drawings and specific implementation methods, the present application is further described. It should be noted that, under the premise of no conflict, the various embodiments described below or the various technical features can be arbitrarily combined to form a new embodiment. The implementation procedures of the present application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation procedures, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0028] The following is a brief description of the technical field and related terms of the embodiments of the present application to facilitate understanding by those skilled in the art.

[0029] Generative Adversarial Network (GAN) is a generative model that learns by playing games between two neural networks.

[0030] Data Matrix (DM) barcode image. The minimum size of DM barcode is the smallest among all barcodes currently. Therefore, it is suitable for the identification of small parts. It can be directly printed on the small parts entity. It is widely used in small items such as circuits and medicines, as well as the assembly line production process of the manufacturing industry. The original barcode image mentioned in this application is the DM barcode image. The ECC200 standard is a two-dimensional matrix barcode, a type of Data Matrix code, which uses the Reed-Solomon error correction algorithm to recover data even if part of the code is damaged, thereby improving the reliability of data reading. The ECC200 code in this application is a square or rectangular.

[0031] In recent years, deep learning technology has developed rapidly, and significant progress has been made in the field of image super-resolution. Image super-resolution methods based on deep learning have been gradually extended to barcode recognition tasks. Super-resolution technologies such as SRCNN and EDSR can effectively improve the clarity of low-resolution images, thereby enhancing the readability of blurred barcodes. However, the above improvements lack consideration of field applications. On the one hand, the super-resolution technology of blurred barcodes mainly relies on simulation data for training, but due to the difference between simulation data and actual blurred barcodes, the effect of improving clarity on some blurred images is not good. On the other hand, in actual industrial environments, the factors that affect barcode recognition are not limited to blur problems, but also include deformation, occlusion, and lighting differences. Once the above problems occur, relying solely on super-resolution technology to improve the decoding success rate is not ideal, so further optimization and improvement are still needed.

[0032] Based on this, in order to solve the recognition failure problem caused by the degradation of DM barcode image quality in industrial environments, this application provides a barcode enhancement image processing method and system based on a generative adversarial network. By constructing an adversarial training architecture consisting of a generator and a discriminator, the generator can learn and convert real barcode data into high-quality ECC200 standard binary barcode images, while the discriminator ensures that the generated results meet the characteristics of real barcodes. The generator adopts a UNet structure, and the discriminator adopts a ResNet18 structure. Combined with the loss function, it ensures that the generated barcode image is highly consistent with the real standard barcode image at multiple levels such as pixel level, semantic level, and structural level.

[0033] The following will be combined with the accompanying drawings and specific implementation methods to specifically describe in detail the technical solutions of the embodiments of the present application and how the technical solutions of the embodiments of the present application solve the above-mentioned technical problems. It should be noted that the embodiments or technical features described below can be arbitrarily combined to form new embodiments, and the same or similar concepts or processes may not be repeated in some embodiments. Obviously, the described embodiments are part of the embodiments of the present application, not all of them.

[0034] Method embodiment.

[0035] See also Figure 1 , Figure 1 : is a flow chart of a barcode enhancement image processing method provided by an embodiment of the present application. This embodiment provides a barcode enhancement image processing method based on a generative adversarial network, the method comprising: S101, obtaining a training set and an initial model; the training set includes a plurality of training data, each of which includes a sample matrix barcode image and its corresponding annotation data; the initial model is an adversarial network constructed by a generator model and a discriminator model; It can be considered that each training data in the training set consists of a sample matrix barcode image and its corresponding annotated data. The sample matrix barcode images cover various situations that occur in industrial environments, such as different printing methods such as thermal, inkjet, and laser engraving, as well as barcode samples with different defect characteristics such as normal, deformed, blocked, and blurred. The annotated data is obtained by annotating the sample barcode images, which is used to guide model training and help the model learn the real characteristics of the barcode.

[0036] The initial model is an adversarial network consisting of a generator model and a discriminator model. The goal of the generator model is to convert the input low-quality barcode image into a binary barcode image that meets the ECC200 standard; the discriminator model is used to distinguish the barcode image generated by the generator model from the real standard barcode image, provide feedback to the generator model, and prompt the generator model to continuously optimize the generation results.

[0037] S102, performing model training using the training set according to the initial model and the target loss function to obtain a barcode enhancement model; the target loss function includes a generator loss function using a weighted composite loss technology and a discriminator loss function using a binary cross entropy loss technology; The target loss function is used to guide the direction of model training. Among them, the generator loss function adopts weighted composite loss technology, which comprehensively considers multiple aspects such as adversarial loss, image restoration loss, feature loss and structural similarity loss. Adversarial loss forces the image generated by the generator to be close to the real barcode image in distribution, making it more realistic; image restoration loss ensures the accuracy of pixel-level restoration by calculating the L1 distance between the generated image and the target image; feature loss improves the semantic consistency of the generated image by comparing the differences in the features of the discriminator's intermediate layers; structural similarity loss optimizes the boundary clarity of the barcode module, making it easier to decode. The discriminator loss function adopts binary cross entropy loss technology to measure the classification accuracy of the discriminator for real barcode images and generated barcode images, helping the discriminator model to better learn to distinguish the features between the two.

[0038] During the training process, the sample matrix barcode image is input into the initial model using the training set prepared by S101. The generator model generates an enhanced barcode image based on the input low-quality barcode image. The discriminator module receives the generated barcode image and the real standard barcode image at the same time, compares and judges them, and outputs the evaluation result of the authenticity of the generated image. The loss values ​​of the generator and the discriminator are calculated according to the target loss function, and the parameters of the model are adjusted through optimization algorithms (such as Adam, SGD, etc.). The training process is continuously iterated, so that the generator can generate enhanced images that are closer and closer to the real standard barcode image. At the same time, the judgment ability of the discriminator module is also improved, and finally a trained barcode enhancement model that meets the training standards is obtained.

[0039] S103, using an industrial camera to collect a data matrix barcode image in a real environment and input it into the barcode enhancement model to obtain a standard binary barcode.

[0040] In specific applications, industrial cameras are used to collect data matrix barcode images in real working environments. Data matrix barcode images may be affected by various practical factors such as uneven lighting, camera motion, barcode contamination, etc., resulting in reduced image quality and difficulty in direct and accurate recognition and decoding.

[0041] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a generative adversarial network provided by the implementation of this application. Input barcode data (sample matrix barcode image in training data) to the generator (model) to obtain generated data (enhanced barcode image). The annotated data corresponding to the sample matrix barcode image is used as label data, and the generated data and label data are input into the discriminator (model) to train the model.

[0042] The data matrix barcode image collected in the real environment is input into the trained barcode enhancement model. The generator processes and enhances the input low-quality barcode image, applies the knowledge and feature mapping learned during the training process, and outputs a binary barcode image (compliant with the ECC200 standard). It can be considered that the enhanced barcode image has higher quality and readability, can better meet the requirements of the subsequent barcode recognition and decoding system, and improve the efficiency and reliability of the entire barcode application process.

[0043] Therefore, compared with the traditional barcode image processing method, a lot of manual intervention is often required to perform operations such as image preprocessing and parameter adjustment to adapt to different barcode quality and environmental conditions. The method based on the generative adversarial network provided by the technical solution, on the one hand, realizes the automated barcode image enhancement processing, reduces the manual complex image processing parameter setting and a large amount of post-correction work, saves manpower and time costs and improves processing efficiency, which is suitable for the barcode data processing needs in large-scale industrial applications. On the other hand, through the generation ability of the generative adversarial network, the low-quality barcode images taken under various complex conditions in the industrial environment are converted into binary barcode images that meet the ECC200 standard, effectively repairing the defects and noise in the barcode image, enhancing the edge and module features of the barcode, making it easier to be accurately read and decoded by the barcode recognition equipment, thereby significantly improving the recognition rate of the barcode, reducing the recognition failure caused by the barcode quality problem, and ensuring the smooth application of the barcode in the production process. On the other hand, by adopting an end-to-end learning architecture, low-quality barcode images are directly input into the model. After one forward propagation calculation, a binary barcode image that meets the standards can be output. Compared with the traditional multi-stage barcode image processing method, it avoids multiple steps such as tedious preprocessing, feature extraction, and post-processing, reduces the data transmission and conversion time between different processing links, and improves the efficiency and real-time performance of the entire barcode image processing process. It can meet the requirements of rapid barcode recognition and processing on high-speed production lines and improve overall production efficiency.

[0044] The training set is used to perform model training, and the training end condition of the barcode enhancement model is obtained. For example, the training ends when the performance of the discriminator model and the generator model reaches a balance (such as the accuracy of the discriminator model is close to 90%); another example is that the training ends when the number of training times (epoch) reaches a preset threshold (such as 10 times, 100 times, etc.); another example is that the training ends when all the data in the training set completes one (1epoch) or multiple times (such as 3epochs, 10epochs, etc.).

[0045] As an example, the training framework uses pytorch when training the model, and the generator learning rate , the batch size is 64, the optimizer uses Adam, and the discriminator learning rate , the batch size is 64, the optimizer uses SGD, and adopts an alternating training strategy, first training the discriminator and then training the generator, continuously improving the capabilities of the two, and finally making the generated DM barcode image close to the ECC200 standard binary barcode.

[0046] In some embodiments, the method of acquiring the sample matrix barcode image includes: An industrial camera is used to collect original barcode images printed by thermal, inkjet and laser engraving in an industrial environment, and data enhancement is performed on them through a data enhancement strategy. The enhanced images are used as sample matrix barcode images; each of the matrix barcode images includes a regular barcode image and an abnormal feature barcode image, and the abnormal features of the abnormal feature barcode image include one or more of the following combinations: shape distortion, foreign body occlusion, dirt, uneven brightness, scratches and blur.

[0047] Therefore, during the model training process, starting from the specific application scenarios, barcode images covering commonly used printing methods and abnormal features were used, so that the trained barcode enhancement model has a strong generalization ability. Therefore, whether it is on different production lines or under different lighting, temperature, humidity and other environmental conditions, the barcode images can be stably output with high-quality enhanced barcode images, ensuring the reliability of barcode processing.

[0048] In specific applications, the above data enhancement strategy can be one of the online data enhancement strategy and the offline data enhancement strategy, or a combination thereof. When the online data enhancement strategy is selected, the data loader will dynamically generate enhanced samples during each training instead of using pre-processed data, reducing storage overhead. When the offline data enhancement strategy is selected, the training efficiency can be improved. When the combination of the two is selected, offline enhancement can generate a large number of fixed enhanced samples and expand the size of the data set; online enhancement can introduce randomness in the training process and further improve the diversity of the data.

[0049] As an example, offline enhancement is first performed to the original barcode image, such as adding Gaussian noise, simulating blur effects, generating images under different lighting conditions, etc. The enhanced image is saved to disk as an expanded training set. During the training process, the offline enhanced image is further subjected to random occlusion, random brightness adjustment and other operations. These operations are generated in real time during each training, reducing storage usage.

[0050] As an example, six groups of images (a, b, c, d, e, f) are provided, group a is an image generated by adding noise to the original barcode image; group b is an image generated by dilating the original barcode image; group c is an image generated by corroding the original barcode image; group d is an image generated by blurring the original barcode image; group e is an image generated by superimposing an occlusion on the original barcode image; group f is an image generated by randomly adjusting the brightness of the original barcode image. Taking the above images as the original barcode images, the present application can enhance the above types of original barcode images by adding noise, dilating, corroding, blurring, superimposing occlusions and randomly adjusting the brightness, and obtain the corresponding enhanced images.

[0051] In some embodiments, the data enhancement strategy includes: applying at least one interference operation to the sample matrix barcode image, the interference operation including noise injection, morphological defect simulation, blur processing, occlusion superposition and illumination adjustment; When the interference operation is noise injection, Gaussian noise and salt and pepper noise are added to the original barcode image to simulate the imaging sensor noise and environmental interference; salt and pepper noise refers to the appearance of white or black pixels at random positions in the image, and Gaussian noise refers to random noise that follows Gaussian distribution; When the interference operation is a morphological defect simulation, the barcode redundancy or missing defects in barcode printing are simulated by dilation or corrosion operation; When the interference operation is blur scene simulation, blur kernel is used to simulate the blur of the barcode caused by device shaking or rapid movement during scanning; for example, Gaussian blur kernel is used to simulate image blur caused by camera inaccurate focus or optical problems, or mean blur kernel is used to blur local areas in the image. When the interference operation is occlusion and dirt simulation, occlusions of different shapes, sizes and transparencies are randomly superimposed on the barcode image to simulate the situation where part of the barcode information is lost due to stains, scratches or external objects; When the interference operation is a simulation of illumination conditions, the change of the barcode brightness is simulated by adjusting the overall brightness value of the barcode image, the contrast is adjusted by histogram stretching or local contrast enhancement, and the brightness distribution of the image is adjusted by gamma correction of nonlinear transformation.

[0052] For noise injection, Gaussian noise and salt and pepper noise are added to the original barcode image to simulate imaging sensor noise and environmental interference. Gaussian noise increases the random disturbance of the image by introducing a random deviation that conforms to the normal distribution to each pixel value of the image; salt and pepper noise simulates serious errors caused by sensor failure or external factors by randomly selecting some pixels of the barcode image and setting their values ​​to the maximum or minimum value. In the process of noise generation, by limiting the amplitude and distribution ratio of the noise, the model's recognition ability for unclear or disturbed barcode images can be improved.

[0053] For morphological defect simulation, the expansion or corrosion operation is used to simulate the quality defects of barcode redundancy or missing caused by ink diffusion or insufficient ink in barcode printing, and different degrees of laser engraving. It can be considered that the expansion operation can make the barcode lines thicker, thereby simulating the phenomenon of overprinting; the corrosion operation makes the barcode lines thinner, simulating the phenomenon of underprinting. In specific applications, by selecting the shape and size of the structural elements and controlling the number of expansion or corrosion operations, the changes in the barcode morphology can be effectively simulated. In addition, the opening operation (corrosion first and then expansion) and the closing operation (expansion first and then corrosion) can be combined to remove redundant noise and fill small holes in the barcode, thereby enhancing the robustness of the model to non-standard barcodes.

[0054] For blurry scene simulation, blur kernel is used to simulate the motion blur effect caused by device shaking or fast movement during scanning of barcodes, and the blur direction and intensity are controlled to reproduce the blur effect under different motion states in actual scenes. For example, Gaussian blur is used to simulate image blur caused by camera focus inaccuracy or optical problems, which can reduce the edge clarity of barcodes and improve the model's enhancement ability for low-quality barcodes. Mean blur operation is used to process local areas in the image, further enhancing the robustness of the model, so that the model can better adapt to blurry barcode images in actual production and improve its enhancement ability. Blur kernel is a two-dimensional matrix used for convolution operation, which achieves blur effect by weighted summation with the neighborhood of image pixels.

[0055] For occlusion and dirt simulation, occluders of different shapes, sizes and transparencies are randomly superimposed on the barcode image to simulate the situation where part of the barcode information is lost due to stains, scratches or external objects. The shape and position of the occluder are randomly generated, and at the same time, they are constrained according to the structural characteristics of the barcode to avoid excessive occlusion of key information areas. The adjustment of transparency makes the occlusion effect present different degrees, thereby simulating the intensity of stains in different environments, which can improve the model's ability to recover when part of the barcode is missing or damaged, so that it can still maintain a high morphological restoration ability in the case of information loss.

[0056] For illumination condition simulation, the overall brightness value of the barcode image is adjusted to simulate the change of the barcode brightness, and the enhancement ability of the model under different illumination intensities is enhanced. The contrast is adjusted by histogram stretching or local contrast enhancement (CLAHE) to improve the barcode recognition effect in low-contrast environments; the brightness distribution of the image is adjusted by gamma correction of nonlinear transformation to enhance the details of the dark or highlight parts, and improve the enhancement effect of the barcode under extreme illumination conditions.

[0057] Therefore, by simulating various adverse conditions that may occur in industrial environments, the adaptability and robustness of the model to barcode images of different qualities and environments are enhanced, and the model's ability to process various complex barcode images in practical applications is improved. The diversified training samples generated by the data enhancement strategy can help the model better learn the essential characteristics of barcode images, rather than just being limited to specific imaging conditions or defect types, so that the trained model has stronger generalization capabilities and can be applied to barcode image enhancement tasks in different industrial scenarios. In actual industrial environments, collecting high-quality barcode image data covering various defects and interference conditions often requires a lot of time and resource investment. Through data enhancement strategies, a large amount of simulated data with different characteristics can be generated on the existing basic data set, effectively expanding the scale of the data set and reducing dependence on large-scale real data collection, thereby reducing data collection costs, improving R&D efficiency, and accelerating model training and optimization processes.

[0058] In some embodiments, the method of obtaining the annotation data includes: In response to the marking instructions for the three vertices of the barcode area of ​​the sample matrix barcode image, the position and orientation of the barcode are determined; a binary barcode that complies with the ECC200 standard is generated in the barcode area according to the decoded plain text information or interactive input information and used as marking data.

[0059] In specific applications, users use annotation tools or software to mark the positions of three adjacent vertices in the barcode area of ​​the sample matrix barcode image. The coordinate information of the three vertices can uniquely determine the position and orientation of the barcode in the image, providing accurate geometric parameters for subsequent barcode area positioning and processing. Based on the coordinates of the three marked vertices, geometric calculation methods, such as the minimum circumscribed rectangle algorithm of a triangle, are used to calculate the precise boundary and direction information of the barcode area, and separate the barcode from the complex background image.

[0060] The collected sample matrix barcode image can be decoded by a decoding algorithm to obtain the plaintext information stored therein. On the other hand, the user can also manually input the plaintext information related to the barcode through interactive input.

[0061] In the determined barcode area, based on the obtained plaintext information, according to the encoding rules and image generation specifications of the ECC200 standard, a corresponding binary barcode image is generated as the annotation data.

[0062] Therefore, a binary barcode that complies with the ECC200 standard is generated as the annotation data based on the decoded plaintext information or the user's interactive input information, ensuring the standard consistency of the annotation data with the barcode in actual industrial applications. The above-mentioned standard-based annotation method enables the model to learn accurate barcode features and specifications during the training process, which helps to reduce the risk of model performance degradation due to non-standard annotation data.

[0063] The method for obtaining the annotation data may also include: inputting the sample matrix barcode image into the regional positioning model to obtain positioning data for determining the barcode area; and extracting a binary barcode that complies with the ECC200 standard from the barcode area in combination with a barcode decoding algorithm and using it as the annotation data.

[0064] The sample matrix barcode image is input into the regional positioning model. The regional positioning model detects the position of the barcode in the image through a convolutional neural network (CNN), and generates the positioning data of the barcode area. It can be considered that the positioning data of the barcode area includes the position and size information of the barcode. According to the positioning data of the barcode area, the image of the barcode area is extracted from the sample matrix barcode image. Then, the barcode decoding algorithm is applied to decode the extracted barcode area image to generate a binary barcode image that conforms to the ECC200 standard, and the binary barcode image that conforms to the ECC200 standard is output as the annotation data. The barcode decoding algorithm is, for example, decoding using the libdmtx barcode decoding library.

[0065] As an example, the training process of the region positioning model includes: A regional positioning training set is obtained, wherein the regional positioning training set includes a plurality of training data, each of the training data includes a sample matrix barcode image and annotated data of the positioning data of the sample matrix barcode image; for each training data in the regional positioning training set, the following processing is performed: the sample matrix barcode image in the training data is input into a preset deep learning model to obtain prediction data including positioning data; based on the prediction data and the annotated data, the model parameters of the deep learning model are updated; it is detected whether a preset training end condition is met; if so, the trained deep learning model is used as the regional positioning model; if not, the deep learning model is continuously trained using the next training data.

[0066] In some embodiments, the generator model adopts a UNet structure, and the generator model includes: The encoder includes multiple convolutional layers and maximum pooling layers, which are used to gradually refine and abstract the features of the input image by stacking them layer by layer. A decoder, which uses deconvolution and upsampling techniques to restore features to their original resolution; Skip connections are used to build direct paths between the encoder and decoder, so that low-level details can be integrated into high-level features; The final output layer is used to generate a standardized grayscale barcode image through a sigmoid activation function.

[0067] The encoder is the first part of the generator model, which contains multiple convolutional layers and maximum pooling layers. The function of the convolutional layer is to extract and transform the features of the input image. The convolution operation is performed by sliding the convolution kernel on the image to extract local features of the image such as edges and textures. The maximum pooling layer is used to downsample the feature map output by the convolutional layer, reduce the size of the feature map, reduce the computational complexity, and enhance the model's invariance to changes such as image translation. In the encoder, by stacking layer by layer, as the network depth increases, the size of the feature map gradually decreases, while the number of feature channels gradually increases, thereby realizing the gradual refinement and abstraction of the input image features, and gradually extracting high-level complex semantic features from low-level simple features.

[0068] The decoder is the second part of the generator model, which is used to gradually restore the high-level features extracted by the encoder back to the original resolution. It uses deconvolution (i.e., transposed convolution) and upsampling techniques. The deconvolution layer can expand the size of the feature map, and the upsampling technology further improves the resolution of the feature map and restores it to the same or similar size as the input image. In the decoder, as the network depth decreases, the size of the feature map gradually increases, the number of feature channels gradually decreases, and finally outputs a feature map of the same size as the input image.

[0069] The jump connection builds a direct path between the encoder and the decoder. Specifically, the feature map of a certain level in the encoder is directly connected to the feature map of the corresponding level in the decoder, so that the low-level details can be integrated into the high-level features. In this way, in the process of restoring the feature map by the decoder, not only the high-level semantic information can be used, but also the low-level details such as the edges and textures of the image can be retained, which helps to generate more accurate and detailed output images.

[0070] The final output layer is located at the end of the decoder. Its function is to perform the final processing on the feature map output by the decoder to generate an output image that meets the requirements. The final output layer uses the sigmoid activation function to map each pixel value of the feature map to the range of 0 to 1 to generate a standardized grayscale barcode image. The output of the sigmoid function can represent the probability that the pixel belongs to a barcode module. By setting a threshold (such as 0.5), the grayscale image can be converted into a binary barcode image, where pixel values ​​greater than the threshold represent barcode modules, and pixels less than the threshold represent background.

[0071] Therefore, compared with the traditional model that only relies on high-level semantic features for generation, the technical solution of this embodiment, due to the jump connection in the UNet structure, the low-level detail features can be directly passed to the decoder part. When generating the final barcode image, the edge, line and other detail information of the barcode can be better retained. In other words, the UNet structure can generate clearer and more accurate barcode images, reducing decoding errors caused by loss of details.

[0072] In some embodiments, the discriminator model adopts a ResNet18 structure, and the discriminator model includes: Multiple residual modules for feature extraction via convolutional kernels and data normalization via batch normalization techniques; The global average pooling layer is used to connect the residual module through skip connections and perform global average pooling operations on the extracted features; The output layer is used to connect the output end of the global average pooling layer and output the predicted value of the probability of whether the input image is a real barcode image through the activation function.

[0073] The discriminator model contains multiple residual modules, each of which performs feature extraction operations by convolution kernels and performs data normalization through batch normalization technology. The convolution operation is used to extract local features of the input image, such as edges and textures, while batch normalization standardizes the output of the convolution layer so that the data has zero mean and unit variance, which accelerates the convergence speed of the model and improves the stability of the model.

[0074] After the residual module, they are connected through skip connections and the extracted features are subjected to global average pooling. The global average pooling layer globally averages the feature maps of each feature channel and compresses the two-dimensional feature map of each feature channel into a single value representing the average response of the feature channel over the entire image.

[0075] The output layer is connected to the output of the global average pooling layer, and outputs a prediction value through an activation function (sigmoid activation function), which is used to indicate the probability of whether the input image is a real barcode image. The sigmoid function limits the output value to between 0 and 1, which can be interpreted as the probability that the input image is a real barcode image. For example, an output value close to 1 indicates that the model believes that the probability that the input image is a real barcode image is high, while an output value close to 0 indicates that the discriminator model believes that the probability that the input image (of the generator model) is a generated barcode image is high.

[0076] Therefore, through the multiple residual modules in the ResNet18 structure, the high-level semantic features of the input image can be extracted layer by layer, and the low-level features such as edges and textures can be gradually abstracted to the high-level barcode structure and pattern features. The ability of layer-by-layer feature extraction and abstraction enables the discriminator model to more accurately capture the difference between the real barcode image and the generated barcode image, thereby improving the ability to distinguish the authenticity of the input image and providing more effective feedback for the generator model. The application of the global average pooling layer not only reduces the number of parameters and complexity of the model, but also enhances the adaptability of the model to different input images.

[0077] As an example, the multiple residual modules of the discriminator use 3×3 convolution kernels to perform feature extraction operations, and then use Batch Normalization technology to normalize the data. Skip connections are used to solve the problem of gradient vanishing or gradient exploding, thereby enhancing the stability of the model during training. A global average pooling layer is set to effectively reduce model parameters while enhancing the generalization ability of the model. The sigmoid prediction value is output to determine whether the input image is a real barcode image.

[0078] In some embodiments, the generator loss function is composed of an adversarial loss function, an image restoration loss function, a feature loss function, and a structural similarity loss function, and is weighted and summed using preset weights.

[0079] Therefore, by comprehensively considering adversarial loss, image restoration loss, feature loss and structural similarity loss, different loss functions constrain and optimize the generator from different angles, so that the model can learn a more comprehensive and richer image feature representation. Adversarial loss helps the model learn data distribution and improves its adaptability to different barcode image styles and types; image restoration loss ensures the accuracy of the model in pixel-level restoration and adapts to barcode images under various imaging conditions; feature loss enables the model to understand the essential characteristics of barcode images at the semantic level and can handle input images of different complexity and quality; structural similarity loss strengthens the model's attention to the details of the barcode structure, so that it can maintain the standardization and integrity of the barcode during the generation process. The comprehensive optimization method enhances the generalization ability of the model, so that it can stably output high-quality enhancement results when facing different barcode images in various industrial environments. The weighted summation method of preset weights provides flexibility for the adjustment and optimization of the model. The weights of different loss functions can be adjusted according to actual application requirements and data characteristics to highlight or balance certain specific aspects of performance. For example, in application scenarios that require extremely high barcode boundary clarity, the weight of the structural similarity loss can be appropriately increased; in cases where it is necessary to focus on the overall realism of the generated image, the weight of the adversarial loss can be increased. The above flexibility enables the model to better adapt to different practical needs and industrial environments. The performance of the generator model can be optimized and customized through simple weight adjustments without large-scale modification and retraining of the model structure.

[0080] As an example, the adversarial loss function , using the adversarial training mechanism of GAN, improve the authenticity of the generated image and make it as close to the real barcode image as possible, which is defined as follows:

[0081] x is a random variable that follows the real data distribution D(G(x)) is the output probability of the discriminator for the generated image G(z), which means the probability that the discriminator believes that the generated image is a real barcode (in the range [0,1]). logD(G(x)) is the probability that the generator hopes to maximize the discriminator's judgment on the generated image. Image restoration loss , calculate the L1 distance between the generated image and the target image to ensure the accuracy of pixel-level restoration, which is defined as follows:

[0082] It is a binary barcode image that complies with the ECC200 standard.

[0083] Feature loss function , by calculating the difference in the features of the intermediate layer of the discriminator, the semantic consistency of the generated image is improved, which is defined as follows:

[0084] D (i) is the intermediate feature extraction part of the i-th layer of the discriminator, G(x gt ) represents the generator G for image x gt The processing result, G(x gt ) is the generator G that converts x gt Output as input.

[0085] Structural Similarity Loss Function , by optimizing the boundary clarity of the barcode module to make it easier to decode, defined as follows:

[0086] Among them, SSIM is the structural similarity index.

[0087] The final weighted loss of the generator loss function The definition is as follows:

[0088] , are their respective weights. The weight parameters are determined through experiments. The weight of adversarial loss is set to 1.0 to maintain the dynamic balance between the generator and the discriminator. The weight of pixel-level restoration loss is 10.0. Since it has the greatest impact on the generated quality, it is given the highest weight to improve the generated quality. The weight of feature loss is moderate and is set to 1.0 to ensure semantic consistency. The weight of structural loss is 5.0. Since it affects the boundary quality, it is given a relatively high weight.

[0089] As an example, the discriminator loss function uses the binary cross entropy loss function to calculate the ECC200 standard binary barcode label image and generate the standard barcode image loss, and applies the label smoothing technology to smooth the label of the real image. Adjust from 1.0 to 0.9 to generate the image label Adjusted from 0 to 0.1 to reduce the overconfidence problem of the discriminator and alleviate the mode collapse problem. The total loss function of the discriminator The definition is as follows:

[0090] is the adversarial loss value of the discriminator, which represents the classification error of the discriminator for real images and generated images. is the label of the real image, is the label of the generated image. D(x) is the discriminator's label of the real image. x The output ofx is the probability of a real image. D(G(z)) is the output of the discriminator for the generated image G(z), indicating the probability that the discriminator believes that G(z) is a real image.

[0091] In some embodiments, the method further comprises: post-processing the standard binary barcode to obtain a target binary barcode; the post-processing comprises the following steps: For the standard binary barcode, an adaptive threshold binarization method is used to calculate the threshold adapted to the brightness of different regions according to the local window to separate the barcode module from the background; corresponding to the following formula:

[0092] Among them, G bin (x, y) is the pixel value after binarization, G (x, y) is the pixel value of the generated barcode image, and T (x, y) represents the dynamic threshold, which is calculated by the adaptive threshold binarization method and is used to distinguish the barcode module from the background. The threshold is calculated based on the pixel value distribution in the local window to adapt to the brightness changes in different areas of the image.

[0093] As an example, the adaptive threshold binarization method defines a local window around each pixel of the image and calculates the statistical characteristics of the pixel value in the window (such as mean, median, etc.) to determine the threshold T of the pixel. This method can automatically adapt to the brightness changes in different areas of the image to ensure accurate separation of the barcode module and the background even in the case of uneven lighting. During the calculation, for each pixel (x, y) in the image, its gray value G(x, y) can be compared with the dynamic threshold T(x, y).

[0094] Morphological opening operation is used to eliminate isolated noise points and connect broken boundaries to obtain the target binary barcode. For example, a 3×3 kernel is used for morphological opening operation (first erosion and then expansion) to eliminate the noise points generated during the generation process and connect broken boundaries to optimize the barcode quality and improve the reading rate.

[0095] Therefore, through the post-processing steps of adaptive threshold binarization and morphological opening operation, the quality of the standard binary barcode image output by the generator is further improved, making it more in line with the ECC200 standard and further reducing the decoding failure rate.

[0096] In order to verify the effectiveness of the present invention, the technical solution (steps S101 to S103) proposed in this application is experimentally compared with the traditional imaging method using the same data set.

[0097] As a comparative example, the traditional imaging method has the following steps: Image preprocessing: convert color images into grayscale images to reduce data volume and increase processing speed; convert grayscale images into black and white images to make it easier to identify the bars and spaces of the barcode. Use filter methods to remove noise from the image and improve image quality.

[0098] Barcode region detection: detect edges in the image and determine the boundaries of the barcode. Use morphological operations such as dilation and erosion to enhance the features of the barcode region. Segment the image into multiple regions and find the region containing the barcode.

[0099] Decode, measure the width of the bars and spaces in the barcode, and determine the barcode encoding based on the width information. Match the detected barcode pattern with the predefined barcode template to determine the type and encoding of the barcode. Extract the key features of the barcode, such as the order of bars and spaces, start and end characters, etc. Convert the recognized barcode pattern into the corresponding digital or character encoding.

[0100] However, the above-mentioned image method requires manual design of appropriate morphological operations and parameters according to the specific barcode image characteristics and application scenarios, which requires high professional knowledge and experience. For the complex barcode image defects or diverse application scenarios mentioned in this application, compared with the technical solution mentioned in this application, it has limitations in processing complex scenarios and adaptability.

[0101] This application can process abnormal (low-quality) DM barcode images in various actual industrial scenarios, such as DM barcode images with scratches or irregular etching (there are several blocks with irregular shapes and small areas in the clock pattern area), barcodes with irregular printing and overexposure of light sources, barcodes with scratches and the barcode area is similar to the background features, dot-type barcodes with scratches and the barcode area is similar to the background features, barcodes with breaks in the alignment pattern (such as the lower left corner area), dirty barcodes, barcodes with large differences in shape from the ECC200 standard barcode, barcodes with unclear silk-screen printing and a certain degree of blur, and barcodes with added noise.

[0102] A comparative experiment was conducted on a self-built real industrial barcode dataset (5,000 images of abnormal DM barcode images in various actual industrial scenes, including 70% training set, 15% validation set, and 15% test set). The results are shown in Table 1.

[0103] Table 1 Results comparison table:

[0104] The experimental results are analyzed as follows: In terms of quantitative indicators, the present invention is significantly better than the comparison method (traditional image method) in SSIM (0.889), proving its advantages in pixel-level reconstruction and structure preservation. In terms of decoding performance, the decoding success rate of the method provided by the present invention is increased to 99.8%, which is 10.7 percentage points higher than that of the comparison method, indicating that the generated image is more in line with the ECC200 standard.

[0105] In specific applications, this system is deployed on the enterprise side, and the hardware configuration is Intel i7 8700k processor + NVIDIA1660ti graphics card. The actual production line deployment test process is as follows: Real-time data collection step: The barcode image is captured by a 12-megapixel camera at 9fps and transmitted to the processing terminal via Gigabit Ethernet.

[0106] Deployment test steps. The collected barcode data is passed into the trained generator model, supporting batch processing, with an average single-sheet processing time of ≤10ms.

[0107] Result feedback step: The decoding engine (OpencvEvision / ZBar) decodes the enhanced image. If the decoding fails, an alarm is triggered and the original data is saved for model iteration and optimization.

[0108] The above process was run continuously for 30 days, and the results before and after the import were compared as follows: The average decoding rate increased from 90.1% before deployment to 98.9% (30-day comparison data); the system delay end-to-end processing delay is ≤20ms, meeting the real-time requirements of the production line.

[0109] This application effectively solves the problem of low recognition rate caused by poor quality of industrial DM barcodes by constructing a generative adversarial network based on U-Net and ResNet18, combined with refined post-processing. Experiments have shown that this method is significantly superior to related technologies in terms of quantitative indicators, decoding performance and real-time performance, providing a reliable barcode processing solution for industrial automation.

[0110] System Embodiments.

[0111] The embodiment of the present application provides a barcode enhancement image processing system based on a generative adversarial network. Its specific implementation method is consistent with the implementation method and the technical effects achieved in the above-mentioned method implementation method, and some contents will not be repeated here.

[0112] See also Figure 3 , Figure 3 Schematic diagram of a module of a barcode enhancement image processing system provided in an embodiment of the present application. The system comprises: An information acquisition module is used to acquire a training set and an initial model; the training set includes a plurality of training data, each of which includes a sample matrix barcode image and its corresponding annotation data; the initial model is an adversarial network constructed by a generator model and a discriminator model; A model training module, used to perform model training using the training set according to the initial model and the target loss function to obtain a barcode enhancement model; the target loss function includes a generator loss function using a weighted composite loss technology and a discriminator loss function using a binary cross entropy loss technology; The barcode acquisition module is used to use an industrial camera to collect a data matrix barcode image in a real environment and input it into the barcode enhancement model to obtain a standard binary barcode.

[0113] In some embodiments, the method of acquiring the sample matrix barcode image includes: An industrial camera is used to collect original barcode images printed by thermal, inkjet and laser engraving in an industrial environment, and data enhancement is performed on them through a data enhancement strategy. The enhanced images are used as sample matrix barcode images; each of the matrix barcode images includes a regular barcode image and an abnormal feature barcode image, and the abnormal features of the abnormal feature barcode image include one or more of the following combinations: shape distortion, foreign body occlusion, dirt, uneven brightness, scratches and blur.

[0114] In some embodiments, the data enhancement strategy includes: applying at least one interference operation to the sample matrix barcode image, the interference operation including noise injection, morphological defect simulation, blur processing, occlusion superposition and illumination adjustment; When the interference operation is noise injection, Gaussian noise and salt and pepper noise are added to the original barcode image to simulate the imaging sensor noise and environmental interference; When the interference operation is a morphological defect simulation, the barcode redundancy or missing defects in barcode printing are simulated by dilation or corrosion operation; When the interference operation is blur scene simulation, blur kernel is used to simulate the blurring of the barcode caused by device shaking or rapid movement during scanning; When the interference operation is occlusion and dirt simulation, occlusions of different shapes, sizes and transparencies are randomly superimposed on the barcode image to simulate the situation where part of the barcode information is lost due to stains, scratches or external objects; When the interference operation is a simulation of illumination conditions, the change of the barcode brightness is simulated by adjusting the overall brightness value of the barcode image, the contrast is adjusted by histogram stretching or local contrast enhancement, and the brightness distribution of the image is adjusted by gamma correction of nonlinear transformation.

[0115] In some embodiments, the method of obtaining the annotation data includes: In response to the labeling instructions for the three vertices of the barcode area of ​​the sample matrix barcode image, the position and orientation of the barcode are determined; a binary barcode that complies with the ECC200 standard is generated in the barcode area according to the decoded plain text information or the interactive input information and used as the labeling data; or, the sample matrix barcode image is input into the area positioning model to obtain the positioning data for determining the barcode area; and a binary barcode that complies with the ECC200 standard is extracted from the barcode area in combination with the barcode decoding algorithm and used as the labeling data.

[0116] In some embodiments, the generator model adopts a UNet structure, and the generator model includes: The encoder includes multiple convolutional layers and maximum pooling layers, which are used to gradually refine and abstract the features of the input image by stacking them layer by layer; A decoder that restores features to their original resolution using deconvolution and upsampling techniques; Skip connections are used to build direct paths between the encoder and decoder, so that low-level details can be integrated into high-level features; The final output layer is used to generate a standardized grayscale barcode image through a sigmoid activation function.

[0117] In some embodiments, the discriminator model adopts a ResNet18 structure, and the discriminator model includes: Multiple residual modules for performing feature extraction operations through convolution kernels and data normalization through batch normalization technology; The global average pooling layer is used to connect the residual module through skip connections and perform global average pooling operations on the extracted features; The output layer is used to connect the output end of the global average pooling layer and output the predicted value of the probability of whether the input image is a real barcode image through the activation function.

[0118] In some embodiments, the generator loss function is composed of an adversarial loss function, an image restoration loss function, a feature loss function, and a structural similarity loss function, and is weighted and summed using preset weights.

[0119] In some embodiments, the system further includes: a post-processing module, which is used to post-process the standard binary barcode to obtain a target binary barcode; the post-processing includes the following steps: For the standard binary barcode, an adaptive threshold binarization method is used to calculate the threshold adapted to the brightness of different regions according to the local window to separate the barcode module from the background; corresponding to the following formula:

[0120] Among them, Gbin (x, y) is the pixel value after binarization, G(x, y) is the pixel value of the generated barcode image, and T(x, y) is the pixel threshold; The morphological opening operation is used to eliminate isolated noise points and connect broken boundaries to obtain the target binary barcode.

[0121] Device Embodiments.

[0122] An embodiment of the present application also provides an electronic device, including one or more processors and a memory; one or more programs are stored in the memory and are configured so that the one or more processors execute any of the methods described in the method embodiments.

[0123] See also Figure 4 , Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown.

[0124] The electronic device may include, for example, at least one memory 11, at least one processor 12, and a bus 13 connecting different platform systems.

[0125] The memory 11 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 111 and / or a cache memory 112 , and may further include a read-only memory (ROM) 113 .

[0126] The memory 11 also stores a computer program, which can be executed by the processor 12 so that the processor 12 implements the steps of any of the above methods.

[0127] The memory 11 may also include a utility 114 having at least one program module 115, such program module 115 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0128] Accordingly, the processor 12 may execute the above-mentioned computer program, and may execute the utility 114 .

[0129] The processor 12 may be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0130] Bus 13 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures.

[0131] The electronic device may also communicate with one or more external devices such as a keyboard, a pointing device, a Bluetooth device, etc., and may also communicate with one or more devices that can interact with the electronic device, and / or communicate with any device (such as a router, a modem, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be performed through an input / output interface 14. In addition, the electronic device may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter 15. The network adapter 15 may communicate with other modules of the electronic device through the bus 13. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device in actual applications, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0132] It should be noted that, in the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b and c can be single or multiple. It is worth noting that "at least one" can also be interpreted as "one or more items".

[0133] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are configured to distinguish similar objects, and do not have to be configured to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0134] This application is explained from the perspectives of purpose of use, effectiveness, progress and novelty, and has met the functional enhancement and usage requirements emphasized by the Patent Law. The above description and drawings of this application are only the preferred embodiments of this application, and are not intended to limit this application. Therefore, all structures, devices, features, etc. that are similar or identical to this application, that is, all equivalent replacements or modifications made in accordance with the scope of the patent application of this application, should fall within the scope of protection of the patent application of this application.

Claims

1. A barcode enhancement image processing method based on generative adversarial network, characterized in that: The method comprises: Obtain a training set and an initial model; the training set includes a plurality of training data, each of which includes a sample matrix barcode image and its corresponding annotation data; the initial model is an adversarial network constructed by a generator model and a discriminator model; According to the initial model and the target loss function, the training set is used to perform model training to obtain a barcode enhancement model; the target loss function includes a generator loss function using a weighted composite loss technology and a discriminator loss function using a binary cross entropy loss technology; An industrial camera is used to collect a data matrix barcode image in a real environment and input it into the barcode enhancement model to obtain a standard binary barcode.

2. The barcode enhancement image processing method according to claim 1, characterized in that: The method of acquiring the sample matrix barcode image includes: An industrial camera is used to collect original barcode images printed by thermal, inkjet and laser engraving in an industrial environment, and data enhancement is performed on them through a data enhancement strategy. The enhanced images are used as sample matrix barcode images; each of the matrix barcode images includes a regular barcode image and an abnormal feature barcode image, and the abnormal features of the abnormal feature barcode image include one or more of the following combinations: shape distortion, foreign body occlusion, dirt, uneven brightness, scratches and blur.

3. The barcode enhancement image processing method according to claim 2, characterized in that: The data enhancement strategy includes: applying at least one interference operation to the sample matrix barcode image, the interference operation including noise injection, morphological defect simulation, blur processing, occlusion superposition and illumination adjustment; When the interference operation is noise injection, Gaussian noise and salt and pepper noise are added to the original barcode image to simulate the imaging sensor noise and environmental interference; When the interference operation is a morphological defect simulation, the barcode redundancy or missing defects in barcode printing are simulated by dilation or corrosion operation; When the interference operation is blur scene simulation, blur kernel is used to simulate the blurring of the barcode caused by device shaking or rapid movement during scanning; When the interference operation is occlusion and dirt simulation, occlusions of different shapes, sizes and transparencies are randomly superimposed on the barcode image to simulate the situation where part of the barcode information is lost due to stains, scratches or external objects; When the interference operation is a simulation of illumination conditions, the change of the barcode brightness is simulated by adjusting the overall brightness value of the barcode image, the contrast is adjusted by histogram stretching or local contrast enhancement, and the brightness distribution of the image is adjusted by gamma correction of nonlinear transformation.

4. The barcode enhancement image processing method according to claim 1, characterized in that: The method for obtaining the labeled data includes: In response to the labeling instructions for the three vertices of the barcode area of ​​the sample matrix barcode image, the position and orientation of the barcode are determined; a binary barcode that complies with the ECC200 standard is generated in the barcode area according to the decoded plain text information or the interactive input information and used as the labeling data; or, the sample matrix barcode image is input into the area positioning model to obtain the positioning data for determining the barcode area; and a binary barcode that complies with the ECC200 standard is extracted from the barcode area in combination with the barcode decoding algorithm and used as the labeling data.

5. The barcode enhancement image processing method according to claim 1, characterized in that: The generator model adopts a UNet structure, and the generator model includes: The encoder includes multiple convolutional layers and maximum pooling layers, which are used to gradually refine and abstract the features of the input image by stacking them layer by layer. A decoder, which uses deconvolution and upsampling techniques to restore features to their original resolution; Skip connections are used to build direct paths between the encoder and decoder, so that low-level details can be integrated into high-level features; The final output layer is used to generate a standardized grayscale barcode image through a sigmoid activation function.

6. The barcode enhancement image processing method according to claim 5, characterized in that: The discriminator model adopts the ResNet18 structure, and the discriminator model includes: Multiple residual modules for feature extraction via convolutional kernels and data normalization via batch normalization techniques; The global average pooling layer is used to connect the residual module through skip connections and perform global average pooling operations on the extracted features; The output layer is used to connect the output end of the global average pooling layer and output the predicted value of the probability of whether the input image is a real barcode image through the activation function.

7. The barcode enhancement image processing method according to claim 5, characterized in that: The generator loss function consists of an adversarial loss function, an image restoration loss function, a feature loss function, and a structural similarity loss function, and is weighted and summed using preset weights.

8. The barcode enhancement image processing method according to claim 1, characterized in that: The method further comprises: post-processing the standard binary barcode to obtain a target binary barcode; the post-processing comprises the following steps: For the standard binary barcode, an adaptive threshold binarization method is used to calculate the threshold adapted to the brightness of different regions according to the local window to separate the barcode module from the background; corresponding to the following formula: Among them, G bin (x, y) is the pixel value after binarization, G(x, y) is the pixel value of the generated barcode image, and T(x, y) is the pixel threshold; The morphological opening operation is used to eliminate isolated noise points and connect broken boundaries to obtain the target binary barcode.

9. A barcode enhanced image processing system based on generative adversarial network, characterized in that: The system comprises: An information acquisition module is used to acquire a training set and an initial model; the training set includes a plurality of training data, each of which includes a sample matrix barcode image and its corresponding annotation data; the initial model is an adversarial network constructed by a generator model and a discriminator model; A model training module, used to perform model training using the training set according to the initial model and the target loss function to obtain a barcode enhancement model; the target loss function includes a generator loss function using a weighted composite loss technology and a discriminator loss function using a binary cross entropy loss technology; The barcode acquisition module is used to use an industrial camera to collect a data matrix barcode image in a real environment and input it into the barcode enhancement model to obtain a standard binary barcode.

10. An electronic device, characterized in that: The method comprises one or more processors and a memory; one or more programs are stored in the memory and are configured so that the one or more processors execute the method according to any one of claims 1 to 8.

Images