A multi-code decoding method, information matching method and product information management system

Abstract

The application provides a multi-code decoding method, an information matching method and a product information management system, comprising: performing first processing on a product image to obtain a plurality of image blocks of a preset size; using an improved YOLOv8 multi-target detection model and a TensorRT inference acceleration technology to perform rapid detection on each image block; removing duplicate codes in all image blocks, cutting each remaining code from the image block to which the code belongs to obtain a code block; for each remaining code, when the confidence exceeds a preset threshold, a plurality of decoding engines are called in priority order to analyze the code block, and it is judged whether there is any decoding engine output decoding information, and if so, the decoding is successful. The application provides a multi-code decoding method, an information matching method and a product information management system to solve the problems of lagging management mode, high management cost, low management efficiency, asymmetric product information and management errors in existing product information management.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a multi-code decoding method, an information matching method, and a product information management system. Background Technology

[0002] Enterprises are accelerating their digital transformation, focusing on building digital factories and intelligent production lines. Especially at the workshop level, the efficient management of products and their information is the core of production operations and directly affects the company's operational efficiency and economic benefits.

[0003] However, most enterprises still face numerous problems in product information management. On the one hand, management methods are outdated, relying heavily on traditional methods such as manual recording and random sampling, lacking an efficient and systematic information collection mechanism. This makes it difficult to track product information in real time, resulting in high management costs and low efficiency. On the other hand, the uneven level of equipment automation further exacerbates the problems of product information asymmetry and management errors. Summary of the Invention

[0004] This invention provides a multi-code decoding method, an information matching method, and a product information management system to solve problems such as outdated management methods, high management costs, low management efficiency, product information asymmetry, and management errors in existing product information management.

[0005] In a first aspect, the present invention provides a multi-code decoding method, comprising:

[0006] The product image is subjected to a first processing to obtain multiple image blocks of preset size. The product image includes multiple codes. The first processing includes deblurring, HDR illumination restoration, compression, overlapping block segmentation, and edge filling.

[0007] An improved YOLOv8 multi-object detection model and TensorRT inference acceleration technology are used to quickly detect each image block, identify the code in each image block, and output the detection coordinates, type number, and confidence score of the code. The improved YOLOv8 multi-object detection model includes Mosaic data augmentation, a backbone network, a neck network, and a head network. The backbone network adopts the CSPDarknet architecture, Transformer, and CBAM modules. The neck network adopts the BiFPN structure and self-attention layer. The head network adopts the CIoU loss function. The improved YOLOv8 multi-object detection model is trained using the PyTorch framework.

[0008] Remove duplicate codes from all image blocks, crop each remaining code from its respective image block to obtain its code block, and perform a second processing on each code block, the second processing including grayscale conversion and binarization.

[0009] For each remaining code, when its confidence exceeds a preset threshold, multiple decoding engines are called in order of priority to parse its code block. It is determined whether any decoding engine outputs decoding information. If yes, the decoding is successful. If no, the resolution of its code block is adjusted to a preset size. The multiple decoding engines are called again in order of priority to parse the adjusted code block. It is then determined whether any decoding engine outputs information. If yes, the decoding is successful. If no, the decoding fails.

[0010] For each code that is successfully decoded, its bounding box and decoding information are marked on the product image to obtain a marked image;

[0011] Output the labeled image and the detection coordinates and decoding information of all successfully decoded codes.

[0012] Optionally, the code includes QR codes and barcodes.

[0013] Optionally, the deblurring is achieved using an improved DeblurGAN-v2 network. This improved network employs an improved loss function, which includes a structure-aware loss. The structure-aware loss comprises perceptual loss, edge gradient loss, and module scaling loss. The formula for the improved loss function is as follows:

[0014] L total =L adv +λ1L percep +λ2L edge +λ3L ratio

[0015] Among them, L adv =αL local +βL global To counteract the loss, α and β are used to balance the realism dimension of the generated image by the dual-scale discriminator in terms of local detail and global structure control, respectively. local For the loss of realism in local areas, L global For the realism loss of the global structure, λ1, λ2, and λ3 are used to quantize the constraint strength of the three physical structures: texture, edge, and scale, respectively. percep For the perceived loss, L edge For the edge gradient loss, L ratio This refers to the proportional loss of the module.

[0016] Optionally, the overlapping segmentation includes cutting the product image into image blocks according to the preset size, with adjacent image blocks having an overlapping area of ​​a preset width.

[0017] Optionally, the removal of duplicate codes from all image blocks includes:

[0018] The detection coordinates of each code are converted into its coordinates in the product image, and the intersection-over-union ratio (IoU) between each code and other codes is calculated after the conversion.

[0019] Determine whether there exists a case where the intersection-union ratio between two codes is greater than a first threshold. If so, discard the code with the smaller area among the two codes.

[0020] Optionally, the step of cropping each remaining code from its corresponding image block to obtain its code block includes:

[0021] Based on the detection coordinates of each remaining code, a preset number of pixels are extended outward as its code block size, and the code block is cropped from its respective image block according to the detection coordinates and code block size of each remaining code.

[0022] The above solution can efficiently and accurately identify multiple small-sized codes in parallel in ultra-high resolution images, suppress interference from complex working conditions, and be compatible with multiple code formats. It provides precise information collection and management support for industrial production and solves the problems of low efficiency, error-proneness, and repetitive labor in traditional manual scanning methods.

[0023] Secondly, the present invention provides an information matching method, comprising, according to any of the above methods:

[0024] Determine whether the intersection-union ratio (IUU) between the detection coordinates and the standard coordinates of each successfully decoded code exceeds a second threshold. If so, determine its bit number based on its standard coordinates. The relationship between the standard coordinates and the bit number is pre-configured.

[0025] Determine whether the number of all successfully decoded codes is equal to the preset number. If not, determine the unrecognized codes based on the labeled image, manually decode the unrecognized codes to obtain their decoding information, and manually determine the bit number of the unrecognized codes.

[0026] Output the labeled image, the detection coordinates of all successfully decoded codes, and the decoding information and bit number of all codes.

[0027] The above solution automatically binds the tag number to product information by detecting the coordinate and standard coordinate matching code and tag number, which facilitates tracking and management and improves production efficiency; decoding unidentified codes ensures the integrity and accuracy of product information and realizes a complete closed-loop collection of product information.

[0028] Thirdly, the present invention provides a product information management system, including a client and a server. The client includes a data acquisition module, a tag matching module, an information supplementation module, and an information output module. The server includes a first processing module, a multi-code detection module, a second processing module, a multi-code decoding module, an image annotation module, and a decoding output module, wherein:

[0029] The data acquisition module is used to acquire product images and send the product images to the server. The product images include multiple codes.

[0030] The first processing module is used to perform a first processing on the product image to obtain multiple image blocks of preset size. The first processing includes deblurring, HDR illumination restoration, compression, overlapping block segmentation, and edge filling.

[0031] The multi-code detection module is used to quickly detect each image block using an improved YOLOv8 multi-object detection model and TensorRT inference acceleration technology, identify the codes in each image block, and output the detection coordinates, type number, and confidence score of the codes. The improved YOLOv8 multi-object detection model includes Mosaic data augmentation, a backbone network, a neck network, and a head network. The backbone network adopts the CSPDarknet architecture, Transformer, and CBAM modules. The neck network adopts the BiFPN structure and self-attention layer. The head network adopts the CIoU loss function. The improved YOLOv8 multi-object detection model is trained using the PyTorch framework.

[0032] The second processing module is used to remove duplicate codes in all image blocks, crop each remaining code from its respective image block to obtain its code block, and perform a second processing on each code block, the second processing including grayscale conversion and binarization.

[0033] The multi-code decoding module is used to, for each remaining code, when its confidence exceeds a preset threshold, call multiple decoding engines in priority order to parse its code block, determine whether any decoding engine outputs decoding information, if yes, decoding is successful, if no, adjust the resolution of its code block to a preset size, and call the multiple decoding engines again in priority order to parse the adjusted code block, continue to determine whether any decoding engine outputs information, if yes, decoding is successful, if no, decoding fails;

[0034] The image annotation module is used to annotate the bounding box and decoding information on the product image for each code that is successfully decoded, thereby obtaining an annotated image.

[0035] The decoding output module is used to output the labeled image and the detection coordinates and decoding information of all successfully decoded codes to the client;

[0036] The bit number matching module is used to determine whether the intersection-union ratio between the detection coordinates and the standard coordinates of each successfully decoded code exceeds a second threshold. If so, the bit number is determined according to its standard coordinates. The relationship between the standard coordinates and the bit number is pre-configured.

[0037] The information supplementation module is used to determine whether the number of all successfully decoded codes is equal to a preset number. If not, it determines the unrecognized codes based on the labeled image, manually decodes the unrecognized codes to obtain their decoding information, and manually determines the bit number of the unrecognized codes.

[0038] The information output module is used to output the labeled image, the detection coordinates of all successfully decoded codes, and the decoding information and bit number of all codes.

[0039] The above solution constructs a product information collection and management system that integrates detection, decoding, and management. It realizes batch, non-contact, and non-interventional product information collection, forming a closed-loop system from image acquisition, information recognition, data verification to online management. It solves the problems of outdated management methods, high management costs, low management efficiency, product information asymmetry, and management errors in existing product information management, and provides manufacturing enterprises with a highly reliable information solution.

[0040] Fourthly, the present invention provides a computing device, comprising:

[0041] Memory, used to store program instructions;

[0042] The processor is configured to invoke program instructions stored in the memory and execute any of the methods described above according to the obtained program.

[0043] Fifthly, the present invention provides a computer-readable non-volatile storage medium including computer-readable instructions that, when read and executed by a computer, cause the computer to perform any of the methods described above. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a multi-code decoding method provided in an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of the architecture of the DeblurGAN-v2 network provided in an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the architecture of the improved YOLOv8 multi-target detection model and the original YOLOv8 model provided in the embodiments of the present invention;

[0047] Figure 4 This is a schematic diagram of the structure of a product information management system provided in an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0049] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] Figure 1 The flowchart of a multi-code decoding method provided by an embodiment of the present invention is shown, including:

[0051] S101, Perform the first processing on the product image to obtain multiple image blocks of preset sizes.

[0052] Specifically, the product image includes multiple codes.

[0053] In one example, the code includes both QR codes and barcodes.

[0054] Since the size of the image detected by YOLOv8 is fixed at 640×640, in one example, the preset size is 640×640 pixels.

[0055] Specifically, the first processing includes deblurring, HDR illumination restoration, compression, overlapping blocks, and edge filling to improve the quality of the image before recognition and the success rate of code recognition.

[0056] In the field of image deblurring, the DeblurGAN series of methods are based on generative adversarial networks and effectively recover motion-blurred images through structures such as residual blocks and feature pyramids. They are highly adaptable and have stable performance.

[0057] In one example, deblurring is achieved using an improved DeblurGAN-v2 network. This improved network employs an enhanced loss function, which incorporates a structure-aware perceptual loss. The structure-aware loss includes perceptual loss, edge gradient loss, and module scaling loss. The formula for the enhanced loss function is as follows:

[0058] L total =L adv +λ1L percep +λ2L edge +λ3L ratio

[0059] Among them, L adv =αL local +βL global To combat the loss, α and β are used to balance the dual-scale discriminator's control over the realism of the generated image in terms of local details and global structure, respectively. Increasing α makes the generator pay more attention to the pixel-level realism of local regions (such as individual QR code modules), avoiding blurring or noise within modules, but may lead to distortion of the global structure (such as the positional relationship between modules). Increasing β forces the generator to maintain the overall geometric structure of the QR code, such as the relative positions of positioning patterns and the regularity of module arrangement, but may lead to over-smoothing of local details (such as module edges). local For the loss of realism in local areas, L global For the realism loss of the global structure, λ1, λ2, and λ3 are used to quantize the constraint strength of the three physical structures of texture, edge, and scale, respectively, to preserve structured features. percep To perceive loss, L edge For edge gradient loss, L ratio This represents the module's proportional loss.

[0060] Figure 2 The architecture of the DeblurGAN-v2 network is shown. Based on the Generative Adversarial Network (GAN) framework, it integrates the Feature Pyramid Network (FPN) and the dual-scale discriminator structure, which significantly improves the ability to recover detailed edge information. The architecture of the DeblurGAN-v2 improved network is basically the same as that of the DeblurGAN-v2 network. The difference is that a structure-aware loss is added to the loss function, making the generated image more suitable for subsequent recognition.

[0061] The DeblurGAN-v2 improved network offers an image reconstruction strategy adapted for industrial environments, specifically addressing the issue of blurred barcodes caused by jitter, motion blur, and imaging degradation during industrial barcode scanning. This allows for fast and low-loss deblurring of images. Introducing the DeblurGAN-v2 improved network for blurred image restoration not only improves barcode clarity and edge discernibility but also further enhances the resolution of blurred images. When processing blurred industrial image datasets, the DeblurGAN-v2 improved network achieves approximately 18% higher restoration accuracy than traditional deblurring methods, increasing the barcode recognition rate in blurred images to over 85%, demonstrating both real-time performance and practicality.

[0062] In one example, HDR lighting restoration, overlap tiling, and edge padding are implemented using OpenCV.

[0063] HDR lighting restoration improves the recognizability of codes in scenarios with reflections, mirrored surfaces, and uneven lighting.

[0064] In one example, overlapping blocks include cutting the product image into image blocks of a preset size, with an overlapping area of ​​a preset width between adjacent image blocks.

[0065] Overlapping blocks are used to ensure the integrity of the code, avoiding the situation where the same code is divided into different image blocks and each image block does not contain the complete state of the code, which would lead to omissions and failures in recognition.

[0066] The preset width is determined based on the size of the QR code and barcode. In one example, since the QR code is smaller than 200×200 pixels, the preset width is 200 pixels. When the barcode is larger than 200×200 pixels, the preset width can be adjusted according to the size of the barcode.

[0067] For example, when the preset size is 640×640 pixels and the preset width is 200 pixels, the overlapping area between adjacent left and right image blocks is 200×640 pixels, and the overlapping area between adjacent top and bottom image blocks is 640×200 pixels.

[0068] When overlapping blocks, if the size of the edge image block is less than 640×640 pixels, edge padding is required, which can be done with white.

[0069] In one example, compression is achieved using the Deflate compression algorithm.

[0070] Image compression and overlapping blocks reduce memory consumption and speed up image processing, ensuring fast response and efficient processing of high-resolution (e.g., 10240×5120 pixels) industrial images.

[0071] S102. The improved YOLOv8 multi-target detection model and TensorRT inference acceleration technology are used to quickly detect each image block, identify the code in each image block, and output the detection coordinates, type number and confidence of the code.

[0072] Since its inception with YOLOv1, the YOLO series of algorithms has been widely applied in industrial image recognition scenarios due to its end-to-end architecture and high-speed detection capabilities. YOLOv4 introduced Mosaic enhancement and the CSPDarknet backbone network, achieving excellent results on public datasets such as COCO. YOLOv8 further integrates the Transformer architecture and attention mechanism, achieving a good balance between accuracy and speed, making it valuable for industrial deployment.

[0073] Specifically, the improved YOLOv8 multi-object detection model includes Mosaic data augmentation, a backbone network, a neck network, and a head network. The backbone network adopts the CSPDarknet architecture, Transformer, and CBAM modules. The neck network adopts the BiFPN structure and self-attention layer. The head network adopts the CIoU loss function. The improved YOLOv8 multi-object detection model is trained using the PyTorch framework.

[0074] Figure 3 The architecture comparison between the improved YOLOv8 multi-object detection model and the original YOLOv8 model is shown, with specific improvements including:

[0075] The backbone network is based on the CSPDarknet architecture, which consists of multiple CSP modules and convolutional layers. The Transformer can be embedded in the end stage of the backbone network. After extracting basic features in the CSPDarknet architecture, the global geometric structure of the code is modeled by the Transformer, and the feature saliency is enhanced by the CBAM module.

[0076] The neck network utilizes the BiFPN (Bidirectional Feature Pyramid Network) structure to perform multi-scale feature fusion to improve the detection accuracy of small targets. The self-attention layer is a lightweight design of Transformer, which is different from the full Transformer in the backbone network. Its purpose is to optimize the efficiency of multi-scale feature fusion.

[0077] The training robustness is improved by combining Mosaic data augmentation with the CIoU loss function in the head network.

[0078] The improved YOLOv8 multi-target detection model is suitable for multi-code detection in industrial scenarios, improving the accuracy of multi-code localization and classification (divided into QR codes and barcodes). It addresses the challenges of multi-code localization and classification in high-density and complex backgrounds, demonstrating significant cutting-edge technology and practicality.

[0079] To address the characteristics of high-resolution industrial images, such as a large number of small targets and blurred boundaries, an improved YOLOv8 target detection model was designed. The CBAM module and BiFPN structure were introduced to improve the detection accuracy and concurrent recognition capability of small-sized codes. It supports the accurate recognition of about 40 codes in product images with an accuracy rate of over 95%, ensuring the complete detection of high-density targets.

[0080] The improved YOLOv8 multi-object detection model is trained using the PyTorch framework and exported as an ONNX format model file for TensorRT inference acceleration technology to compile and optimize, thereby improving inference speed.

[0081] TensorRT inference acceleration technology is a high-performance deep learning inference optimization engine developed by NVIDIA, designed specifically for GPU platforms to accelerate the computation process. TensorRT inference acceleration technology enables high-performance inference in a GPU environment, achieving a total time of ≤5 seconds per image from image input to code recognition, ensuring rapid response and adapting to real-time industrial production environments.

[0082] The improved YOLOv8 multi-target detection model adopts the PyTorch framework and utilizes GPUs for parallel processing, which improves computational and processing efficiency, optimizes memory utilization, and greatly shortens the target detection time. Combined with the improved YOLOv8 multi-target detection model and TensorRT inference acceleration technology, it achieves efficient and accurate multi-code localization and classification.

[0083] It should be noted that there may be multiple codes identified in each image patch.

[0084] In one example, the detection coordinates include the coordinates of the top left and bottom right corners in each image patch.

[0085] In one example, type numbers are used to distinguish between QR codes and barcodes.

[0086] S103. Remove duplicate codes from all image blocks, cut out each remaining code from its image block to obtain its code block, and perform a second processing on each code block.

[0087] In one example, removing duplicate codes from all image patches includes:

[0088] The detection coordinates of each code are converted into its coordinates in the product image, and the intersection-union ratio (IUU) between each code and other codes is calculated after the conversion.

[0089] Determine if there exists a case where the intersection-union ratio between two codes is greater than the first threshold. If so, discard the code with the smaller area.

[0090] It should be noted that the intersection-union ratio is an indicator used to measure the degree of overlap between two rectangles, ranging from 0 to 1. The higher the value, the more overlap there is between the two rectangles.

[0091] In one example, the first threshold is 0.7.

[0092] In one example, each remaining code is cropped from its corresponding image block to obtain its code block, which includes:

[0093] Based on the detection coordinates of each remaining code, a preset number of pixels are extended outward as its code block size, and the code block is cropped from its respective image block according to the detection coordinates and code block size of each remaining code.

[0094] In one example, the preset pixel is 10 pixels, and expanding outward by 10 pixels means expanding outward by 10 pixels both horizontally and vertically based on the detection coordinates of each remaining code.

[0095] Specifically, the second processing includes grayscale conversion and binarization.

[0096] In one example, grayscale and binarization are implemented using OpenCV.

[0097] S104. For each remaining code, when its confidence exceeds a preset threshold, multiple decoding engines are called in order of priority to parse its code block. It is determined whether any decoding engine outputs decoding information. If yes, the decoding is successful. If no, the resolution of its code block is adjusted to a preset size. Multiple decoding engines are called again in order of priority to parse the adjusted code block. It is determined whether any decoding engine outputs information. If yes, the decoding is successful. If no, the decoding fails.

[0098] In one example, multiple decoding engines were used, including WeChatCV, ZBar, and Libdmtx.

[0099] Multiple decoding engines ensure compatibility with different QR codes (such as QR Code, DataMatrix, PDF417, etc.) and barcodes, improving fault tolerance and recognition success rate, and meeting the diverse needs of industrial production.

[0100] The decoded information is a string of characters in a specific form obtained through decoding, which represents product information. The first 8 digits represent the material code of the component, and the remaining part represents the serial number (SN) of the component.

[0101] During decoding, Python multithreading and CPU are used for parallel processing to achieve parallel computing, enabling multiple codes to be decoded simultaneously, reducing overall processing latency and shortening runtime.

[0102] S105. For each code that is successfully decoded, mark its bounding box and decoding information on the product image to obtain an annotated image.

[0103] In one example, the annotation is implemented using OpenCV's function cv2.rectangle and cv2.putText.

[0104] Based on the annotation information in the labeled image, you can more intuitively view the recognition status of the code in the product image, find the location of the missed code, and scan it in time.

[0105] S106. Output the labeled image and the detection coordinates and decoding information of all successfully decoded codes.

[0106] Because the JSON format is standardized, compatible, easy to parse, and more flexible, in one example, the detection coordinates and decoding information of the labeled image and all successfully decoded codes are output in JSON format.

[0107] Statistical analysis of actual test experiments shows that the multi-code recognition accuracy reaches ≥95%, the missed scan rate does not exceed 3%, and the multi-code false recognition rate is controlled at ≤3%, ensuring efficient and accurate recognition of multi-codes in complex industrial environments. In particular, considering different lighting conditions, background interference, and blurring effects, it can still output correct results stably.

[0108] The above solution can efficiently and accurately identify multiple small-sized codes in parallel in ultra-high resolution images, suppress interference from complex working conditions, and be compatible with multiple code formats. It provides precise information collection and management support for industrial production and solves the problems of low efficiency, error-proneness, and repetitive labor in traditional manual scanning methods.

[0109] Based on the same inventive concept, an information matching method is provided in this embodiment of the invention, comprising:

[0110] S201. Determine whether the intersection-union ratio between the detection coordinates and the standard coordinates of each successfully decoded code exceeds the second threshold. If so, determine its position number based on its standard coordinates.

[0111] Specifically, the relationship between standard coordinates and tag numbers is pre-configured.

[0112] The standard coordinates refer to the position of the QR code on the component, and the tag number refers to the location number of the component. Within the same batch of products, the position of the component is fixed, and a QR code (or barcode) is printed at a fixed position on the component.

[0113] By detecting the coordinates and matching the standard coordinates with the code and tag number, the tag number is automatically bound to the product information, which facilitates tracking and management and improves production efficiency.

[0114] S202. Determine whether the number of all successfully decoded codes is equal to the preset number. If not, determine the unrecognized codes based on the labeled image, manually decode the unrecognized codes to obtain their decoding information, and manually determine the bit number of the unrecognized codes.

[0115] The number of codes on a single side of a product of a fixed model is fixed, and this number is used to determine whether all codes on the product image have been identified.

[0116] Decoding unidentified codes ensures the integrity and accuracy of product information, achieving a complete closed-loop collection of product information.

[0117] In one example, the unrecognized code is manually decoded to obtain its decoding information using a barcode scanner.

[0118] S203. Output the labeled image, the detection coordinates of all successfully decoded codes, and the decoding information and bit number of all codes.

[0119] In one example, the labeled image, the detection coordinates of all successfully decoded codes, and the decoding information and bit numbers of all codes are output in JSON format for subsequent storage, analysis, and display.

[0120] Based on the same inventive concept. Figure 4 This invention illustrates the structure of a product information management system according to an embodiment of the present invention, including a client and a server. The client includes a data acquisition module, a tag matching module, an information supplementation scanning module, and an information output module. The server includes a first processing module, a multi-code detection module, a second processing module, a multi-code decoding module, an image annotation module, and a decoding output module, wherein:

[0121] The data acquisition module is used to acquire product images and send them to the server. The product images include multiple codes.

[0122] The first processing module is used to perform a first processing on the product image to obtain multiple image blocks of preset size. The first processing includes deblurring, HDR illumination restoration, compression, overlapping block segmentation, and edge filling.

[0123] The multi-code detection module is used to quickly detect each image block using an improved YOLOv8 multi-object detection model and TensorRT inference acceleration technology, identify the codes in each image block, and output the detection coordinates, type number, and confidence of the codes. The improved YOLOv8 multi-object detection model includes Mosaic data augmentation, a backbone network, a neck network, and a head network. The backbone network adopts the CSPDarknet architecture, Transformer, and CBAM modules. The neck network adopts the BiFPN structure and self-attention layer. The head network adopts the CIoU loss function. The improved YOLOv8 multi-object detection model is trained using the PyTorch framework.

[0124] The second processing module is used to remove duplicate codes in all image blocks, crop each remaining code from its respective image block to obtain its code block, and perform a second processing on each code block, which includes grayscale conversion and binarization.

[0125] The multi-code decoding module is used to, for each remaining code, call multiple decoding engines in priority order to parse its code block when its confidence exceeds a preset threshold, and determine whether any decoding engine outputs decoding information. If yes, the decoding is successful; if no, the resolution of its code block is adjusted to a preset size, and multiple decoding engines are called again in priority order to parse the adjusted code block. The module continues to determine whether any decoding engine outputs information. If yes, the decoding is successful; if no, the decoding fails.

[0126] The image annotation module is used to annotate the bounding box and decoding information on the product image for each successfully decoded code, resulting in an annotated image;

[0127] The decoding output module is used to output the labeled image and the detection coordinates and decoding information of all successfully decoded codes to the client;

[0128] The bit number matching module is used to determine whether the intersection-union ratio between the detection coordinates and the standard coordinates of each successfully decoded code exceeds the second threshold. If so, the bit number is determined according to its standard coordinates. The relationship between the standard coordinates and the bit number is pre-configured.

[0129] The information supplementation module is used to determine whether the number of all successfully decoded codes is equal to the preset number. If not, it determines the unrecognized codes based on the labeled image, manually decodes the unrecognized codes to obtain their decoding information, and manually determines the bit number of the unrecognized codes.

[0130] The information output module is used to output the labeled image, the detection coordinates of all successfully decoded codes, and the decoding information and bit number of all codes.

[0131] The server uses a combination of GPUs and CPUs. The GPU is responsible for performing the core computations for model inference, using parallel computing to accelerate deep learning tasks; the CPU is responsible for tasks such as data preprocessing, post-processing, and system scheduling.

[0132] In one example, product images are captured by taking photos with a camera and uploading local images.

[0133] When taking photos with a camera, if the image content is missing or severely interfered with, historical image caching technology can be used to repair the captured images, thereby improving decoding success rate and fault tolerance. Historical image caching technology stores recently processed high-quality image frames in memory, forming a time-series cache pool, for use in repairing subsequent missing or corrupted frames.

[0134] Because the Base64 format has better compatibility during transmission and can avoid errors caused by encoding issues, in one example, the product image and the annotation image are in Base64 format.

[0135] The system adopts a modular architecture. The client-side interface for image acquisition and information entry is developed using C#, while the server-side uses Python and a MySQL database for image decoding and data storage. This hybrid development architecture, combining C# and Python, enhances the system's cross-platform compatibility and deployment flexibility, ensuring long-term stable operation in practical use and meeting industrial-grade real-time requirements.

[0136] By interfaceing with the process data interface of the process platform, the system can automatically associate the identification results with information such as product flow, procedures, and equipment numbers, building a complete data loop and supporting traceability analysis by time, process, and product dimensions. This system improves product management efficiency by more than three times, providing traditional manufacturing enterprises with a feasible digital transformation technology path.

[0137] The above solution constructs a product information collection and management system that integrates detection, decoding, and management. It realizes batch, non-contact, and non-interventional product information collection, forming a closed-loop system from image acquisition, information recognition, data verification to online management. It solves the problems of outdated management methods, high management costs, low management efficiency, product information asymmetry, and management errors in existing product information management, and provides manufacturing enterprises with a highly reliable information solution.

[0138] Based on the same inventive concept, embodiments of the present invention provide a computing device, including:

[0139] Memory, used to store program instructions;

[0140] The processor is used to call program instructions stored in memory and execute the above methods according to the obtained program.

[0141] Based on the same inventive concept, embodiments of the present invention provide a computer-readable non-volatile storage medium, characterized in that it includes computer-readable instructions, which, when read and executed by a computer, cause the computer to perform the above-described method.

[0142] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0144] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0145] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0146] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0147] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A multi-code decoding method, characterized in that, include: The product image is subjected to a first processing to obtain multiple image blocks of preset size. The product image includes multiple codes. The first processing includes deblurring, HDR illumination restoration, compression, overlapping block segmentation, and edge filling. An improved YOLOv8 multi-object detection model and TensorRT inference acceleration technology are used to quickly detect each image block, identify the code in each image block, and output the detection coordinates, type number, and confidence score of the code. The improved YOLOv8 multi-object detection model includes Mosaic data augmentation, a backbone network, a neck network, and a head network. The backbone network adopts the CSPDarknet architecture, Transformer, and CBAM modules. The neck network adopts the BiFPN structure and self-attention layer. The head network adopts the CIoU loss function. The improved YOLOv8 multi-object detection model is trained using the PyTorch framework. Remove duplicate codes from all image blocks, crop each remaining code from its respective image block to obtain its code block, and perform a second processing on each code block, the second processing including grayscale conversion and binarization. For each remaining code, when its confidence exceeds a preset threshold, multiple decoding engines are called in order of priority to parse its code block. It is determined whether any decoding engine outputs decoding information. If yes, the decoding is successful. If no, the resolution of its code block is adjusted to a preset size. The multiple decoding engines are called again in order of priority to parse the adjusted code block. It is then determined whether any decoding engine outputs information. If yes, the decoding is successful. If no, the decoding fails. For each code that is successfully decoded, its bounding box and decoding information are marked on the product image to obtain a marked image; Output the labeled image and the detection coordinates and decoding information of all successfully decoded codes.

2. The method according to claim 1, characterized in that, The codes include QR codes and barcodes.

3. The method according to claim 1, characterized in that, The deblurring is achieved through an improved DeblurGAN-v2 network, which employs an improved loss function. This improved loss function incorporates a structure-aware loss, which includes perceptual loss, edge gradient loss, and module scaling loss. The formula for the improved loss function is as follows: L total =L adv +λ1L percep +λ2L edge +λ3L ratio Among them, L adv =αL local +βL global To counteract the loss, α and β are used to balance the realism dimension of the generated image by the dual-scale discriminator in terms of local detail and global structure control, respectively. local For the loss of realism in local areas, L global For the realism loss of the global structure, λ1, λ2, and λ3 are used to quantize the constraint strength of the three physical structures: texture, edge, and scale, respectively. percep For the perceived loss, L edge For the edge gradient loss, L ratio This refers to the proportional loss of the module.

4. The method according to claim 1, characterized in that, The overlapping segmentation includes cutting the product image into image blocks according to the preset size, with an overlapping area of ​​preset width between adjacent image blocks.

5. The method according to claim 1, characterized in that, The process of removing duplicate codes from all image blocks includes: The detection coordinates of each code are converted into its coordinates in the product image, and the intersection-over-union ratio (IoU) between each code and other codes is calculated after the conversion. Determine whether there exists a case where the intersection-union ratio between two codes is greater than a first threshold. If so, discard the code with the smaller area among the two codes.

6. The method according to claim 1, characterized in that, The step of cropping each remaining code from its corresponding image block to obtain its code block includes: Based on the detection coordinates of each remaining code, a preset number of pixels are extended outward as its code block size, and the code block is cropped from its respective image block according to the detection coordinates and code block size of each remaining code.

7. An information matching method, according to any one of claims 1-6, characterized in that, include: Determine whether the intersection-union ratio (IUU) between the detection coordinates and the standard coordinates of each successfully decoded code exceeds a second threshold. If so, determine its bit number based on its standard coordinates. The relationship between the standard coordinates and the bit number is pre-configured. Determine whether the number of all successfully decoded codes is equal to the preset number. If not, determine the unrecognized codes based on the labeled image, manually decode the unrecognized codes to obtain their decoding information, and manually determine the bit number of the unrecognized codes. Output the labeled image, the detection coordinates of all successfully decoded codes, and the decoding information and bit number of all codes.

8. A product information collection and management system, characterized in that, The system includes a client and a server. The client includes a data acquisition module, a tag matching module, an information supplementation module, and an information output module. The server includes a first processing module, a multi-code detection module, a second processing module, a multi-code decoding module, an image annotation module, and a decoding output module. The data acquisition module is used to acquire product images and send the product images to the server. The product images include multiple codes. The first processing module is used to perform a first processing on the product image to obtain multiple image blocks of preset size. The first processing includes deblurring, HDR illumination restoration, compression, overlapping block segmentation, and edge filling. The multi-code detection module is used to quickly detect each image block using an improved YOLOv8 multi-object detection model and TensorRT inference acceleration technology, identify the codes in each image block, and output the detection coordinates, type number, and confidence score of the codes. The improved YOLOv8 multi-object detection model includes Mosaic data augmentation, a backbone network, a neck network, and a head network. The backbone network adopts the CSPDarknet architecture, Transformer, and CBAM modules. The neck network adopts the BiFPN structure and self-attention layer. The head network adopts the CIoU loss function. The improved YOLOv8 multi-object detection model is trained using the PyTorch framework. The second processing module is used to remove duplicate codes in all image blocks, crop each remaining code from its respective image block to obtain its code block, and perform a second processing on each code block, the second processing including grayscale conversion and binarization. The multi-code decoding module is used to, for each remaining code, when its confidence exceeds a preset threshold, call multiple decoding engines in priority order to parse its code block, determine whether any decoding engine outputs decoding information, if yes, decoding is successful, if no, adjust the resolution of its code block to a preset size, and call the multiple decoding engines again in priority order to parse the adjusted code block, continue to determine whether any decoding engine outputs information, if yes, decoding is successful, if no, decoding fails; The image annotation module is used to annotate the bounding box and decoding information on the product image for each code that is successfully decoded, thereby obtaining an annotated image. The decoding output module is used to output the labeled image and the detection coordinates and decoding information of all successfully decoded codes to the client; The bit number matching module is used to determine whether the intersection-union ratio between the detection coordinates and the standard coordinates of each successfully decoded code exceeds a second threshold. If so, the bit number is determined according to its standard coordinates. The relationship between the standard coordinates and the bit number is pre-configured. The information supplementation module is used to determine whether the number of all successfully decoded codes is equal to a preset number. If not, it determines the unrecognized codes based on the labeled image, manually decodes the unrecognized codes to obtain their decoding information, and manually determines the bit number of the unrecognized codes. The information output module is used to output the labeled image, the detection coordinates of all successfully decoded codes, and the decoding information and bit number of all codes.

9. A computing device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1-6 according to the obtained program.

10. A computer-readable non-volatile storage medium, characterized in that, It includes computer-readable instructions that, when read and executed by a computer, cause the computer to perform the method as described in any one of claims 1-6.

Images