Intelligent two-dimensional code quick scanning tallying method for steel coil warehouse

Abstract

The invention provides an intelligent steel coil warehouse two-dimensional code quick scanning tallying method, and belongs to the technical field of label image recognition. Comprising the following steps that S1, primary scanning units are arranged on the two sides of a driving route of a transport vehicle correspondingly and used for capturing first images of the two sides of a steel coil on the vehicle when the transport vehicle passes; a secondary scanning unit is arranged on at least one side of the driving route of the transport vehicle; s2, preprocessing the first image; s3, inputting the preprocessed first image into a trained first image processing algorithm, and extracting and identifying label content in the first image; s4, according to the content identification condition, different levels of alarm signals are triggered, and abnormal steel coil position serial numbers are stored; s5, identifying the abnormal steel coil according to the label content on the surface of the steel coil, starting a secondary scanning unit, and obtaining a second image of the label in the coil hole of the steel coil; and S6, inputting the second image into a trained second image processing algorithm, and extracting label content in the second image. After the first image is preprocessed, the label area is searched through the first image processing algorithm, the content of the label area is recognized, and if recognition fails to reach a certain number of times, corresponding measures are taken. If the label content is recognized again, the first image is obtained again, or a secondary scanning unit is adopted to carry out secondary shooting on the label in the coil hole of the steel coil, and a second image obtained through secondary shooting is recognized, so that the purpose of secondary scanning is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of label image recognition, and in particular to an intelligent steel coil warehouse two-dimensional code fast scanning tallying method. Background Art

[0002] Steel coils play a vital role in the steel industry and related fields, and are widely used in manufacturing, construction, automotive, and other fields. Steel coil labels are a crucial means of identifying and managing steel coils, containing essential data such as basic information, specifications, and batch numbers. Traditional manual identification methods suffer from slow speeds, low accuracy, and proneness to errors, making them inadequate for modern production and logistics.

[0003] To improve the accuracy and speed of steel coil material label recognition, machine vision recognition technology has become a new solution. Leveraging computer vision and image processing algorithms, machine vision recognition can automatically sense and identify objects in images, enabling high-speed and accurate identification and classification. However, due to factors such as collisions, contamination, and occlusion during transportation, the success rate of identifying label content on the surface of steel coils is low.

[0004] Therefore, it is very necessary to provide an intelligent steel coil warehouse QR code fast scanning tallying method, which can effectively identify and read the label on the surface of the steel coil, and when an error occurs in reading the label on the surface of the steel coil, promptly issue a corresponding alarm message and perform a secondary confirmation of the label in the coil hole of the steel coil, thereby improving the work efficiency of label recognition and reducing the need for manual intervention. Summary of the Invention

[0005] In view of this, the present invention proposes an intelligent steel coil warehouse QR code fast scanning and tallying method that can automatically scan the material labels of steel coils loaded on vehicles, and enable secondary scanning when the recognition failure reaches a certain level. The secondary scanning unit extends into the coil hole of the steel coil to perform a secondary compensation scan, thereby improving the recognition success rate of the label.

[0006] The present invention provides an intelligent steel coil warehouse QR code fast scanning tallying method, comprising the following steps:

[0007] S1: A primary scanning unit is respectively arranged on both sides of the transport vehicle's travel route, for capturing a first image of both sides of the steel coil on the vehicle when the transport vehicle passes by; a secondary scanning unit is arranged on at least one side of the transport vehicle's travel route;

[0008] S2: preprocessing the first image;

[0009] S3: Inputting the preprocessed first image into the trained first image processing algorithm to extract and identify label content in the first image;

[0010] S4: According to the content recognition situation, trigger different levels of alarm signals and save the abnormal steel coil position number;

[0011] S5: Identify the steel coil with abnormal label content on the surface of the steel coil, activate the secondary scanning unit, and obtain a second image of the label inside the coil hole of the steel coil;

[0012] S6: Input the second image into the trained second image processing algorithm to extract label content in the second image.

[0013] Based on the above technical solution, preferably, the primary scanning unit includes an industrial camera, an industrial computer, a near-infrared LED light source and a sensor, and the industrial camera, the near-infrared LED light source and the sensor are all communicatively connected to the industrial computer; the industrial computer is used to pre-process the first image and preset the trained first image processing algorithm;

[0014] The secondary scanning unit consists of a multi-degree-of-freedom robotic arm, a wide-angle camera and a host computer. The wide-angle camera is set at the end of the multi-degree-of-freedom robotic arm, which can accurately locate the coil hole of the steel coil and drive the wide-angle camera to extend into the coil hole; the wide-angle camera is used to obtain the internal image of the steel coil hole; the host computer is equipped with a visual guidance function, which locates the center position of the steel coil hole by obtaining the image input by the wide-angle camera, and sends the center position of the steel coil hole to the multi-degree-of-freedom robotic arm; the host computer is pre-installed with a trained second image processing algorithm.

[0015] Preferably, the preprocessing of the first image includes the following:

[0016] Denoising processing; Denoising parameters and filter parameters are determined according to environmental conditions: ambient brightness L, dust index D, number of image noise points N and label area clarity S, where the ambient brightness L is obtained by the sensor, and the first threshold of the ambient brightness is set according to the on-site environment; the dust index D is detected by the sensor, and the second threshold of the dust index is set according to the on-site environment; the number of image noise points N is obtained by counting the number of discrete noise points in the first image, and the third threshold of the number of image noise points is set according to the on-site environment; the label area clarity S is the clarity of the label area evaluated by edge detection, and the clarity of the label area is set according to the on-site environment. Fourth threshold; when the ambient brightness L is less than the first threshold of the ambient brightness, adaptive histogram equalization is applied to enhance the contrast, and Gaussian filtering is used to remove noise to avoid over-smoothing; when the dust index D is greater than the second threshold of the dust index, non-local means denoising is used to remove noise caused by dust, and median filtering is combined to remove granular noise; when the number of image noise points N is greater than the third threshold of the number of image noise points, bilateral filtering is used to remove noise while retaining edge information, and wavelet transform is combined to remove high-frequency noise; when the label area clarity S is less than the fourth threshold of the label area clarity, Laplacian operator sharpening or Unsharp Masking is used to sharpen the edge, and super-resolution reconstruction is combined to improve the image resolution; according to the value and weight of each denoising parameter, the filter strength parameter of the filter is determined, and the filter strength of the filter = w1×L+w2×D+w3×N+w4×S, where w1, w2, w3, and w4 are the weights of each denoising parameter, and w1+w2+w3+w4=1;

[0017] Correction processing; Grayscale correction: improve the overall brightness and contrast of the first image; γ correction: nonlinear adjustment, expand the dynamic range of the pixel value of the first image; Perspective correction: eliminate image perspective distortion and make the front of the label parallel to the image plane.

[0018] Further preferably, the content of step S3 is to collect image data of the side of the steel coil, draw a bounding box of the steel coil for each image data, and mark the center position; according to the annotated image data, expand the number of image data through cropping, scaling, rotation, and mirroring operations to obtain a first sample set, and divide the first sample set into a first training set and a first verification set; select the improved YOLOX target detection algorithm as the first image processing algorithm, input the image data in the first training set for training, and use the first verification set to verify the effect of the training to obtain the trained YOLOX target detection algorithm; then input the preprocessed first image into the trained YOLOX target detection algorithm, and output the image of the label in the preprocessed first image.

[0019] More preferably, the content of step S4 is that after obtaining the image of the label in the first image, use the ZebraCrossing library to identify the content of the label in the first image, obtain the label recognition content, and compare and match it with the material information corresponding to the vehicle transportation; if the number of consecutive failures n in recognizing the label recognition content < 3 times, trigger the first-level alarm signal, and at this time, change the focal length of the industrial camera and the illumination intensity of the near-infrared LED light source, and rescan; if the number of consecutive failures 3 < n < 5 in recognizing the label recognition content, trigger the second-level alarm signal, and rescan the current steel coil at multiple angles; if the number of consecutive failures n > 5 in recognizing the label recognition content, trigger the third-level alarm signal and prompt for manual intervention;

[0020] For the first alarm signal or the second-level alarm signal, if there is dirt or occlusion in the label area of the steel coil, use an image restoration algorithm based on deep learning to repair the damaged area, infer the missing content of the occluded label through context information, or reshoot the image of the label from different angles, and select the unoccluded label image to re-recognize;

[0021] Record the position serial number of the steel coil with abnormal label recognition corresponding to the third-level alarm signal, and execute step S5.

[0022] Even more preferably, the host computer pre-sets a trained second image processing algorithm, and the training process of the second image processing algorithm includes:

[0023] Use an industrial camera to take pictures of steel coils from different angles, illumination conditions and distances, collect steel coil images of different specifications, different surface states and coil hole shapes, and ensure the diversity of steel coil images;

[0024] Use a labeling tool to label the steel coil contour and coil holes, generate a steel coil bounding box or mask, and manually correct the blurred, occluded or partially missing contour to obtain a second sample;

[0025] Perform rotation, scaling, translation, and flipping transformations on the second sample to increase sample diversity, add random noise, simulate illumination changes and dust interference, and obtain a second sample set; divide the second sample set into a second training set and a second validation set;

[0026] Use a neural network deep learning training algorithm as the second image processing algorithm, initialize the model weights of the second image processing algorithm, design a loss function, use the second training set to train the second image processing algorithm, and use the second validation set to verify the training results, dynamically adjust the learning rate and monitor the training process;

[0027] After the above process, obtain a trained second image processing algorithm for identifying the position of the steel coil holes.

[0028] More preferably, the host computer also performs path planning for the multi-degree-of-freedom robotic arm, specifically including: introducing a position offset compensation mechanism, detecting the actual position of the steel coil by a sensor, combining the position of the steel coil hole identified by the second image processing algorithm, calculating the position deviation between the current end of the multi-degree-of-freedom robotic arm and the center of the steel coil hole, and dynamically adjusting the posture of the multi-degree-of-freedom robotic arm according to the position deviation, so that the wide-angle camera extends into the inside of the steel coil hole, and the field of view of the wide-angle camera covers the area where the label in the steel coil hole is located; and obtaining a second image of the label inside the steel coil hole.

[0029] Further preferably, when there is an overexposed or reflective area in the first image or the second image, which affects the recognition of the label, any one of the following measures is taken:

[0030] 1) Adjust the exposure time of industrial cameras or wide-angle cameras to 50%-70% of the default value to reduce the time light enters the camera;

[0031] 2) Adjust the aperture of industrial cameras or wide-angle cameras to 80%-90% of the default value to limit the amount of light entering and reduce the degree of reflection;

[0032] 3) Adjust the ISO sensitivity of the industrial camera or wide-angle camera to ISO100-ISO200;

[0033] 4) Install a polarization filter on the industrial camera or wide-angle camera to eliminate polarized light on the surface of the steel coil;

[0034] 5) Install a neutral density filter on an industrial camera or wide-angle camera to reduce the amount of light entering and avoid overexposure;

[0035] 6) Use an industrial camera or a wide-angle camera to capture multiple images with different exposure parameters, obtain the camera response curve (CRF) of the industrial camera or wide-angle camera, convert the pixel values of each image into scene brightness values based on the camera response curve (CRF), perform a weighted average on the converted scene brightness values to balance the brightness differences between images with different exposure parameters, fuse the multiple images with different exposure parameters, and remap the fused brightness values back to pixel values to obtain an image without overexposure.

[0036] More preferably, the dynamically adjusting the posture of the multi-degree-of-freedom robotic arm according to the position deviation so that the wide-angle camera extends into the coil hole of the steel coil includes the following:

[0037] Obtain the starting position coordinates of the multi-degree-of-freedom robotic arm, design the optimal path to calculate the best path from the current position to the target position, set a safety distance in the path planning to ensure that the end effector of the multi-degree-of-freedom robotic arm does not touch the inner wall of the steel coil during movement; set the angle adjustment range of each joint of the multi-degree-of-freedom robotic arm to prevent the robotic arm from touching the steel coil and causing damage to the steel coil;

[0038] Based on the structure and kinematic model of the multi-DOF manipulator, the inverse solution method is used to determine the angle of each joint so that the end effector of the multi-DOF manipulator reaches the target position. The inverse solution is the process of converting the target position into the joint angle.

[0039] The joint angles obtained by the inverse solution are sent as control instructions to the robot controller to control the robot to move along the calculated path.

[0040] More preferably, after the multi-degree-of-freedom robotic arm drives the wide-angle camera into the inside of the steel coil hole, it captures the label image inside the steel coil hole, performs denoising, grayscale, edge detection and edge sharpening processing on the image, calculates the feature matching points between the label images inside the steel coil hole through the SIFT feature matching algorithm, performs feature matching on the multiple collected label images inside the steel coil holes, aligns the label images inside the steel coil holes in the same coordinate system, obtains a complete second image, uses the Zebra Crossing library to perform label content recognition in the second image, and extracts label content information.

[0041] The intelligent steel coil warehouse QR code fast scanning tallying method provided by the present invention has the following beneficial effects compared with the prior art:

[0042] (1) The material labels of the steel coils loaded on the vehicle can be photographed and scanned by the primary scanning units on both sides of the vehicle in the direction of travel. The label area is found by the first image processing algorithm after preprocessing the first image, and the content of the label area is identified. If the recognition fails for a certain number of times, corresponding measures are taken, such as re-identifying the label content, re-acquiring the first image, or using the secondary scanning unit to take a second photo of the label in the coil hole of the steel coil, and identifying the second image obtained by the secondary photo, thereby achieving the purpose of secondary scanning. It can not only effectively overcome the situation where the label is dirty, damaged, or reflective, but also reduce the link of manual intervention and improve the efficiency of label recognition;

[0043] (2) Constraints were set for the image preprocessing of the first and second shots, and the training content of the corresponding image processing algorithms, which improved the image quality and provided data support for the recognition and content reading of specific label areas;

[0044] (3) When overexposure or reflective areas affect the recognition of label content, the present invention provides corresponding adjustment measures to reduce the impact of overexposure and reflective areas on the image quality of the label area;

[0045] (4) Path planning content is provided for the secondary scanning unit to drive the wide-angle camera into the coil hole to take pictures, so as to avoid movement interference, damage to the wide-angle camera or scratching the inner surface of the steel coil, and clearly obtain the label image inside the coil hole of the steel coil. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 This is a flowchart of the steps of a method for quickly scanning and tallying QR codes in an intelligent steel coil warehouse according to the present invention;

[0048] Figure 2 This is a structural block diagram of a single scanning unit of an intelligent steel coil warehouse QR code fast scanning tallying method according to the present invention;

[0049] Figure 3 This is a three-dimensional diagram of the secondary scanning unit of the intelligent steel coil warehouse QR code rapid scanning tallying method of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] like Figure 1 As shown, the present invention provides an intelligent steel coil warehouse QR code fast scanning tallying method, comprising the following steps:

[0052] S1: A primary scanning unit is configured on both sides of the transport vehicle's travel route to capture the first image of both sides of the steel coil on the vehicle when the transport vehicle passes by; a secondary scanning unit is configured on at least one side of the transport vehicle's travel route.

[0053] Among them, Figure 2As shown, a primary scanning unit includes an industrial camera, an industrial computer, a near-infrared LED light source, and a sensor. The industrial camera, the near-infrared LED light source, and the sensor are all communicatively connected to the industrial computer. The industrial computer is used to pre-process the first image and pre-install the trained first image processing algorithm. Of course, a PC can also be further configured to store the first image.

[0054] Considering that industrial cameras need to adapt to outdoor working environments around the clock and avoid irritating the human eye, high-intensity near-infrared LED light sources with triggerable photography are preferred for supplemental lighting. Industrial cameras are preferably CMOS cameras with high quantum efficiency in the near-infrared band, with a wide operating temperature range that meets technical requirements. Furthermore, the cameras should feature high-speed capture and a high dynamic range. GigE is the preferred data transmission protocol for facilitating later data networking. For transmission distances greater than 100 meters, optical fiber is used as the data transmission medium. Industrial cameras can be equipped with infrared high-bandpass and high-resolution industrial lenses. A PLC industrial computer can be used to detect mismatches in material information within the system after scanning, triggering an alarm prompt and initiating a rescan of the concave unit or a secondary scan using the secondary scanning unit. The industrial computer is equipped with a deployment platform for the first image processing algorithm, which includes the industrial camera driver and SDK, as well as the runtime environment for the first image processing algorithm, such as PyTorch and OpenCV. The sensors include vision sensors, light sensors and dust sensors. The vision sensor is used to obtain the current position of the vehicle; the light sensor is used to obtain the ambient light intensity; and the dust sensor is used to obtain the ambient dust concentration.

[0055] like Figure 3 As shown, the secondary scanning unit consists of a multi-degree-of-freedom robotic arm, a wide-angle camera, and a host computer. The wide-angle camera, located at the end of the multi-degree-of-freedom robotic arm, can precisely locate the coil hole and drive the wide-angle camera into the hole. The wide-angle camera is used to capture images of the coil hole. The host computer is equipped with a visual guidance function, which uses the image input from the wide-angle camera to locate the center of the coil hole and transmits the center position of the coil hole to the multi-degree-of-freedom robotic arm. The host computer is pre-installed with a trained secondary image processing algorithm. To enable movement over a larger range, the secondary scanning unit is equipped with a track-based linear motion mechanism and a turntable. The turntable is mounted on top of the track-based linear motion mechanism, and the multi-degree-of-freedom robotic arm is mounted on top of the turntable.

[0056] S2: Preprocess the first image.

[0057] The preprocessing of the first image specifically includes the following contents:

[0058] Denoising: Denoising parameters and filter parameters are determined based on environmental conditions: ambient brightness L, dust index D, number of image noise points N, and label area clarity S, where:

[0059] The ambient brightness L is obtained through the sensor, and the first threshold of the ambient brightness is set according to the on-site environment; when the ambient brightness L is less than the first threshold of the ambient brightness, adaptive histogram equalization is applied to enhance the contrast, and Gaussian filtering is used to remove noise to avoid over-smoothing; adaptive histogram equalization is to divide the first image into several small blocks, each small block is used as a local window, and the grayscale histogram of the pixel value of each local window is calculated. By redistributing the grayscale value of each local window, the grayscale distribution of each local window is more uniform, and then the local windows are merged again to obtain the first image after adaptive histogram equalization to enhance the contrast. However, adaptive histogram equalization may amplify the noise in the image, so Gaussian filtering is further used to remove noise. Gaussian filtering is a smoothing filtering technology. The Gaussian kernel is convolved with the first image after adaptive histogram equalization. In the Gaussian filter, each pixel in the image is updated by the weighted average of the adjacent pixels, and the weight is determined by the Gaussian kernel function.

[0060] The dust index D is detected by sensors, and a second threshold for the dust index is set based on the on-site environment. When the dust index D exceeds the second threshold, non-local means denoising (NLM) is used to remove noise caused by dust, combined with median filtering to remove granular noise. Non-local means denoising (NLM) is a denoising method based on image self-similarity. It compares the neighborhood of pixels in an image, finds similar areas, and performs a weighted average of these similar areas to eliminate noise. It is particularly suitable for removing noise caused by dust and particulate matter. Self-similarity measures the similarity of local areas of an image, such as 3×3 or 5×5 pixel blocks, using Euclidean distance or cosine similarity.

[0061] The number of image noise points N is obtained by counting the number of discrete noise points in the first image, and the third threshold of the number of image noise points is set according to the scene environment; when the number of image noise points N>the third threshold of the number of image noise points, bilateral filtering is used to remove noise while retaining edge information, and wavelet transform is combined to remove high-frequency noise; bilateral filtering is a nonlinear filtering that comprehensively considers spatial distance and pixel value similarity, so that the edges are not blurred when smoothing the image; wavelet transformation is to clearly separate the high-frequency and low-frequency components in the image, and then use the inverse wavelet transform to achieve image reconstruction and output the denoised image.

[0062] The label area clarity S is the clarity of the label area evaluated based on edge detection, and the fourth threshold of the label area clarity is set according to the scene environment; when the label area clarity S is less than the fourth threshold of the label area clarity, Laplace operator sharpening or Unsharp Masking is used to enhance the edge, and the image resolution is improved in combination with super-resolution reconstruction; Laplace operator sharpening is a second-order differential calculation that enhances the edges and details of the labels in the image by applying a Laplace filter to the original image; Unsharp Masking sharpening is also a commonly used image enhancement technology that performs Gaussian blur on the original image and subtracts the original image, and applies an enhanced sharpening filter to the result to enhance the details and edges of the image.

[0063] It should be noted that the first, second, third, and fourth thresholds are not fixed but are dynamically determined based on weather conditions or changes in the focal length of the industrial camera. For example, on a sunny day, the maximum value for the first threshold is 200 LU, the maximum value for the second threshold is 30 LU / m³, the maximum value for the third threshold is 300 LU, and the maximum value for the fourth threshold is 0.9. On cloudy or foggy days, where ambient brightness is low and dust is abundant, the maximum values for the first threshold are 60 LU, the maximum value for the second threshold is 60 LU, the maximum value for the third threshold is 500 LU, and the maximum value for the fourth threshold is 0.5.

[0064] According to the values and weights of each denoising parameter, the filtering strength parameter of the filter is determined. The filtering strength of the filter = w1×L+w2×D+w3×N+w4×S, where w1, w2, w3, and w4 are the weights of each denoising parameter, and w1+w2+w3+w4=1.

[0065] Correction processing specifically includes the following:

[0066] Grayscale correction: improving the overall brightness and contrast of the first image. Specifically, this can be achieved by adjusting the grayscale value of the first image through histogram equalization, which is conducive to preserving image details.

[0067] Gamma correction: Nonlinear adjustment to expand the dynamic range of the first image pixel values. By exponentially adjusting the normalized image pixel values by the gamma value, when γ < 1, the image is brighter, which can enhance the details of dark areas or night images. When γ > 1, the image is darker and the highlights are suppressed, which can be used to remove light spots.

[0068] Perspective Correction: Eliminates image perspective distortion, aligning the front of the label with the image plane. This is achieved by applying a projective transformation to the image, such as applying a homography to the four vertices of the label's rectangular region. This ensures the label appears as a correct rectangle in the image, facilitating subsequent label content recognition.

[0069] S3: Input the preprocessed first image into the trained first image processing algorithm to extract and recognize the label content in the first image;

[0070] Specifically, collect the image data of the side of the steel coil, draw the bounding box of the steel coil for each image data, and mark the center position. A labeling tool such as labelImg can be used; according to the labeled image data, through operations such as cropping, scaling, rotating, and mirroring, expand the quantity of the image data to obtain the first sample set, and divide the first sample set into the first training set and the first validation set; select the improved YOLOX object detection algorithm as the first image processing algorithm, input the image data in the first training set for training, and use the first validation set to verify the training effect to obtain the trained YOLOX object detection algorithm; then input the preprocessed first image into the trained YOLOX object detection algorithm to output the image of the label in the preprocessed first image.

[0071] In this embodiment, the improvement of the YOLOX object detection algorithm is as follows: Replace the Backbone with EfficientNet to make the model more efficient; use CloU to improve the IoU loss and enhance the bounding box regression accuracy; use mixed-precision training to speed up the training speed of the model and reduce the video memory occupancy. The YOLOX object detection algorithm can select common versions such as YOLOV5 and YOLOV8.

[0072] S4: Trigger alarm signals of different levels according to the content recognition situation, and save the serial numbers of the abnormal steel coil positions.

[0073] The content of step S4 is that after obtaining the image of the label in the first image, use the Zebra Crossing library to recognize the label content in the first image to obtain the label recognition content, and compare and match it with the material information corresponding to the vehicle transportation; if the number of consecutive failures n in recognizing the label recognition content is less than 3 times, trigger the first-level alarm signal, and at this time, change the focal length of the industrial camera and the illumination intensity of the near-infrared LED light source and rescan; if the number of consecutive failures 3 < n < 5 in recognizing the label recognition content, trigger the second-level alarm signal and rescan the current steel coil from multiple angles; if the number of consecutive failures n > 5 in recognizing the label recognition content, trigger the third-level alarm signal and prompt for manual intervention; The ZebraCrossing library is an open-source library mainly used for recognizing the content of barcodes and QR codes.

[0074] For the first or second level alarm signals, if the label area of the steel coil is dirty or blocked, a deep learning-based image restoration algorithm is used to repair the damaged area, infer the missing content of the blocked label based on contextual information, or re-take the label image from a different angle and select the unblocked label image for re-recognition;

[0075] Record the position serial number of the steel coil with abnormal identification corresponding to the third-level alarm signal, and execute step S5.

[0076] S5: Identify abnormal steel coils based on the label content on the surface of the steel coil, activate the secondary scanning unit, and obtain a second image of the label inside the coil hole of the steel coil.

[0077] The host computer is pre-installed with a trained second image processing algorithm. A neural network deep learning training algorithm is used as the second image processing algorithm. The model weights of the second image processing algorithm are initialized, a loss function is designed, the second image processing algorithm is trained using the second training set, and the training results are verified using the second validation set. The learning rate is dynamically adjusted and the training process is monitored. Through the neural network deep learning training algorithm, samples of steel coil contours are collected and trained to learn the characteristics of steel coil contours and coil holes. The neural network model is trained using the prepared training set. During training, the model weights and biases are adjusted using the backpropagation algorithm and optimizer to enable the model to accurately identify steel coil contours and coil holes. The trained model is evaluated using the validation set. Evaluation indicators may include accuracy, recall rate, precision, etc.

[0078] The training process of the second image processing algorithm can be broken down into the following steps:

[0079] (1) Sample acquisition and preprocessing: Use a high-resolution industrial camera to capture steel coil images from different angles, lighting conditions, and distances to ensure sample diversity. Collect images of steel coils with different specifications, surface conditions (such as rust and stain), and coil hole shapes to cover various situations in real-world scenarios.

[0080] Sample annotation: Use LabelImg to accurately annotate the steel coil outline and coil holes, generating corresponding bounding boxes or masks. Manually correct blurred, obscured, or partially missing outlines to ensure annotation quality.

[0081] Data augmentation: Perform geometric transformations such as rotation, scaling, translation, and flipping on samples to increase data diversity. Adding random noise, simulating lighting changes, and dust interference improves the model's robustness in complex environments.

[0082] After step (1), a second sample set is obtained, and the second sample set is divided into a second training set and a second validation set.

[0083] (2) Training convergence process, model initialization: use the weights of pre-trained models (such as ResNet, EfficientNet) for initialization to accelerate convergence.

[0084] Loss function design: Use the cross entropy loss function to classify steel coil contours and coil holes. Combined with Dice loss or IoU loss to optimize the accuracy of contour segmentation.

[0085] Training process monitoring: Monitor training loss and validation loss in real time and plot loss curves. Use early stopping to terminate training when validation loss stops decreasing to prevent overfitting.

[0086] Learning rate adjustment: The initial learning rate is set to 10 -3 , use the learning rate scheduler to dynamically adjust the learning rate. When the validation loss stagnates, reduce the learning rate to 1 / 10 of the original value and continue training.

[0087] (3) Training evaluation indicators. The task of this solution is to extract the outline of the label area. The segmentation task evaluation is adopted: the intersection over union (IoU) and Dice coefficient are used to evaluate the accuracy of the outline segmentation. The pixel accuracy (Pixel Accuracy) and mean IoU (Mean IoU) are calculated.

[0088] Core indicator: Dice coefficient: sensitive to contour accuracy, suitable for industrial detection.

[0089] Compliance value: ≥0.8 (the steel coil label usually has a clear structure and should reach this threshold).

[0090] IoU: Complementary to Dice.

[0091] Standard value: ≥0.7 (corresponding to Dice ≥0.8).

[0092] Auxiliary indicators: Pixel Accuracy ≥ 95%.

[0093] The following scheme is used for parameter optimization:

[0094] Hyperparameter tuning: Use Bayesian Optimization Hyperparameters: Learning rate (sampling range is 10 -4 -10 -3 ), batch size (between 32 and 64 to speed up training), and network depth (number of convolutional layers: 3-8 layers).

[0095] Model structure adjustment: Gradually adjust the network structure based on model performance (such as adding convolutional layers and adjusting the number of channels). Use pruning (removing redundant neurons or convolution kernels) and quantization (converting 32-bit floating-point weights to 8-bit integers) to compress the model and improve inference speed.

[0096] Regularization and optimizer selection: Using the AdamW optimizer combined with weight decay (Weight Decay = 0.05) can significantly improve the model's generalization ability.

[0097] Step Adjustment Strategy: A phased training strategy is used, with pre-training on a large dataset followed by fine-tuning on a smaller dataset. The data augmentation intensity and learning rate are gradually adjusted based on the validation set performance to balance the model's generalization and fitting capabilities.

[0098] Based on the evaluation results, the model is fine-tuned, such as adjusting the network structure and hyperparameters. The trained model is then deployed to an actual wide-angle camera. In the images captured by the wide-angle camera, the trained model inputs a second image to accurately identify the coil holes, obtaining their coordinates. These coordinates are then output to a multi-degree-of-freedom robotic arm, which then moves to capture the labels within the holes.

[0099] The second image processing algorithm trained through the above process is used to identify the positions of the coil holes of the steel coil.

[0100] In this embodiment, the host computer also performs path planning for the multi-degree-of-freedom robotic arm. The specific contents include: introducing a position offset compensation mechanism, detecting the actual position of the steel coil by a sensor, combining the center position of the steel coil hole identified by the second image processing algorithm, calculating the position deviation between the current end of the multi-degree-of-freedom robotic arm and the center of the steel coil hole, and dynamically adjusting the posture of the multi-degree-of-freedom robotic arm according to the position deviation, so that the wide-angle camera extends into the interior of the steel coil hole, and the field of view of the wide-angle camera covers the area where the label in the steel coil hole is located; and obtaining a second image of the label inside the steel coil hole.

[0101] The following steps are involved: dynamically adjusting the posture of the multi-degree-of-freedom robotic arm based on the position deviation to allow the wide-angle camera to penetrate the coil hole.

[0102] Get the starting position coordinates of the multi-degree-of-freedom robot arm, extract the geometric parameters of the inner wall of the cylinder (radius, depth), establish the steel coil hole coordinate system, use the Z axis as the center axis of the hole, integrate it with the robot arm base coordinate system, and design the optimal path: Step 1: Quickly approach the label area in the hole from a safe position; Step 2: The end of the robot arm makes a spiral motion along the inner wall of the cylinder to determine the orientation of the label in the hole; Step 3: Finely adjust the camera posture to scan the label in the hole. Calculate the optimal path from the current position to the target position, set a safety distance in the path planning, and ensure that the end effector of the multi-degree-of-freedom robot arm does not touch the inner wall of the steel coil during movement; set the angle adjustment range of each joint of the multi-degree-of-freedom robot arm; prevent the robot arm from touching the steel coil and causing damage to the steel coil;

[0103] Based on the structure and kinematic model of the multi-DOF manipulator, an inverse kinematics solution is used to determine the angle of each joint. The inverse solution method is as follows: first, the position and attitude equations are separated, then the first three joints (position) are solved using geometric relationships, followed by the last three joints (attitude) using Euler angles or rotation matrices, and finally the feasibility and uniqueness of the solution are checked. This allows the end effector of the multi-DOF manipulator to reach the target position. Inverse kinematics solution is the process of converting the target position into joint angles. In this embodiment, conventional methods such as the gradient projection method can be used for inverse kinematics solution, which will not be repeated here.

[0104] The joint angles obtained from the inverse solution are sent as control commands to the robot controller, controlling the robot to move along the calculated path. A PID controller and a real-time monitoring system for the robot are also designed to prevent the robot from contacting the coil or obstacles. Once the robot successfully enters the coil hole, it rotates and simultaneously takes a photo to obtain a second image of the coil hole.

[0105] S6: Input the second image into the trained second image processing algorithm to extract label content in the second image.

[0106] After the multi-degree-of-freedom robotic arm drives the wide-angle camera into the inside of the steel coil hole, it captures the label image inside the steel coil hole, performs denoising, grayscale, edge detection and edge sharpening on the image, and calculates the feature matching points between the label images inside the steel coil hole using the SIFT feature matching algorithm. The feature matching of the multiple label images collected inside the steel coil holes is performed, and the label images inside each steel coil hole are aligned in the same coordinate system to obtain a complete second image. The Zebra Crossing library is used to identify the label content in the second image and extract the label content information.

[0107] As a preferred embodiment, during the label recognition process of the first image and the second image, when there is an overexposed or reflective area in the first image or the second image, which affects the recognition of the label, any one of the following measures is taken:

[0108] 1) Adjust the exposure time of industrial cameras or wide-angle cameras to 50%-70% of the default value to reduce the time light enters the camera;

[0109] 2) Adjust the aperture of industrial cameras or wide-angle cameras to 80%-90% of the default value to limit the amount of light entering and reduce the degree of reflection;

[0110] 3) Adjust the ISO sensitivity of the industrial camera or wide-angle camera to ISO100-ISO200;

[0111] 4) Install a polarization filter on the industrial camera or wide-angle camera to eliminate polarized light on the surface of the steel coil;

[0112] 5) Install a neutral density filter on an industrial camera or wide-angle camera to reduce the amount of light entering and avoid overexposure;

[0113] 6) Use an industrial camera or a wide-angle camera to capture multiple images with different exposure parameters, obtain the camera response curve (CRF) of the industrial camera or wide-angle camera, convert the pixel values of each image into scene brightness values based on the camera response curve (CRF), perform a weighted average on the converted scene brightness values to balance the brightness differences between images with different exposure parameters, fuse the multiple images with different exposure parameters, and remap the fused brightness values back to pixel values to obtain an image without overexposure.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent steel coil warehouse QR code fast scanning tallying method, characterized in that: The steps include: S1: A primary scanning unit is respectively arranged on both sides of the transport vehicle's travel route, for capturing a first image of both sides of the steel coil on the vehicle when the transport vehicle passes by; a secondary scanning unit is arranged on at least one side of the transport vehicle's travel route; S2: preprocessing the first image; S3: Inputting the preprocessed first image into the trained first image processing algorithm to extract and identify label content in the first image; S4: According to the content recognition situation, trigger different levels of alarm signals and save the abnormal steel coil position number; S5: Identify the steel coil with abnormal label content on the surface of the steel coil, activate the secondary scanning unit, and obtain a second image of the label inside the coil hole of the steel coil; S6: Input the second image into the trained second image processing algorithm to extract label content in the second image.

2. The intelligent steel coil warehouse QR code fast scanning tallying method according to claim 1 is characterized in that: The primary scanning unit includes an industrial camera, an industrial computer, a near-infrared LED light source, and a sensor, wherein the industrial camera, the near-infrared LED light source, and the sensor are all communicatively connected to the industrial computer; the industrial computer is used to pre-process the first image and pre-set the trained first image processing algorithm; The secondary scanning unit consists of a multi-degree-of-freedom robotic arm, a wide-angle camera, and a host computer. The wide-angle camera is installed at the end of the multi-degree-of-freedom robotic arm, which can accurately locate the coil hole and drive the wide-angle camera into the coil hole; the wide-angle camera is used to obtain images of the interior of the coil hole. The host computer is equipped with a visual guidance function. By acquiring the image input by the wide-angle camera, it locates the center position of the steel coil hole and sends the center position of the steel coil hole to the multi-degree-of-freedom robotic arm; the host computer is pre-installed with a trained second image processing algorithm.

3. The intelligent steel coil warehouse QR code fast scanning tallying method according to claim 2 is characterized in that: The preprocessing of the first image includes the following: Denoising processing; Denoising parameters and filter parameters are determined according to environmental conditions: ambient brightness L, dust index D, number of image noise points N and label area clarity S, where the ambient brightness L is obtained by the sensor, and the first threshold of the ambient brightness is set according to the on-site environment; the dust index D is detected by the sensor, and the second threshold of the dust index is set according to the on-site environment; the number of image noise points N is obtained by counting the number of discrete noise points in the first image, and the third threshold of the number of image noise points is set according to the on-site environment; the label area clarity S is the clarity of the label area evaluated by edge detection, and the clarity of the label area is set according to the on-site environment. Fourth threshold; when the ambient brightness L is less than the first threshold of the ambient brightness, adaptive histogram equalization is applied to enhance the contrast, and Gaussian filtering is used to remove noise to avoid over-smoothing; when the dust index D is greater than the second threshold of the dust index, non-local mean denoising is used to remove noise caused by dust, and median filtering is combined to remove granular noise; when the number of image noise points N is greater than the third threshold of the number of image noise points, bilateral filtering is used to remove noise while retaining edge information, and wavelet transform is combined to remove high-frequency noise; when the label area clarity S is less than the fourth threshold of the label area clarity, Laplacian operator sharpening or Unsharp Masking is used to sharpen and enhance the edge, and super-resolution reconstruction is combined to improve the image resolution; Determine the filtering intensity parameter of the filter according to the values and weights of the denoising parameters. The filtering intensity of the filter = w1×L + w2×D + w3×N + w4×S, where w1, w2, w3, and w4 are the weights of the denoising parameters, and w1 + w2 + w3 + w4 = 1; Calibration process; Gray-scale calibration: Improve the overall brightness and contrast of the first image; Gamma calibration: Nonlinear adjustment to expand the dynamic range of the pixel values of the first image; Perspective correction: Eliminate image perspective distortion and make the front of the label parallel to the image plane.

4. The intelligent steel coil warehouse QR code fast scanning tallying method according to claim 3 is characterized in that: The content of step S3 is to collect the image data of the side of the steel coil, draw the bounding box of the steel coil for each image data, and mark the center position; According to the marked image data, expand the quantity of the image data through operations such as cropping, scaling, rotating, and mirroring to obtain the first sample set, and divide the first sample set into the first training set and the first validation set; Select the improved YOLOX object detection algorithm as the first image processing algorithm, input the image data in the first training set for training, and use the first validation set to verify the training effect to obtain the trained YOLOX object detection algorithm; Then input the preprocessed first image into the trained YOLOX object detection algorithm to output the image of the label in the preprocessed first image.

5. The intelligent steel coil warehouse QR code rapid scanning tallying method according to claim 4 is characterized in that: The content of step S4 is that after obtaining the image of the label in the first image, use the Zebra Crossing library to identify the content of the label in the first image to obtain the label recognition content, and compare and match it with the material information corresponding to the vehicle transportation; If the number of consecutive failures n in recognizing the label recognition content < 3 times, trigger the first-level alarm signal, and at this time, change the focal length of the industrial camera and the illumination intensity of the near-infrared LED light source, and rescan; If the number of consecutive failures 3 < n < 5 in recognizing the label recognition content, trigger the second-level alarm signal and rescan the current steel coil at multiple angles; If the number of consecutive failures n > 5 in recognizing the label recognition content, trigger the third-level alarm signal and prompt for manual intervention; For the first alarm signal or the second-level alarm signal, if there are situations of dirt and occlusion in the label area of the steel coil, then use an image restoration algorithm based on deep learning to repair the damaged area, infer the missing content of the occluded label through context information, or reshoot the image of the label from different angles and select the unoccluded label image for re-identification; Record the position serial number of the steel coil with abnormal label recognition corresponding to the third-level alarm signal, and execute step S5.

6. The intelligent steel coil warehouse QR code rapid scanning tallying method according to claim 5 is characterized in that: The upper computer pre-sets the trained second image processing algorithm, and the training process of the second image processing algorithm includes: Use an industrial camera to take pictures of the steel coil from different angles, illumination conditions, and distances, collect steel coil images with different specifications, different surface states, and coil hole shapes to ensure the diversity of the steel coil images; Use a labeling tool to label the steel coil contour and coil holes, generate a steel coil bounding box or mask, and manually correct the blurred, occluded, or partially missing contour to obtain the second sample; The second sample is rotated, scaled, translated, and flipped to increase sample diversity, and random noise is added to simulate illumination changes and dust interference to obtain a second sample set; the second sample set is divided into a second training set and a second validation set; Using a neural network deep learning training algorithm as the second image processing algorithm, initializing the model weights of the second image processing algorithm, designing a loss function, training the second image processing algorithm using a second training set, verifying the training results using a second validation set, dynamically adjusting the learning rate, and monitoring the training process; The second image processing algorithm trained through the above process is used to identify the positions of the coil holes of the steel coil.

7. The intelligent steel coil warehouse QR code fast scanning tallying method according to claim 6 is characterized in that: The host computer also performs path planning for the multi-degree-of-freedom robotic arm. Specifically, it introduces a position offset compensation mechanism, detects the actual position of the steel coil through sensors, combines the position of the steel coil hole identified by the second image processing algorithm, calculates the position deviation between the current end of the multi-degree-of-freedom robotic arm and the center of the steel coil hole, and dynamically adjusts the posture of the multi-degree-of-freedom robotic arm according to the position deviation, so that the wide-angle camera extends into the inside of the steel coil hole. The field of view of the wide-angle camera covers the area where the label in the steel coil hole is located; and obtains a second image of the label inside the steel coil hole.

8. An intelligent steel coil warehouse QR code fast scanning tallying method according to any one of claim 5 or claim 7, characterized in that: When there are overexposed or reflective areas in the first or second image that affect label recognition, take any of the following measures: 1) Adjust the exposure time of industrial cameras or wide-angle cameras to 50%-70% of the default value to reduce the time light enters the camera; 2) Adjust the aperture of industrial cameras or wide-angle cameras to 80%-90% of the default value to limit the amount of light entering and reduce the degree of reflection; 3) Adjust the ISO sensitivity of the industrial camera or wide-angle camera to ISO100-ISO200; 4) Install a polarization filter on the industrial camera or wide-angle camera to eliminate polarized light on the surface of the steel coil; 5) Install a neutral density filter on an industrial camera or wide-angle camera to reduce the amount of light entering and avoid overexposure; 6) Use an industrial camera or a wide-angle camera to capture multiple images with different exposure parameters, obtain the camera response curve (CRF) of the industrial camera or wide-angle camera, convert the pixel values of each image into scene brightness values based on the camera response curve (CRF), perform a weighted average on the converted scene brightness values to balance the brightness differences between images with different exposure parameters, fuse the multiple images with different exposure parameters, and remap the fused brightness values back to pixel values to obtain an image without overexposure.

9. The intelligent steel coil warehouse QR code rapid scanning tallying method according to claim 7 is characterized in that: The method of dynamically adjusting the posture of the multi-degree-of-freedom robotic arm according to the position deviation so that the wide-angle camera extends into the coil hole of the steel coil includes the following: Obtain the starting position coordinates of the multi-degree-of-freedom robot arm, design the optimal path to calculate the best path from the current position to the target position, and set a safety distance in the path planning to ensure that the end effector of the multi-degree-of-freedom robot arm does not touch the inner wall of the steel coil during movement; Set the angle adjustment range of each joint of the multi-degree-of-freedom robotic arm; prevent the robotic arm from touching the steel coil and causing damage to the steel coil; According to the structure and kinematic model of the multi-degree-of-freedom manipulator, the angle of each joint is determined using the inverse solution method so that the end effector of the multi-degree-of-freedom manipulator reaches the target position; Inverse solution is the process of converting target position into joint angle; The joint angles obtained by the inverse solution are sent as control instructions to the robot controller to control the robot to move along the calculated path.

10. The intelligent steel coil warehouse QR code rapid scanning tallying method according to claim 7 is characterized in that: After the multi-degree-of-freedom robotic arm drives the wide-angle camera into the inside of the steel coil hole, it captures the label image inside the steel coil hole, performs denoising, grayscale, edge detection and edge sharpening on the image, and calculates the feature matching points between the label images inside the steel coil hole using the SIFT feature matching algorithm. The feature matching of the multiple label images collected inside the steel coil holes is performed, and the label images inside each steel coil hole are aligned in the same coordinate system to obtain a complete second image. The Zebra Crossing library is used to identify the label content in the second image and extract the label content information.

Images