Intelligent steel label automatic identification method and system based on machine vision

Abstract

The invention relates to the technical field of label identification, and discloses an intelligent steel label automatic identification method and system based on machine vision, and the method comprises the steps: obtaining scene image data directly facing a steel coil, and taking the scene image data as identification input; performing image enhancement and row target detection based on the scene image data, and identifying and obtaining a bounding box of a steel two-dimensional code label; based on the bounding box, a simplified 2d-3d pose solving process is adopted, and the spatial position coordinates of the steel two-dimensional code label relative to the image acquisition equipment are calculated; and performing image capture and recognition on the steel two-dimensional code label at the target spatial position coordinate, reading identity information on the steel two-dimensional code label, uploading a result to a warehouse management system, and completing steel identity registration.

Description

Technical Field

[0001] This invention relates to the field of label recognition technology, specifically to an intelligent steel label automatic recognition method and system based on machine vision. Background Technology

[0002] To reduce costs and increase efficiency, steel companies have widely established automated storage and retrieval systems (AS / RS). Currently, the common practice in steel inventory management is to attach a physical tag to each bundle of steel as a unique identifier. This tag is typically made of galvanized steel sheet or rigid cardboard, encapsulates an NFC chip, and has a waterproof ink-printed surface containing visible information such as QR codes, barcodes, and production dates. However, existing technology has significant drawbacks: First, due to the irregular shape of steel and the variable tag placement, manual scanning with a handheld scanner is necessary. Operators must move between heavy equipment, posing a high risk to personal safety and resulting in frequent accidents and injuries. Second, while the NFC chip solution enables contactless identification, the high cost of the chip itself increases the cost per ton of steel by approximately two yuan, a substantial expense for large-scale production. Therefore, the industry urgently needs a contactless automated identification solution that can both ensure personnel safety and control costs.

[0003] In view of the shortcomings of the prior art, the purpose of this invention is to provide an intelligent steel label automatic identification method and system to solve the problems of high cost, low efficiency and poor security of NFC and manual scanning methods. Summary of the Invention

[0004] This invention provides a machine vision-based intelligent steel label automatic identification method and system, which helps to solve the problems mentioned in the background art.

[0005] This invention provides the following technical solution: a machine vision-based intelligent steel label automatic identification method, comprising: including: Acquire scene image data facing the steel coil as recognition input; Image enhancement and line target detection are performed based on the scene image data to identify and obtain the bounding box of the steel QR code label; Based on the bounding box, a simplified 2D-3D pose solving process is used to calculate the spatial coordinates of the steel QR code label relative to the image acquisition device. The system captures and recognizes images of steel QR code labels located at the target spatial coordinates, reads the identity information on the steel QR code labels, uploads the results to the warehouse management system, and completes the steel identity registration.

[0006] Optionally, acquiring scene image data facing the steel coil includes: Adjust the industrial camera so that its optical axis is aligned with the long axis of the steel coil, and capture an image of the scene with the steel coil at this point.

[0007] Optionally, the step of performing image enhancement and row target detection based on the scene image data to identify and obtain the bounding box of the steel label specifically includes: Image enhancement workflow: An SRGAN-based augmentation model based on generative adversarial networks was used to train the training data, and the parameters of the augmentation model were obtained through adversarial training between the generator and the discriminator. When constructing the training data, the data is expanded by simulating reflections, changes in brightness and darkness, and dust noise in industrial scenarios. During runtime, the scene image is input into the enhancement model, and the enhanced image is output. Target detection process: Adopting based on improvement A lightweight object detection algorithm that replaces the standard convolutional structure with a depthwise separable convolutional structure to reduce the number of model parameters and improve the detection speed. The enhanced image is input into the target detection algorithm, which outputs the detected target of the steel label and its bounding box coordinate parameter set.

[0008] Optionally, the step of calculating the spatial coordinates of the steel label relative to the image acquisition device based on the bounding box and using a simplified 2D-3D pose solving process specifically includes: Load the camera's internal and external parameter sets; Image features for pose determination are obtained within the bounding box; The rigid body transformation step, perspective projection step, and coordinate transformation step are executed sequentially to convert world coordinates into pixel coordinates. The depth information and spatial coordinate parameters of the label relative to the image acquisition device are calculated based on the transformation results.

[0009] Optional, including: The spatial location coordinates are sent to the control server via the network; The control server generates control commands based on the received data, and uses the set of control commands to control the collection device to reach the designated space for image capture and recognition.

[0010] Optionally, the collection device is a robotic arm or a drone, which controls a camera to move to the location of the steel label and take a picture of the steel label.

[0011] Optionally, before capturing and recognizing the image of the steel label located at spatial coordinates, the method further includes: The system determines the condition of steel labels. If the labels are found to be peeling, obscured, or severely damaged, the system will issue an alarm and notify manual intervention. If the steel QR code label is in normal condition, then the steel QR code label is identified and decoded. The identification and decoding results are uploaded to the warehouse management system.

[0012] Optionally, if the steel QR code label is in normal condition, then the process of recognizing and decoding the steel QR code label specifically includes: Capture close-up images of the steel QR code label; The steel QR code label is identified and decoded using a QR code recognition algorithm model based on traditional image processing. Based on the decoding results, registration record data containing the steel QR code label identity information is generated, and the steel identity registration is completed.

[0013] A system for implementing the machine vision-based intelligent steel warehouse label automatic recognition method includes: Image acquisition module: Acquires scene images of steel using an industrial camera; QR code recognition and decoding module: Recognizes the QR code on steel labels.

[0014] The present invention has the following beneficial effects: High safety: The scanning operation can be carried out by a robotic arm or drone at a designated location, which completely replaces the manual scanning process, eliminates the safety risks of personnel working in heavy equipment, and achieves inherent safety.

[0015] Low cost: Traditional data collection methods require NFC chips, which are costly. This method eliminates the expensive NFC chips and relies solely on visual recognition of printed QR codes, significantly reducing the cost per ton of steel.

[0016] Highly efficient and accurate: Based on the improved YOLOv5 algorithm, it has a fast recognition speed and high accuracy, and can work continuously 24 / 7, with efficiency far exceeding that of manual methods.

[0017] Automation and Intelligence: The system deeply integrates visual recognition, precise positioning, and robot control technologies to form a complete automation closed loop. The algorithm model is optimized for harsh industrial environments, such as reflection, dust, and changes in light and dark, ensuring the high availability and stability of the system. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the process of the present invention.

[0019] Figure 2 This is a schematic diagram of coordinate transformation according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example, refer to Figure 1 A machine vision-based method for automatic identification of intelligent steel labels includes: Acquire scene image data facing the steel coil as recognition input; Image enhancement and line target detection are performed based on the scene image data to identify and obtain the bounding box of the steel QR code label; Based on the bounding box, a simplified 2D-3D pose solving process is used to calculate the spatial coordinates of the steel QR code label relative to the image acquisition device. The system captures and recognizes images of steel QR code labels located at the target spatial coordinates, reads the identity information on the steel QR code labels, uploads the results to the warehouse management system, and completes the steel identity registration.

[0022] The step of parking the transport trolley carrying the steel in a predetermined position so that the industrial camera is directly facing the center of the long axis of the steel coil to form a direct viewing angle specifically includes: The transport vehicle will be parked at the designated location based on the parking location parameters; Adjust the industrial camera fixed in the warehouse so that its optical axis is aligned with the long axis of the steel coil and kept directly facing it; After the alignment is completed, output the acquisition preparation status parameters.

[0023] The step of acquiring scene image data facing the steel coil when the transport trolley is at the predetermined position, as recognition input, specifically includes: When the transport trolley reaches the predetermined position, scene image data facing the steel coil is collected. Generate time and location identifier parameters corresponding to this data acquisition; The scene image is used as input data for image enhancement and object detection.

[0024] The process of first performing image enhancement based on the scene image data, then performing target detection, and identifying and obtaining the bounding box of the steel label specifically includes: Image enhancement workflow: An SRGAN-based augmentation model based on generative adversarial networks was used to train the training data, and the parameters of the augmentation model were obtained through adversarial training between the generator and the discriminator. When constructing the training data, the data is expanded by simulating reflections, changes in brightness and darkness, and dust noise in industrial scenarios. During runtime, the scene image is input into the enhancement model, and the enhanced image is output. Employing Generative Adversarial Networks network Perform image enhancement. Through generator and discriminator Adversarial training can generate perceived quality Higher resolution, more realistic and natural visuals. The network proposed the concept of perceived loss ( (Used to replace traditional mean square error) The loss function is composed of adversarial loss (from the discriminator) and content loss (based on...). Feature map computation for the network, rather than pixel-level computation. The weighted sum of ).

[0025] It mainly consists of two modules: generator and discriminator .

[0026] The generator's task is to receive a low-resolution image. Image, and output a high-resolution image. The image was processed using a deep residual network. As the main trunk, its workflow is as follows: S1, Low-level feature extraction: First, a convolutional layer is used to extract initial features.

[0027] S2, Core Residual Block: Next identical residual blocks This is crucial for performance. Each residual block contains two. Convolutional layers, followed by batch normalization and Activate the function and use skip connections.

[0028] S3, Upsampling Module: Following the residual block is the upsampling section, responsible for increasing the image size. It first uses a convolutional layer to increase the number of channels by a magnification factor of 2^3, then uses... The operation rearranges the pixels, thereby increasing the spatial size and reducing the number of channels.

[0029] S4, High-Resolution Reconstruction: The last convolutional layer produces the final 3-channel super-resolution image.

[0030] The discriminator's task is to act as a binary classifier, determining whether an image is a real, high-resolution image. Or is it a fake image created by a generator? It adopted a similar approach. The network architecture is a continuously downsampling network. Its workflow is as follows: S5. Input an image (which may be...) or ).

[0031] S6, after a series of The convolutional layers are downsampled, resulting in smaller and smaller feature map sizes and an increasing number of channels.

[0032] S7, finally through the fully connected layer and The activation function outputs a probability value (between 0 and 1) indicating whether the discriminator considers the image to be real. The possibilities of images.

[0033] In short, To distinguish between genuine and fake as accurately as possible, The goal is to generate images that are as realistic as possible to deceive the eye. .

[0034] In terms of datasets, steel image data was augmented by simulating steel reflection, changes in ambient light and darkness, and adding dust noise. This enhanced the model's generalization ability and resistance to interference in complex industrial environments, achieving image enhancement. The goal is to teach the model to ignore these disturbances and focus on the defects or features of the steel itself by simulating common visual changes in real industrial scenarios.

[0035] Simulating Steel Reflection: Generating simulated light source effects at random locations on the image. Using a Gaussian function to create light spots, the Gaussian function makes the center of the light source brightest, then smoothly and gradually darkens outwards, forming a very natural circular or elliptical halo, just like real reflections. This gradient light spot pattern (essentially a brightness matrix) is overlaid onto the original steel image. Different brightness levels of the overlaid brightness matrix are simulated by adjusting the "intensity," set within the range [0.1, 0.3]. Simulating Environmental Light and Dark Changes: 1) Randomly adjust global brightness (±30%) (randomly brightening or darkening the image). 2) Randomly adjust contrast (±20%) to make bright areas brighter and dark areas darker, or to make the overall image more muted. 3) Add local shadows, creating a shadow layer that gradually darkens from the center of the image outwards. The shadow is lightest at the center and darkest at the four corners. Multiplying this shadow layer with the original image realistically simulates the effect of ambient light blocking. Add dust noise: Simulate the tiny, granular noise (1-8 pixels in size) formed by dust particles in the image. On a brand new, completely black canvas, randomly generate "white dots" of random brightness and size to simulate dust particles. Overlay this dust-particle-filled canvas with a certain degree of transparency onto the original steel image to simulate the dust particle occlusion effect. Add an additional layer of weak, uniform Gaussian random noise to simulate noise generated by the camera sensor in low light, as well as extremely fine suspended particles in the air, making the overall image texture more closely resemble a realistic industrial shooting environment.

[0036] Target detection process: A lightweight improvement to the object detection algorithm is made by replacing the standard convolutional structure with a depthwise separable convolutional structure to reduce the number of model parameters and improve the detection rate. At the same time, the cross-stage partial network CSPNet (CVPR 2019) is integrated into the network architecture to solve the problems of computational redundancy, high memory consumption and optimization difficulties while retaining the advantages of feature reuse of traditional dense networks. The enhanced image is input into the target detection algorithm, which outputs the detected target of the steel label and its bounding box coordinate parameter set.

[0037] This paper replaces the standard convolutional structure in the network with a depthwise separable convolutional structure to reduce the number of model parameters and improve the detection speed and real-time performance in industrial embedded environments.

[0038] The improvement of the object detection algorithm in this paper lies in replacing the standard convolutional structure with a depthwise separable convolutional structure. The standard convolution and the depthwise separable convolution will be introduced separately below.

[0039] Standard convolution operation: Input a An input feature map, after The convolution kernel produces a The output feature map. If we have at this time From these feature maps, we will obtain a The output feature map. Depthwise separable convolution: Depthwise separable convolution = depthwise convolution + pointwise convolution; Depthwise separable convolution is simply splitting a regular convolution into a depthwise convolution and a pointwise convolution. Unlike standard convolutional networks, depthwise convolution splits the convolution kernel into single-channel forms, performing convolution operations on each channel without changing the depth of the input feature map. This results in an output feature map with the same number of channels as the input feature map. Input Feature maps, after After depthwise convolution, we obtain The output feature map. The dimensions of the input and output are invariant. Pointwise convolution is... Convolution, primarily used to increase or decrease the dimensionality of feature maps, yields the following results during depthwise convolution: The output feature map, we use indivual The convolution kernel performs a convolution operation on the input feature map, and the output feature map is the same as that of a standard convolution operation. This paper improves upon traditional dense networks by incorporating the cross-stage partial network CSPNet, which will be introduced below. Network Structure: The input feature map is divided into two equal parts along the channel dimension. Main Branch: Without any complex computation, it is directly passed to the end as the original feature. This preserves the original information and establishes an efficient gradient path. Convolutional Branch: It is preprocessed with a standard convolution, normalization, and non-linear activation (Mish function) operation, then enters a "residual block stacking" sequence, which stacks multiple residual blocks. Each residual block contains two standard convolution, normalization, and non-linear activation (Mish function) operation modules, and finally undergoes post-processing with an activation function. The outputs of the main branch and the convolutional branch are concatenated along the channel dimension. A final standard convolution, normalization, and non-linear activation (Mish function) operation is used as a transition layer to fuse the concatenated features and control the number of output channels. The advantages of this network structure are that the features in the backbone provide a "clean" gradient flow, which alleviates the gradient vanishing problem. At the same time, since only half of the channels (convolutional branches) participate in the expensive residual calculation, the number of floating-point operations and memory usage are significantly reduced. Therefore, this structure integrates unprocessed features and depth-transformed features, enhancing the expressive power of the features.

[0040] The method, based on bounding boxes and employing a simplified 2D-3D pose calculation process, calculates the depth information and spatial coordinates of the steel label relative to the image acquisition device, specifically including: Load the camera's internal and external parameter sets; Image features for pose determination are obtained within the bounding box; The rigid body transformation step, perspective projection step, and coordinate transformation step are executed sequentially to convert world coordinates into pixel coordinates. The depth information and spatial coordinate parameters of the label relative to the image acquisition device are calculated based on the transformation results.

[0041] Reference Figure 2 , Essentially, it calculates a transformation from world coordinates to pixel coordinates. World coordinates are the coordinates of the steel coil in the 3D world, while pixel coordinates are the coordinates of the steel coil in the pixel coordinate system. World coordinate system It's manually set. Camera coordinate system. The origin is the center of the camera lens. The optical axis is perpendicular to the image plane and passes through the center of the image. A two-dimensional image coordinate system is established with the center of the image plane as the origin. To establish a pixel coordinate system with the top left corner of the image as the origin. The transformation process consists of a rigid body transformation, a perspective transformation, and a coordinate transformation. The transformation formula allows direct calculation of the steel coil's coordinates in the 3D world based on its position in the image. The first step involves a rigid body transformation from world coordinates to camera coordinates, using a rotation matrix... Translation vector Transform world coordinates to camera coordinates, where the rotation matrix... It is to revolve the world coordinate system The axes are rotated counterclockwise so that the three coordinate axes of the world coordinate system and the camera coordinate system are oriented in the same direction. Therefore, the rotation matrix can be decomposed into three rotation transformations about the three coordinate axes. As shown in the formula below: , , ; : around Axis rotation , : around Axis rotation , : around Axis rotation Next, a translation is made between the world coordinate system and the camera coordinate system. The translation vector is... : The rigid body transformation matrix can be obtained by combining the three rotation matrices with the translation matrix. : The second step involves transforming the camera coordinates to image coordinates using perspective projection, the mathematical description of which is as follows: The third step involves a coordinate transformation to convert the image coordinates to pixel coordinates. .

[0042] The step of sending the spatial position coordinates to the control server, and the control server generating control commands for the robotic arm, specifically includes: The spatial location coordinates are sent to the control server via the network; The control server generates a set of control instructions for the robotic arm based on the received data and can coordinate with the PLC of the vehicle for scheduling. The control instruction set is used for device execution.

[0043] The step of driving the robotic arm to move to the location of the steel label according to the control command specifically includes: The control arm gimbal initiates movement according to the control command; The robotic arm is moved to the location of the steel label and positioned, completing the posture preparation for close-range recognition.

[0044] The process of capturing and recognizing images of steel labels, reading the identity information on the labels, and completing steel identity registration specifically includes: Once in place, a close-up image of the steel label is captured. The label is identified and decoded by calling a QR code recognition algorithm model based on traditional image processing; Based on the decoding results, registration record data containing tag identity information is generated, and the steel identity registration is completed.

[0045] When a label is detected to be peeling, obscured, or severely damaged, the system triggers an alarm and notifies manual intervention. When the label is in normal condition, the QR code is recognized and decoded, and the recognition result is uploaded to the warehouse management system, specifically including: The system determines the status of the labels. If the labels are found to be peeling, obscured, or severely damaged, the system will issue an alarm and notify manual intervention. If the label is in normal condition, the QR code will be recognized and decoded. The identification and decoding results are uploaded to the warehouse management system.

[0046] This embodiment also provides a system for automatic identification of intelligent steel labels based on machine vision, including: Image acquisition module: Acquires scene images of steel using an industrial camera; QR code recognition and decoding module: Recognizes the QR code on steel labels.

[0047] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0048] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for automatic identification of intelligent steel labels based on machine vision, characterized in that, include: Acquire scene image data facing the steel coil as recognition input; Image enhancement and line target detection are performed based on the scene image data to identify and obtain the bounding box of the steel QR code label; Based on the bounding box, a simplified 2D-3D pose solving process is used to calculate the spatial coordinates of the steel QR code label relative to the image acquisition device. The system captures and recognizes images of steel QR code labels located at the target spatial coordinates, reads the identity information on the steel QR code labels, uploads the results to the warehouse management system, and completes the steel identity registration.

2. The automatic identification method for intelligent steel labels based on machine vision according to claim 1, characterized in that, The acquisition of scene image data facing the steel includes: Adjust the industrial camera so that its optical axis is aligned with the center of the long axis of the steel and keeps it directly facing the camera, and then capture an image of the steel scene at this point.

3. The method for automatic identification of intelligent steel labels based on machine vision according to claim 1, characterized in that, The step of performing image enhancement and row target detection based on the scene image data to identify and obtain the bounding box of the steel label specifically includes: Image enhancement workflow: An SRGAN-based augmentation model based on generative adversarial networks was used to train the training data, and the parameters of the augmentation model were obtained through adversarial training between the generator and the discriminator. When constructing the training data, the data is expanded by simulating reflections, changes in brightness and darkness, and dust noise in industrial scenarios. During runtime, the scene image is input into the enhancement model, and the enhanced image is output. Target detection process: Adopting based on improvement A lightweight object detection algorithm that replaces the standard convolutional structure with a depthwise separable convolutional structure; The enhanced image is input into the target detection algorithm, which outputs the detected target of the steel label and its bounding box coordinate parameter set.

4. The automatic identification method for intelligent steel labels based on machine vision according to claim 3, characterized in that, The method, based on bounding boxes and employing a simplified 2D-3D pose calculation process, calculates the spatial coordinates of the steel label relative to the image acquisition device, specifically including: Load the camera's internal and external parameter sets; Image features for pose determination are obtained within the bounding box; The rigid body transformation step, perspective projection step, and coordinate transformation step are executed sequentially to convert world coordinates into pixel coordinates. The depth information and spatial coordinate parameters of the label relative to the image acquisition device are calculated based on the transformation results.

5. The automatic identification method for intelligent steel labels based on machine vision according to any one of claims 1-4, characterized in that, include: The spatial location coordinates are sent to the control server via the network; The control server generates control commands based on the received data, and uses the set of control commands to control the collection device to reach the designated space for image capture and recognition.

6. The automatic identification method for intelligent steel labels based on machine vision according to claim 5, characterized in that, The collection device is a robotic arm or a drone, which controls a camera to move to the location of the steel label and take a picture of the steel label.

7. The method for automatic identification of intelligent steel labels based on machine vision according to claim 1, characterized in that, Before capturing and recognizing the steel label located at spatial coordinates, the following steps are also included: The system determines the condition of steel labels. If the labels are found to be peeling, obscured, or severely damaged, the system will issue an alarm and notify manual intervention. If the steel QR code label is in normal condition, then the steel QR code label is identified and decoded. The identification and decoding results are uploaded to the warehouse management system.

8. The automatic identification method for intelligent steel labels based on machine vision according to claim 7, characterized in that, If the steel QR code label is in normal condition, then the identification and decoding of the steel QR code label specifically includes: Capture close-up images of the steel QR code label; The steel QR code label is identified and decoded using a QR code recognition algorithm model based on traditional image processing. Based on the decoding results, registration record data containing the steel QR code label identity information is generated, and the steel identity registration is completed.

9. A system employing the machine vision-based intelligent steel warehouse label automatic identification method as described in claim 8, characterized in that, include: Image acquisition module: Acquires scene images of steel using an industrial camera; QR code recognition and decoding module: Recognizes the QR code on steel labels.

Images