A hot-rolled rail material tracking method and system based on machine vision

By using industrial cameras on steel production lines combined with deep learning algorithms, a fast and accurate steel rail stamp recognition model was developed, which solved the problem of low recognition accuracy in the existing technology, and achieved efficient material tracking and identification, suitable for steel rails in steel enterprises Rolling production line.

CN113643340BActive Publication Date: 2024-11-05青岛东方融智数字科技有限公司
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202110938197.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-16
Publication Date
2024-11-05
Estimated Expiration
2041-08-16

AI Technical Summary

Technical Problem

Existing machine vision algorithms are difficult to meet the accuracy requirements of steel companies for the identification of rail stamps. Traditional logic tracking methods are difficult to trace when encountering problems, and the recognition accuracy is less than 10%.

Method used

Using machine vision-based hot-rolled rail material tracking method, mark numbers are identified through industrial cameras, combined with traditional algorithms and deep learning algorithms, fast and accurate rail mark recognition models are developed, and model compression is used using knowledge distillation, and lightweight neural network is adopted. Structure and multi-classification network architecture, cancel border regression calculations, and improve recognition speed and efficiency.

Benefits of technology

It improves the accuracy and efficiency of rail stamp recognition, achieves a high recognition rate on lightweight models, is suitable for edge computing equipment, and meets the material tracking needs of rail rolling production lines.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113643340B_ABST
    Figure CN113643340B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for tracking hot-rolled rail materials based on machine vision, comprising: acquiring surface image data of the hot-rolled rail; performing feature extraction on the surface image data to obtain position information of rail stamp characters and center line information between characters; inputting the position information of rail stamp characters and center line information between characters into a preset tracking model to obtain tracking results of the hot-rolled rail materials; wherein the tracking model is compressed by knowledge distillation; the present invention adds an industrial camera to the area of ​​the hot-rolled rail finishing area that is difficult to track, and realizes the tracking of the rail material by identifying the stamp number through machine vision; and develops a fast and accurate rail stamp recognition model based on the characteristics of the collected rail stamps and combining traditional algorithms with deep learning algorithms.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and in particular, relates to a hot-rolled rail material tracking method and system based on machine vision. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] With the development of railway construction and the continuous improvement of rail production technology, the demand for rails is increasing. In order to improve the output and quality of rails, rolling mills need to trace every link of rail production. At the same time, material tracking is very critical for production scheduling in the rolling area, which can greatly increase the output of rails. The process characteristics of rail rolling are long production line length, a large number of required production equipment, and complex production paths. Therefore, on the rolling production line, material tracking of the manufacturing execution process, as well as post-production process analysis and production management are of vital importance to the entire rail rolling production. Through material tracking, the full dynamic display of production process data and automatic recording of the processing process can be realized.

[0004] Manufacturing technology is gradually developing from automation, digitization and networking to intelligence. At present, the steel industry is in a period of transformation in its development strategy, and intelligent manufacturing is an important means to achieve the transformation of the steel industry. Traditional material tracking of steel rolling lines mainly adopts logical tracking. When a problem occurs in a certain link, the logic begins to become chaotic and difficult to trace.

[0005] The difficulty in rail stamp number recognition lies in the large difference in character quality between rail stamp number recognition and traditional optical character recognition. The inventors found that existing machine vision algorithms are difficult to meet the recognition accuracy requirements of steel companies. After testing, the accuracy of existing open source high-precision OCR models applied to rail stamp recognition is less than 10%. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a hot-rolled rail material tracking method and system based on machine vision; the present invention adds an industrial camera to the area where the hot-rolled rail finishing area is difficult to track, and realizes the tracking of the rail material by identifying the steel stamp number through machine vision; according to the characteristics of the collected rail steel stamp, a fast and accurate rail steel stamp recognition model is developed by combining traditional algorithms and deep learning algorithms.

[0007] In order to achieve the above objectives, the present disclosure provides a method for tracking hot-rolled rail materials based on machine vision, which adopts the following technical solutions:

[0008] A hot-rolled rail material tracking method based on machine vision, comprising:

[0009] Acquire surface image data of the hot-rolled rail; perform feature extraction on the surface image data to obtain position information of rail stamped characters and center line information between characters;

[0010] Input the position information of the steel stamp characters on the rail and the center line information between the characters into a preset tracking model to obtain the tracking result of the hot-rolled rail material;

[0011] Wherein, the tracking model is compressed through knowledge distillation.

[0012] Further, the position information of the rail stamp characters and the center line information between the characters are divided into a training set, a test set and a validation set; and data enhancement processing is performed on the training set;

[0013] The training set after the enhancement processing is input into the neural network for training to obtain a preset tracking model;

[0014] The test set and the verification set are input into the compressed hot-rolled rail material tracking model to test and verify the tracking model.

[0015] Furthermore, the neural network is a lightweight neural network structure, and some network nodes are modified according to the characteristics of the steel stamp collected image.

[0016] Furthermore, knowledge distillation compression includes:

[0017] A teacher model is trained on the training set using a large network architecture of resnet101 to guide the training of the student model on the data set;

[0018] Use the student model to train an initial model on the training set, combine the student model with the teacher model into a new network, and output the predicted distribution of the student model and the teacher model respectively; the new network outputs the predicted distribution of the student model and the teacher model respectively;

[0019] Fix the gradient of the entire network of the teacher model, perform normal backpropagation on the student model, and start distillation model training.

[0020] Furthermore, the acquisition position of the surface image data is determined according to the rail rolling production process, and the surface image is acquired by line scanning.

[0021] Furthermore, the image is cropped to a uniform size according to the character position and a preset tracking model input size.

[0022] Furthermore, the center lines and center line spacing between characters are used as annotation boxes, and a steel stamp character dataset is formed through manual annotation.

[0023] In order to achieve the above-mentioned purpose, in a second aspect, the present disclosure further provides a hot-rolled rail material tracking system based on machine vision, which adopts the following technical solutions:

[0024] A hot-rolled rail material tracking system based on machine vision, comprising a data acquisition module, a feature extraction module and a material tracking module;

[0025] The data acquisition module is configured to: acquire surface image data of the hot-rolled rail;

[0026] The feature extraction module is configured to: perform feature extraction on the surface image data to obtain position information of rail stamped characters and center line information between characters;

[0027] The material tracking module is configured to: input the position information of the steel stamp characters on the rail and the center line information between the characters into a preset tracking model to obtain the tracking result of the hot-rolled rail material;

[0028] Wherein, the tracking model is compressed through knowledge distillation.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present disclosure utilizes the characteristics of fixed character positions, same size and same spacing, and adopts traditional algorithms to locate characters, thereby improving recognition efficiency;

[0031] 2. The present invention adopts a multi-classification network architecture instead of a multi-target detection network architecture, which eliminates a large amount of calculations for bounding box regression, speeds up training time, and increases recognition speed;

[0032] 3. The present invention adopts a model compression method of knowledge distillation, so that a high recognition rate can be achieved on a lightweight model. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings in the specification that constitute a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments of this embodiment and their descriptions are used to explain this embodiment and do not constitute improper limitations on this embodiment.

[0034] Figure 1 This is a flowchart of the implementation of Example 1 of the present disclosure;

[0035] Figure 2 A schematic diagram of generating a data set of an original picture of Embodiment 1 of the present disclosure with a size of 1600*256;

[0036] Figure 3 A schematic diagram of generating a data set of an original picture of embodiment 1 of the present disclosure with a size of 1200*256;

[0037] Figure 4This is a neural network architecture diagram of Example 1 of the present disclosure. DETAILED DESCRIPTION

[0038] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0040] Embodiment 1:

[0041] like Figure 1 As shown, this embodiment provides a method for hot rolling rail materials based on machine vision, which specifically includes the following contents:

[0042] Determine the data collection plan: Determine the appropriate installation location, camera type, light source, camera protection fixture design and triggering method of the industrial camera according to the rail rolling production process and the on-site environment; Specifically, determine the appropriate installation location of the industrial camera according to the rail rolling production process, select a line scan camera for the industrial camera, and add a light source. Trigger it through a laser photoelectric switch. When a rail passes through the photoelectric switch, the camera is triggered to collect images of the rail surface.

[0043] Preparation of steel stamp data set: In this embodiment, a large number of rail surface images are collected on site, and cropped and annotated to form a self-owned steel stamp character data set, and the data set is divided into a training set, a test set, and a validation set according to a ratio of 6:2:2; Specifically, a large number of rail surface images are collected on site. Since the steel stamp characters are equidistant and the character style is fixed, traditional filtering, binarization, and feature extraction algorithms are used to extract the character positions and the center lines between the characters; Figure 2 and Figure 3 As shown in the figure, according to the character position and the pre-designed neural network input size, the image is cropped to a uniform size, specifically 224*960. The center line and center line spacing between characters are used as annotation boxes, and the types are manually annotated to form a stamped character dataset. Finally, the dataset is divided into training set, test set, and validation set in a ratio of 6:2:2.

[0044] Training set enhancement: Determine the factors that may affect the recognition of steel stamps based on the process and pictures collected on site, and then confirm the data enhancement scheme to increase the robustness of the later training model; specifically, since the collected rail surface images contain not only steel stamp characters, but also surface defects such as scratches that are close to the characters, these defects may cause false detections. In addition, the printer may have blurred characters when printing, so it is necessary to increase the robustness of the later training model through data enhancement; preferably, the data enhancement methods used in this embodiment are mainly Cutout and CutMix.

[0045] Neural network design: Figure 4 As shown, due to the high production speed of the rolling line, the recognition algorithm needs to ensure accuracy while the recognition speed needs to be faster than the production rhythm. Therefore, a lightweight neural network structure is required for application implementation. In this embodiment, the GhosttNet neural network architecture is adopted and some network nodes are modified according to the characteristics of the steel stamp collection image; Figure 4 In the figure, G-bneck indicates the Ghost bottleneck, #exp indicates the expansion size, #out indicates the number of output channels, and #SE indicates whether the SE module is used.

[0046] Knowledge distillation compression model: Use the teacher model to guide the student model to learn specific tasks, ensuring that the small model can achieve a relatively large performance improvement while the number of parameters remains unchanged, and even obtains accuracy indicators similar to those of the large model; the compressed model can run well in edge computing devices without computing graphics cards on site, and the recognition speed and recognition rate can be guaranteed; specifically, to ensure the recognition accuracy of the network model we designed ourselves, we first need to use the large network architecture of resnet101 to train a teacher model on the training set we made. The modified nodes of the resnet network are the same as above. The teacher model is used to guide the training of the student model GhosttNet on this data set. Then use GhosttNet to train an initial model on the training set, and then we combine the student model and the teacher model into a new network, which outputs the predicted distribution of the student model and the teacher model respectively. At the same time, the gradient of the entire network of the teacher model is fixed, and the student model can do normal back propagation to start distillation model training. This can ensure that our modified lightweight model GhosttNet can achieve a relatively large performance improvement while the number of parameters remains unchanged, and even obtain accuracy indicators similar to those of the large model.

[0047] Implemented 2:

[0048] This embodiment proposes a hot-rolled rail material tracking system based on machine vision, including a data acquisition module, a feature extraction module and a material tracking module;

[0049] The data acquisition module is configured to: acquire surface image data of the hot-rolled rail;

[0050] The feature extraction module is configured to: perform feature extraction on the surface image data to obtain position information of rail stamped characters and center line information between characters;

[0051] The material tracking module is configured to: input the position information of the steel stamp characters on the rail and the center line information between the characters into a preset tracking model to obtain the tracking result of the hot-rolled rail material;

[0052] Wherein, the tracking model is compressed through knowledge distillation.

[0053] In this embodiment, the hot-rolled rail material tracking system is integrated with the automated logic tracking system to collect the location, temperature, and furnace batch number information of the material throughout the entire process. Specifically, the visual recognition system is integrated with the automated logic tracking system to form a steel-rolled material tracking system, which is capable of collecting information such as the location, temperature, and furnace batch number of the material throughout the entire process. The system is deployed on the on-site edge computing device through docker deployment.

[0054] Among them, the automated logic tracking system refers to the cooling bed area of ​​the steel rolling line, etc., which meets the first-in-first-out logic rule of a single rail, calculates the entry and delivery time of a single rail through the timing logic of automated control, and then deduces the steel grade of the rail to achieve logical tracking.

[0055] Embodiment 3:

[0056] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for tracking hot-rolled rail materials based on machine vision described in Embodiment 1 is implemented.

[0057] Embodiment 4:

[0058] This embodiment provides an edge computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the machine vision-based hot-rolled rail material tracking method described in Example 1 is implemented.

[0059] The above description is only a preferred embodiment of the present embodiment and is not intended to limit the present embodiment. For those skilled in the art, the present embodiment may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.

Claims

1. A hot-rolled rail material tracking method based on machine vision, It is characterized in that include: Acquiring surface image data of hot-rolled rails; Extracting features from the surface image data to obtain position information of rail stamped characters and center line information between characters; Input the position information of the steel stamp characters on the rail and the center line information between the characters into a preset tracking model to obtain the tracking result of the hot-rolled rail material; Wherein, the tracking model is compressed by knowledge distillation; Inputting the position information of the steel stamp characters on the rail and the center line information between the characters into a preset tracking model to obtain the tracking result of the hot-rolled rail material, including: dividing the position information of the steel stamp characters on the rail and the center line information between the characters into a training set, a test set and a verification set; performing data enhancement processing on the training set; inputting the enhanced training set into a neural network for training to obtain a preset tracking model; inputting the test set and the verification set into a compressed hot-rolled rail material tracking model to test and verify the tracking model; The content of knowledge distillation compression includes: training a teacher model on the training set using the large network architecture of resnet101 to guide the training of the student model on the data set; using the student model to train an initial model on the training set, combining the student model and the teacher model into a new network, and outputting the predicted distribution of the student model and the teacher model respectively; the new network outputs the predicted distribution of the student model and the teacher model respectively; fixing the gradient of the entire network of the teacher model, the student model performs normal back propagation, and starts distillation model training.

2. A method for tracking hot-rolled rail material based on machine vision as claimed in claim 1, It is characterized in that The neural network is a lightweight neural network structure, and some network nodes are modified according to the characteristics of the steel stamp collection image.

3. A method for tracking hot-rolled rail material based on machine vision as claimed in claim 1, It is characterized in that The acquisition position of the surface image data is determined according to the rail rolling production process, and the surface image is acquired by line scanning.

4. A method for tracking hot-rolled rail material based on machine vision as claimed in claim 1, It is characterized in that The image is cropped to a uniform size based on the character position and the preset tracking model input size.

5. A method for tracking hot-rolled rail material based on machine vision as claimed in claim 1, It is characterized in that The center lines and center line spacing between characters are used as annotation boxes, and a steel stamp character dataset is formed through manual annotation.

6. A hot-rolled rail material tracking system based on machine vision, It is characterized in that It includes data acquisition module, feature extraction module and material tracking module; The data acquisition module is configured to: acquire surface image data of the hot-rolled rail; The feature extraction module is configured to: perform feature extraction on the surface image data to obtain position information of rail stamped characters and center line information between characters; The material tracking module is configured to: input the position information of the steel stamp characters on the rail and the center line information between the characters into a preset tracking model to obtain the tracking result of the hot-rolled rail material; Wherein, the tracking model is compressed by knowledge distillation; Inputting the position information of the steel stamp characters on the rail and the center line information between the characters into a preset tracking model to obtain the tracking result of the hot-rolled rail material, including: dividing the position information of the steel stamp characters on the rail and the center line information between the characters into a training set, a test set and a verification set; performing data enhancement processing on the training set; inputting the enhanced training set into a neural network for training to obtain a preset tracking model; inputting the test set and the verification set into a compressed hot-rolled rail material tracking model to test and verify the tracking model; The content of knowledge distillation compression includes: training a teacher model on the training set using the large network architecture of resnet101 to guide the training of the student model on the data set; using the student model to train an initial model on the training set, combining the student model and the teacher model into a new network, and outputting the predicted distribution of the student model and the teacher model respectively; the new network outputs the predicted distribution of the student model and the teacher model respectively; fixing the gradient of the entire network of the teacher model, the student model performs normal back propagation, and starts distillation model training.

7. A computer-readable storage medium having a computer program stored thereon for calculating fingerprint similarity, It is characterized in that When the program is executed by a processor, a method for tracking hot-rolled rail materials based on machine vision as described in any one of claims 1 to 5 is implemented.

8. An edge computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, a hot-rolled rail material tracking method based on machine vision as described in any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Simple steel seal digital automatic identification method with high accuracy rate

    CN102426649A

  • Steel rail tread defect recognition method based on combination of gray image and depth image

    CN108921846A

  • BP neural network-based steel seal character recognition method

    CN110929713A

  • Steel coil number recognition method, system and device based on machine vision and storage medium

    CN111259908A

  • Steel code recognition system and method

    CN111353494A