Full-process material automatic tracking method and system for smelting production line

Abstract

The invention discloses a smelting production line full-process material automatic tracking method. The method comprises the steps that S1, video images shot by a camera assembly on a production site when cast materials enter a smelting production line are obtained; obtaining production number information of the material through a production system; and identifying the material in the video image through the neural network model, and binding the production number information with the material. And S2, identifying materials in the video image through a neural network model, and extracting image features of the materials. And S3, determining the same material in two continuous frames through the neural network model. And S4, the materials are tracked within the shooting range of each camera. Image features are shared between the cameras to track the material across the cameras. The position information of the materials is determined while the materials are tracked, and the working procedures experienced by the materials and the working procedure experiencing time are recorded according to the position information. The camera and the neural network model are adopted to track the materials, recording is accurate, and the management efficiency is high.

Description

Technical Field

[0001] The present invention relates to the technical field of material identification and tracking in a smelting process, and in particular to a method and system for automatic tracking of materials in the entire process of a smelting production line. Background Art

[0002] Vacuum induction furnace is a kind of equipment used for metal smelting. It uses the principle of induction heating to heat the metal material in the induction coil to high temperature and melt through electromagnetic induction. After the composition, inclusions, temperature, etc. meet the requirements, it is molded. Vacuum induction furnace is widely used in metallurgical production, and is often used in the production of stainless steel, cobalt-based alloys, nickel-based alloys and other materials.

[0003] At present, material tracking of the vacuum induction furnace smelting production line includes processes such as vacuum induction furnace smelting, casting, demolding, slow cooling, and warehousing. Material management of the vacuum induction smelting production line, such as demolding time, slow cooling start and end time, and storage location in the warehouse, mostly relies on manual tracking and manual recording. This method is prone to problems such as untimely recording and human errors, resulting in inaccurate material records and low management efficiency. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present application provides a full-process material automatic tracking system for a smelting production line.

[0005] The method for automatic material tracking of the entire process of a smelting production line provided in this application adopts the following technical solutions:

[0006] A method for automatically tracking materials in the entire process of a smelting production line, the method comprising:

[0007] S1. Obtain a video image of a material that has completed casting and enters a smelting production line, which is captured by a camera assembly at a production site; obtain production number information of the material through a production system; identify the material in the video image through a neural network model, and bind the production number information to the material;

[0008] S2. Identify the material in each frame of the video image through a neural network model, and extract image features of the material;

[0009] S3, calculating the similarity of the image features in two consecutive frames of the video image by using the neural network model, and determining the same material in two consecutive frames by measuring the similarity;

[0010] S4, the camera assembly includes a plurality of cameras; performing step S3 on the video image captured by each of the cameras to track the material within the shooting range of each of the cameras;

[0011] Sharing the image features between the cameras so that when the material disappears from the field of view of one camera and simultaneously appears in the field of view of another camera, it can be immediately identified and continued to be tracked;

[0012] While tracking the material, the location information of the material is determined, and the process that the material has gone through and the time of going through the process are recorded according to the location information.

[0013] Optionally, the obtaining of the production number information of the material includes: after the casting of the material is completed, the production system automatically generates a production number information corresponding to the material.

[0014] Optionally, the neural network model includes a VisionTransformer algorithm module; in step S2, the material in the video image taken by each of the cameras is detected by the VisionTransformer algorithm module, and the position of the material and the image features are output.

[0015] Optionally, the VisionTransformer algorithm module includes an image segmentation submodule, an embedding layer, a position encoding and a Transformer encoder; the image segmentation submodule is used to divide any frame image in the video image into a number of small blocks; the embedding layer and the position encoding are used to convert the several small blocks into PatchEmbeddings and map them to a high-dimensional space, and input them into the Transformer encoder, thereby realizing the extraction of the image features.

[0016] Optionally, the neural network model also includes a DeepSORT algorithm module; in step S3, the DeepSORT algorithm module is used to associate the position information and image features of the material in two consecutive frames of the video image in real time, thereby realizing the tracking of the material within the field of view of a single camera.

[0017] Optionally, the DeepSORT algorithm module extracts the image features of the material in any frame of the video image through YOLOv5, segments the material with a detection frame, and then uses a Kalman filter algorithm to predict the state of the detection frame in the next frame, and then matches the prediction result with the image features in the next frame through the Hungarian algorithm. In this way, the position information and the image features of the material in two consecutive frames of the video image can be associated in real time to achieve tracking of the material within the field of view of a single camera.

[0018] Optionally, in step S4, the overlapping area of ​​the fields of view of the two cameras is calculated through the position relationship mapping model; when one of the materials enters the overlapping area, the first position of the material in the field of view of one of the two cameras constituting the overlapping area is obtained; the theoretical second position of the material in the field of view of the other camera is calculated through the position relationship mapping model and the first position, and the theoretical second position is compared with the actual second position of the material measured by the other camera; when the difference between the actual second position and the theoretical second position is less than a set threshold, the image features of the material are shared between the two cameras constituting the overlapping area to achieve continuous tracking of the material.

[0019] Optionally, the cameras in the camera assembly are arranged at different positions of the production site so that the field of view of the cameras covers the entire production site.

[0020] Based on the above concept, the present application also provides an automatic material tracking system for the entire process of a smelting production line, which tracks materials based on the above method; it includes a data processing module, a camera assembly and an output module; wherein the data processing module includes a neural network submodule, and is communicatively connected to the output module and the production system; the camera assembly includes multiple cameras; the multiple cameras are used to capture video images of the production site; the neural network submodule is used to calculate the similarity of the image features in two consecutive frames of the video image, and by measuring the similarity, determine the same material in two consecutive frames; the neural network submodule is also used to share the image features between the cameras, so that when the material disappears from the field of view of one camera and the material appears in the field of view of another camera at the same time, it can be immediately identified and continued to be tracked; and while tracking the material, determine the location information of the material, and record the process the material has experienced and the time it has experienced the process based on the location information.

[0021] Optionally, the neural network module includes a VisionTransformer algorithm unit and a DeepSORT algorithm unit; the VisionTransformer algorithm unit is used to detect the material in the video image taken by each of the cameras, and output the position and image features of the material; the DeepSORT algorithm unit is used to associate the position information and image features of the material in two consecutive frames of the video image in real time, thereby realizing the tracking of the material within the field of view of a single camera.

[0022] As described above, the method for automatic material tracking in the whole process of the smelting production line of the present application has at least the following beneficial effects:

[0023] 1. Multiple cameras are arranged to capture video images of the production site, and the materials in each frame of the video image are identified through a neural network model to track the materials in the field of view of a single camera. After linking multiple cameras, the tracking of materials in the production site is realized. This effectively overcomes the problem that the material management of the vacuum induction smelting production line in the prior art, such as demoulding time, slow cooling start and end time, storage location in the warehouse, etc., relies on manual operation, is prone to human errors, and leads to inaccurate records and low management efficiency. The camera is used throughout the process in conjunction with the neural network model to track the materials, with accurate records and high management efficiency.

[0024] 2. In the process of tracking materials, through cooperation with the production system, the status of materials can be monitored in real time, and the flow and processing of materials can be understood in time, thereby improving the timeliness and effectiveness of decision-making, and helping to improve the efficiency of management and scheduling.

[0025] 3. Centralized processing of data from the production site through cameras and neural network models eliminates information islands and effectively integrates data from different links of the production site for comprehensive utilization and management analysis, which helps improve production efficiency.

[0026] 4. By tracking the materials and recording the processes they go through and the time they go through, detailed traceability information can be provided when quality problems occur during the production process, reducing the difficulty of quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart corresponding to the method of automatic material tracking throughout the entire process of the smelting production line.

[0028] Figure 2 It is a schematic diagram used to exemplarily show the arrangement of cameras from a top-down perspective.

[0029] Figure 3 This is the architecture diagram of the full-process material automatic tracking system for the smelting production line. DETAILED DESCRIPTION

[0030] The following is an explanation of the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the features in the following embodiments and the embodiments can be combined with each other without conflict. It should also be understood that the terms used in the embodiments of the present invention are intended to describe specific embodiments, rather than to limit the scope of protection of the present invention. The test methods for which specific conditions are not specified in the following examples are usually carried out under conventional conditions or according to the conditions recommended by the manufacturers.

[0031] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention, so they have no substantial technical significance. Any modification of the structure, change in the proportional relationship, or adjustment of the size should still fall within the scope of the technical content disclosed by the present invention without affecting the effects and purposes that can be achieved by the present invention. At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" quoted in this specification are only for the convenience of description, and are not used to limit the scope of the implementation of the present invention. The change or adjustment of their relative relationship should also be regarded as the scope of the implementation of the present invention without substantially changing the technical content.

[0032] Please refer to Figure 1-2 The present application discloses a method for automatically tracking materials in the entire process of a smelting production line, the method comprising:

[0033] S1. Obtain a video image of the finished casting material entering the smelting production line captured by a camera assembly at the production site. Obtain the production number information of the material through the production system. Identify the material in the video image through a neural network model, and bind the production number information to the material.

[0034] The production system is the production management system, whose main functions are to plan production, control quality and cost management, as well as track and manage the logistics and information flow in the production process, and provide timely feedback on the execution of the plan to achieve two-way coordination between planning and execution.

[0035] After the material is cast, it has to go through multiple processes such as demoulding, heat treatment, and storage. Therefore, in order to facilitate the tracking of each newly cast material at the production site, when the material is cast, the production system automatically generates a production number corresponding to the material and binds the production number information to the material. At the production site, no matter what process the material goes through, its production number information always remains unchanged.

[0036] S2. Identify the material in each frame of the video image through a neural network model and extract the image features of the material.

[0037] Traditional algorithms require manual design of complex feature extraction logic, while deep learning algorithms can automatically learn the most relevant features from data without manual intervention. Therefore, this application uses a neural network model for feature extraction.

[0038] The neural network model includes the VisionTransformer algorithm module. The function of the VisionTransformer algorithm module is to detect the target (material) in the video image collected by each camera through the VisionTransformer algorithm, and output the position and detection frame of the material to achieve the overall contour detection and positioning of the material. Compared with the traditional convolutional neural network CNN algorithm, the VisionTransformer algorithm can better detect the overall contour of the material and reduce the interference of light and background debris. The output of the algorithm adopts a multitask approach and supports multiple output formats such as output classification, quality regression, target detection frame, semantic segmentation, etc., which is more adaptable to various application scenarios.

[0039] Specifically, the implementation principle of the VisionTransformer algorithm module is as follows: the VisionTransformer algorithm module detects the materials in the video images taken by each camera, and outputs the location and image features of the materials. The VisionTransformer algorithm module includes an image segmentation submodule, an embedding layer, a position encoding, and a Transformer encoder. The image segmentation submodule is used to divide any frame of the video image into several small blocks. The embedding layer and position encoding are used to convert several small blocks into Patch Embeddings and map them to a high-dimensional space, and input them into the Transformer encoder to extract image features. The image feature is the detection frame of the material. While extracting the detection frame, the location information of the material can also be easily determined.

[0040] In some other embodiments of the present application, the VisionTransformer algorithm may also be replaced with other suitable algorithms.

[0041] S3, calculating the similarity of image features in two consecutive frames of the video image through a neural network model, and determining the same material in the two consecutive frames by measuring the similarity;

[0042] In order to realize continuous tracking of materials, it is necessary to introduce a tracking mechanism based on the target detection in step S3 to track the position information of the same material in different frames of the video image.

[0043] Therefore, the neural network model also includes the DeepSORT algorithm module. The DeepSORT algorithm module associates the location information and image features of the material in two consecutive frames of the video image in real time, thereby realizing the tracking of the material within the field of view of a single camera. The DeepSORT algorithm introduces deep learning features and cascade matching into the SORT algorithm to improve tracking performance, while maintaining high tracking accuracy and having good real-time performance.

[0044] Specifically, the DeepSORT algorithm module uses YOLOv5 to extract the image features of the material in any frame of the video image, and uses the detection frame to segment the material. It then uses the Kalman filter algorithm to predict the state of the detection frame in the next frame, and then uses the Hungarian algorithm to match the prediction result with the image features in the next frame. The location information and image features of the material in two consecutive frames of the video image can be associated in real time, thereby realizing the tracking of the material within the field of view of a single camera.

[0045] The role of the Kalman filter algorithm is to predict the coordinates and speed of the object's position from a limited, noisy, observation sequence of the object's position. Therefore, the Kalman filter algorithm can predict the state of the detection box in the next frame.

[0046] The Hungarian algorithm is a combinatorial optimization algorithm for solving the task allocation problem in polynomial time. In multi-target tracking, its main purpose is to match multiple targets between frames, including the appearance of new targets, the disappearance of old targets, and the matching of the target ID (production number information in this application) of the previous frame and the current frame. After the production number information is matched, the location information and image features of the material are also associated in real time, thereby realizing the tracking of the material within the field of view of a single camera.

[0047] S4, the camera assembly includes a plurality of cameras; executing step S3 for the video image captured by each camera to track the material within the shooting range of each camera; sharing image features between cameras so that when the material disappears from the field of view of one camera and appears in the field of view of another camera at the same time, it can be immediately identified and continued to be tracked; while tracking the material, determining the location information of the material, and recording the process the material has gone through and the time of going through the process according to the location information.

[0048] In order to enable the camera's field of view to cover the entire production site, cameras in the camera assembly are arranged at different locations of the production site.

[0049] Specifically, Figure 2 An exemplary arrangement of cameras is shown from a top-down perspective. In the figure, cam1-cam8 represent eight different cameras respectively. The triangular area emitted from the camera represents the field of view of the camera. When the material is in the slow cooling zone, it is in the field of view of cam8 and cam7. When the material leaves the slow cooling zone and passes through the two induction furnaces on the left, the demoulding area, the induction furnace on the right and the finished product area in turn, the material will pass through cam6, cam5, cam4, cam3, cam2, and cam1 in turn. In this way, the field of view of the camera covers the entire production site, thereby avoiding the interruption of material tracking due to blind spots in the field of view of the camera.

[0050] Specifically, step S4 calculates the overlapping area of ​​the fields of view of the two cameras through the position relationship mapping model. When a material enters the overlapping area, the first position of the material in the field of view of one of the two cameras constituting the overlapping area is obtained. The theoretical second position of the material in the field of view of the other camera is calculated through the position relationship mapping model and the first position, and the theoretical second position is compared with the actual second position of the material measured by the other camera. When the difference between the actual second position and the theoretical second position is less than the set threshold, the image features of the material are shared between the two cameras constituting the overlapping area to achieve continuous tracking of the material.

[0051] In addition, the production system, cameras, and neural network models all use a unified timestamp. The information that a material disappears from the field of view of one camera and appears in the field of view of another camera at the same timestamp can also assist in associating materials across cameras.

[0052] While implementing step S4, the production number information and corresponding location information of the material can be transmitted to the production system in real time and displayed. The recorded process and time of the material can also be uploaded to the production system and automatically displayed while displaying the production number information of the material.

[0053] In summary, the method for automatic material tracking in the whole process of the smelting production line of the present application has the following advantages:

[0054] 1. Multiple cameras are arranged to capture video images of the production site, and the materials in each frame of the video image are identified through a neural network model to track the materials in the field of view of a single camera. After linking multiple cameras, the tracking of materials in the production site is realized. This effectively overcomes the problem that the material management of the vacuum induction smelting production line in the prior art, such as demoulding time, slow cooling start and end time, storage location in the warehouse, etc., relies on manual operation, is prone to human errors, and leads to inaccurate records and low management efficiency. The camera is used throughout the process in conjunction with the neural network model to track the materials, with accurate records and high management efficiency.

[0055] 2. In the process of tracking materials, through cooperation with the production system, the status of materials can be monitored in real time, and the flow and processing of materials can be understood in time, thereby improving the timeliness and effectiveness of decision-making, and helping to improve the efficiency of management and scheduling.

[0056] 3. Centralized processing of data from the production site through cameras and neural network models eliminates information islands and effectively integrates data from different links of the production site for comprehensive utilization and management analysis, which helps improve production efficiency.

[0057] 4. By tracking the materials and recording the processes they go through and the time they go through, detailed traceability information can be provided when quality problems occur during the production process, reducing the difficulty of quality control.

[0058] On the other hand, please refer to Figure 3 The present application also discloses a full-process material automatic tracking system for a smelting production line, which tracks materials based on the above method. It includes a data processing module, a camera component and an output module.

[0059] The data processing module includes a neural network submodule, and is communicatively connected with the output module and the production system. The camera assembly includes multiple cameras. The multiple cameras are used to capture video images of the production site. The neural network submodule is used to calculate the similarity of image features in two consecutive frames of the video image, and determine the same materials in two consecutive frames by measuring the similarity. The neural network submodule is also used to share image features between cameras, so that when the material disappears from the field of view of one camera and the material appears in the field of view of another camera at the same time, it can be immediately identified and continued to be tracked. And while tracking the material, determine the location information of the material, and record the process the material has gone through and the time it has gone through the process based on the location information.

[0060] Specifically, the neural network module includes a VisionTransformer algorithm unit and a DeepSORT algorithm unit. The VisionTransformer algorithm unit is used to detect materials in the video images taken by each camera and output the location and image features of the materials. The DeepSORT algorithm unit is used to associate the location information and image features of the materials in two consecutive frames of the video image in real time, thereby realizing the tracking of the materials within the field of view of a single camera.

[0061] The full-process material automatic tracking system of the smelting production line of this application also has the following advantages:

[0062] 1. Multiple cameras are arranged to capture video images of the production site, and the materials in each frame of the video image are identified through a neural network model to track the materials in the field of view of a single camera. After linking multiple cameras, the tracking of materials in the production site is realized. This effectively overcomes the problem that the material management of the vacuum induction smelting production line in the prior art, such as demoulding time, slow cooling start and end time, storage location in the warehouse, etc., relies on manual operation, is prone to human errors, and leads to inaccurate records and low management efficiency. The camera is used throughout the process in conjunction with the neural network model to track the materials, with accurate records and high management efficiency.

[0063] 2. In the process of tracking materials, through cooperation with the production system, the status of materials can be monitored in real time, and the flow and processing of materials can be understood in time, thereby improving the timeliness and effectiveness of decision-making, and helping to improve the efficiency of management and scheduling.

[0064] 3. Centralized processing of data from the production site through cameras and neural network models eliminates information islands and effectively integrates data from different links of the production site for comprehensive utilization and management analysis, which helps improve production efficiency.

[0065] 4. By tracking the materials and recording the processes they go through and the time they go through, detailed traceability information can be provided when quality problems occur during the production process, reducing the difficulty of quality control.

[0066] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for automatically tracking materials in the entire process of a smelting production line, characterized in that: The method comprises: S1. Obtain a video image of a cast material entering a smelting production line, captured by a camera assembly at a production site; obtain production number information of the material through a production system; identify the material in the video image using a neural network model, and bind the production number information to the material; S2. Identify the material in each frame of the video image through a neural network model and extract image features of the material; S3. Calculating the similarity of the image features in two consecutive frames of the video image using the neural network model, and determining the same material in the two consecutive frames by measuring the similarity; S4, the camera assembly includes a plurality of cameras; performing step S3 on the video image captured by each camera to track the material within the shooting range of each camera; Sharing the image features between the cameras so that when the material disappears from the field of view of one camera and simultaneously appears in the field of view of another camera, it can be immediately identified and continued to be tracked; While tracking the material, the location information of the material is determined, and the process the material has gone through and the time it has gone through the process are recorded based on the location information.

2. The method according to claim 1, characterized in that Acquiring the production number information of the material includes: when the material casting is completed, the production system automatically generates a production number information corresponding to the material.

3. The method according to claim 1, wherein: The neural network model includes a VisionTransformer algorithm module; The VisionTransformer algorithm module detects the material in the video image captured by each camera, and outputs the position of the material and the image features.

4. The method according to claim 3, wherein: The VisionTransformer algorithm module includes an image segmentation submodule, an embedding layer, a position encoding, and a Transformer encoder; The image segmentation submodule is used to segment any frame of the video image into a plurality of small blocks; The embedding layer and the position encoding are used to convert the plurality of small blocks into Patch Embeddings and map them to a high-dimensional space, and input them into the Transformer encoder, thereby realizing the extraction of the image features.

5. The method according to claim 3, wherein: The neural network model also includes a DeepSORT algorithm module; in step S3, The DeepSORT algorithm module is used to correlate the position information and image features of the material in two consecutive frames of the video image in real time, thereby enabling tracking of the material within the field of view of a single camera.

6. The method according to claim 5, characterized in that: The DeepSORT algorithm module extracts the image features of the material in any frame of the video image through YOLOv5, segments the material with a detection frame, and then uses the Kalman filter algorithm to predict the state of the detection frame in the next frame. The prediction result is then matched with the image features in the next frame through the Hungarian algorithm. In this way, the position information and image features of the material in two consecutive frames of the video image can be associated in real time, thereby realizing the tracking of the material within the field of view of a single camera.

7. The method according to claim 1, wherein: In step S4, Calculating the overlapping area of ​​the two camera fields of view by using a position relationship mapping model; When one of the materials enters the overlapping area, obtaining a first position of the material in a field of view of one of the two cameras constituting the overlapping area; Calculating a theoretical second position of the material in the field of view of another camera using the position relationship mapping model and the first position, and comparing the theoretical second position with the actual second position of the material measured by the other camera; When the difference between the actual second position and the theoretical second position is less than a set threshold, the image features of the material are shared between the two cameras constituting the overlapping area to achieve continuous tracking of the material.

8. The method according to claim 1, wherein: The cameras in the camera assembly are arranged at different positions on the production site so that the field of view of the cameras covers the entire production site.

9. An automatic tracking system for materials in the entire process of a smelting production line, which tracks materials based on the method according to any one of claims 1 to 8; characterized in that: It includes a data processing module, a camera component and an output module; wherein, The data processing module includes a neural network submodule and is in communication with the output module and the production system; The camera assembly includes a plurality of cameras; the plurality of cameras are used to capture video images of the production site; The neural network submodule is used to calculate the similarity of the image features in two consecutive frames of the video image, and determine the same material in the two consecutive frames by measuring the similarity; The neural network submodule is also used to share the image features between the cameras so that when the material disappears from the field of view of one camera and appears in the field of view of another camera at the same time, it can be immediately identified and tracked; and while tracking the material, the location information of the material is determined, and the process that the material has gone through and the time it has gone through the process are recorded based on the location information.

10. The full-process material automatic tracking system for a smelting production line according to claim 9 is characterized in that: The neural network submodule includes a VisionTransformer algorithm unit and a DeepSORT algorithm unit; the VisionTransformer algorithm unit is used to detect the material in the video image captured by each camera and output the location of the material and the image features; The DeepSORT algorithm unit is used to associate the position information and the image features of the material in two consecutive frames of the video image in real time, thereby realizing tracking of the material within the field of view of a single camera.

Images