A method for identifying steel plant bale numbers

Abstract

This invention discloses a method for identifying ladle numbers in steel plants. First, by deploying surveillance cameras, switches, video recorders, and servers, images of the steel / iron ladle hoisting process within the workshop are acquired, transmitted, and stored. Then, the acquired images are labeled and trained using a YOLOv3 object detection neural network to establish a ladle lug / hook detection model and a single-digit detection model. Finally, images from the surveillance cameras are acquired at regular intervals, and for each image, the following operations are performed: first, the ladle lug / hook detection model is used to detect whether the steel / iron ladle has been dropped; if dropped, the single-digit detection model is then used to detect the digits and their coordinates in the image, and the ladle number is calculated based on the number of digits, the magnitude of the X-axis between the digits, and the confidence level. This invention can eliminate interference from other types of dropped ladle behavior and digits, reducing the recognition area and recognition time; and through deep learning neural networks, it can adapt well to the complex environment of steel workshops and extract deeper image features.

Description

Technical Field

[0001] This method belongs to the field of target detection technology and provides a method for automatically identifying the ladle number by analyzing steel / iron ladles in steel plants through video analysis. Background Technology

[0002] The ladle number is a key identifier in steel production, used to identify the information of the molten steel / iron it carries. There are three types of ladle numbers: the first type is the numerical ladle number, which is formed by welding a metal block in the shape of a number to a specific location on the ladle or by painting a number in the shape of a number on it; this is common in steel production sites. The second type is the barcode / QR code ladle number, which is formed by casting a barcode / QR code onto the ladle or by attaching a metal plate containing a barcode / QR code to the ladle. The third type is the radio frequency (RFID) ladle number, which is formed by attaching an RFID tag to the ladle that can emit the ladle number information.

[0003] There are three main types of package number recognition technologies. The first is digital package number recognition technology, which involves capturing images of the package number with a camera and building an AI model that combines the image and the number to identify the package number. The second is barcode / QR code package number recognition technology, which involves scanning the package number with a barcode scanner or capturing an image with a camera and then identifying the package number using barcode / QR code recognition. The third is radio frequency (RFID) package number recognition technology, which uses a reader to receive radio frequency signals to extract the package number information.

[0004] The advantage of the digital package number identification method is that the identification equipment is easy to install / maintain, but the disadvantage is that the identification is easily affected by ambient light interference. The other two identification methods have the advantage that the identification is not easily affected by ambient light interference, but the identification equipment is difficult to install / maintain.

[0005] Therefore, a technology is needed that allows for convenient installation / maintenance of identification equipment and is not easily affected by ambient light interference. Summary of the Invention

[0006] The purpose of this invention is to provide a method for identifying steel buns in a steel plant, which can minimize the impact of ambient light interference such as plant lights, sunlight, and overhead crane shadows on bun identification. The method is easy to install and maintain, and has a high accuracy rate.

[0007] The steel plant bale number identification method of the present invention comprises the following specific steps:

[0008] Step 1: Deploy surveillance cameras, switches, video recorders, and servers in the workshop; the surveillance cameras are deployed near the fixed position where the overhead crane lowers the steel / iron ladle, and the surveillance cameras capture clear images of the steel / iron ladle lowering process; the switches can provide PoE power to the cameras, and the cameras are connected to the switches via fiber optic cables and network cables, and the video recorders and servers are connected to the switches via network cables.

[0009] Step 2: Establish an ear / hook detection model using the YOLOv3 object detection neural network.

[0010] Step 3: Establish a single-digit detection model for package numbers from 0 to 9 using the YOLOv3 object detection neural network.

[0011] Step 4: Collect images from the monitoring camera at regular intervals; then, for each collected image, call the ear / hook detection model to detect the presence or absence of the ear or crane hook and their coordinates in the image. The judgment criterion is: for any target, if the model gives a confidence level greater than a set threshold, the target is determined to be detected; next, after simultaneously identifying the ear and crane hook, further detect their coordinates in the image to determine their relative positions; if the relative positions match the position interval of the hook hanging on the ear, it is determined that the hook is hanging on the ear, and the ordinate Y of the ear is recorded; finally, if it is found that the Y coordinate of multiple consecutive images increases continuously, it is determined that the ear has fallen.

[0012] Step 5: After detecting the steel / iron ladle being dropped, first acquire the monitoring image, then call the lug / hook detection model to determine the ladle number recognition area, then process the ladle number recognition area to make the image details clear, then use the single digit detection model to detect the digits in the image and their XY coordinates, and then calculate the ladle number based on the number of digits, the size of the X coordinates between the digits and the confidence level: if it is a single digit and the confidence level is greater than the set threshold, it is determined to be a single-digit ladle number; if it is multiple digits and the confidence level of each digit is greater than the set threshold, then arrange the digits in ascending order of their X coordinates to form a multi-digit ladle number.

[0013] Compared with the prior art, the beneficial effects of the present invention are:

[0014] 1. The present invention provides a steel plant bale number identification method that captures images of a specific area in the surveillance camera footage to identify the bale number, thereby eliminating interference from other areas of bale number loss and numbers, reducing the identification area, and reducing identification time.

[0015] 2. The steel plant lot number recognition method of the present invention performs bilateral filtering and contrast-limited adaptive histogram equalization processing, which can remove image noise and enhance image details in specific areas for lot number recognition.

[0016] 3. The steel plant bale number recognition method of the present invention, through deep learning neural network, can adapt well to the complex environment of steel workshop and extract deep image features related to bale number recognition;

[0017] 4. The steel plant bale number identification method of the present invention uses identification equipment with low cost and low on-site maintenance cost;

[0018] 5. The steel plant bale number identification method of the present invention has a high detection accuracy. Attached Figure Description

[0019] Figure 1 This is an overall flowchart of the steel plant bale number identification method of the present invention;

[0020] Figure 2 This is a block diagram of the deployment scheme of the identification equipment in the steel plant bale number identification method of the present invention;

[0021] Figure 3 This is a flowchart of the method for establishing the lug / hook detection model in the steel plant ladle number identification method of the present invention;

[0022] Figure 4 This is a flowchart of the method for establishing a single-digit detection model in the steel plant lot number identification method of the present invention;

[0023] Figure 5 This is a flowchart of the baggage drop detection method in the steel plant baggage number identification method of the present invention;

[0024] Figure 6 This is a flowchart of the package number detection method in the steel plant package number identification method of the present invention;

[0025] Figure 7 For casting and printing the package number on the outer shell of the iron ladle;

[0026] Figure 8 This is a schematic diagram of the rectangular box annotation results of the data in the image collected in the steel plant lot number identification method of the present invention;

[0027] Figure 9 This is a schematic diagram of the hook and lug results collected in the steel plant bale number identification method of the present invention;

[0028] Figure 10 A structural diagram illustrating the unit number package number detection method for steel plant packages according to the present invention;

[0029] Figure 11 This is a structural diagram of the multi-digit package number detection method for steel plant package number identification applied to this invention. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to the accompanying drawings.

[0031] The present invention provides a method for identifying steel bale numbers in steel plants, such as... Figure 1 As shown, it includes the following steps:

[0032] Step 1: Identify Device Deployment

[0033] Surveillance cameras, switches, video recorders, and servers are deployed in the steel workshop. The surveillance cameras are positioned near the fixed position where the overhead crane lowers the steel / ladies. The cameras capture images that clearly show the process of lowering the steel / ladies. The captured images must include the ladle number and lugs on the ladle as the overhead crane lowers it onto the ladle rack, the movement of the hook from top to bottom and from nothing to something, and the curved part of the ladle number / lugs and hook after it lands on the rack. The total height of the ladle number / lugs and hook must occupy more than half of the entire image height.

[0034] like Figure 2 As shown, the switch, video recorder, and server are located in the electrical room. The switch provides PoE power to the cameras, which are connected to the switch via fiber optic cables and network cables. The video recorder and server are then connected to the switch via network cables. The server and video recorder are powered on, and the switch is powered on as well. The video recorder is configured to continuously record live video.

[0035] Step 2: Establish a detection model for the lug / hook.

[0036] like Figure 3 As shown, firstly, video of the workshop was recorded for D days and nights (D is a set value). Then, the bag-dropping process was manually identified from the video (from the moment the bag ear and hook are fully displayed in the frame until the bag is completely dropped into the fixed position), for a total of L bag-dropping processes (L is a set value). Next, M images (M is a set value) were extracted from each bag-dropping process, for a total of P images. The bag ear and hook in each image were labeled with rectangular boxes, and then trained using a YOLOv3 object detection neural network to obtain a bag ear / hook detection model.

[0037] Step 3: Establish a single-digit detection model for package numbers from 0 to 9.

[0038] like Figure 4 As shown, firstly, based on the video recordings of the workshop for D days and nights in step 2, video segments within T seconds after each package unloading process are found. Then, N images (N is a set value) are extracted from the video segments. The ear / hook detection model is called to identify the X and Y coordinates of the ear target. The value obtained by subtracting V (V is a set value) from X and Y are used as the horizontal and vertical coordinates of the upper left corner of the package number recognition area, respectively. V is used as the width and height of the package number recognition area. The images are cropped to obtain the package number recognition area. Then, the package number recognition area image is cropped from each image. Next, bilateral filtering is applied to the package number recognition area image to remove environmental noise and strengthen the boundaries. Then, contrast-limited adaptive histogram equalization is used to clarify image details. The processed package number recognition area image is stored as an image. Then, rectangular boxes are labeled on the stored package number recognition area image to label each number. The single digit detection model is obtained by training the YOLOv3 object detection neural network.

[0039] Step 4: Packet Drop Detection

[0040] like Figure 5 As shown, the server first collects images from the monitoring camera at predetermined intervals. Then, for each collected image, the hook / hook detection model is called to detect the presence or absence of the hook or crane hook and their coordinates in the image. The judgment criterion is: for any target, if the model gives a confidence level greater than a set threshold, the target is determined to be detected. Next, when both the hook and the crane hook are detected simultaneously, their coordinates in the image are further detected to determine their relative positions. If their relative positions match the position interval of the hook hanging on the hook, it is determined that the hook is hanging on the hook, and the ordinate Y of the hook is recorded. Finally, if the Y coordinate of K consecutive images continuously increases (K is a set value), it is determined that the hook has fallen.

[0041] Step 5: Package Number Detection

[0042] like Figure 6 As shown, after detecting the falling of a steel / iron ladle, the server first acquires a monitoring image, then calls the ladle / hook detection model to identify the X and Y coordinates of the ladle target. Next, following the method in step 3, the ladle number identification area is obtained. Then, bilateral filtering is performed on this area to remove environmental noise and strengthen the boundary. Next, the image details are clarified by limiting the contrast adaptive histogram equalization. Then, the single-digit detection model detects the digits in the image and their X and Y coordinates. The ladle number is then calculated based on the number of digits, the size of the X coordinates between the digits, and the confidence level. If it is a single digit and the confidence level is greater than the set threshold, it is determined to be a single-digit ladle number. If it is multiple digits and the confidence level of each digit is greater than the set threshold, the digits are arranged in ascending order of their X coordinates to form a multi-digit ladle number.

[0043] Example:

[0044] The process of identifying the ladle number at a fixed location (referred to as the East Ladle Location) in a steel plant is as follows:

[0045] The package number to be identified is the numerical package number for welding, ranging from 1 to 99. The package number is located within the peripheral wall area between the small spout, upper reinforcing strip, right lug, and lower reinforcing strip of the ladle; the specific location is not fixed, as shown below. Figure 7 As shown.

[0046] As described in step 1 above, cameras, switches, video recorders, and servers are deployed in the workshop. The cameras have a resolution of 2 megapixels and a pixel depth of 32-bit true color.

[0047] Videos of the workshop were recorded for 30 days and nights. The unloading process was manually identified from the videos, totaling 600 unloading processes. Three images were then extracted from each unloading process, for a total of 1500 images. The lugs and hooks in these 1500 images were labeled with rectangular bounding boxes, and then trained using a YOLOv3 object detection neural network to obtain a lug / hook detection model.

[0048] Based on the aforementioned video, a 30-second video segment was found after each packet was dropped. One image was then extracted from each video segment, and the packet number recognition area was cropped from each image. This area was then subjected to bilateral filtering to remove environmental noise and enhance boundaries. Next, contrast-limited adaptive histogram equalization was used to clarify image details, and the resulting image was stored. Finally, rectangular bounding boxes were added to the images, labeling each number within them, such as... Figure 8 As shown; a single digit detection model is obtained by training the YOLOv3 object detection neural network;

[0049] The server acquires images from the surveillance camera every 500 milliseconds. For each acquired image, a hook / hook detection model is used to detect the presence or coordinates of either the hook or the overhead crane hook. The criterion is: for any target, if the model gives a confidence level greater than a set threshold of 0.6, the target is considered detected. Then, when both the hook and the overhead crane hook are detected simultaneously, and their relative positions match the positional interval of the hook suspending on the hook, it is determined that the hook is suspending the hook. Figure 9 As shown in the box, record the Y coordinate of the bag ear at this time; finally, if it is found that the Y coordinate of 3 consecutive images increases continuously, it is judged as a bag drop.

[0050] After detecting a dropped ladle, a monitoring image is first acquired. Then, the ladle lug / hook detection model is invoked, and V is set to 416. The ladle number identification area is obtained using the aforementioned method. Next, bilateral filtering is applied to this area to remove environmental noise and strengthen the boundaries. Then, contrast-limited adaptive histogram equalization is used to clarify image details. Finally, a single-digit detection model is used to detect the digits and their XY coordinates in the image. The ladle number is then calculated based on the number of digits, the magnitude of the X coordinates between digits, and the confidence level. If it is a single digit and the confidence level is greater than the set threshold of 0.6, it is determined to be a single-digit ladle number. Figure 10 As shown; if there are multiple numbers and the confidence level of each number is greater than a set threshold, then the numbers are arranged in ascending order of their X-coordinates to form a multi-digit package number, such as... Figure 11 As shown.

Claims

1. A method for identifying steel bale numbers, characterized in that: The steps are designed as: Step 1: deploying monitoring cameras, switches, video recorders and servers in the workshop; the monitoring cameras are deployed near the fixed positions of the overhead crane for hoisting empty steel / iron ladles, and clear hoisting processes of the ladles are collected by the monitoring cameras; the switches can provide POE power supply for the cameras, the cameras are connected to the switches through optical fiber and network cables, and the video recorders and servers are connected to the switches through network cables respectively; Step 2: establishing a ladle ear / hook detection model through a yolov3 target detection neural network; Step 3: establishing a 0~9 single digit detection model corresponding to the ladle number through a yolov3 target detection neural network; Step 4: collecting images taken by the monitoring cameras at certain intervals; then, for each collected image, calling the ladle ear / hook detection model to detect whether the ladle ear or the overhead crane hook exists and the coordinates of the two targets in the image, and the judgment standard is that for any target, when the model gives a confidence of the target greater than a set threshold, it is determined that the target is detected; then, after recognizing the ladle ear and the overhead crane hook, further detecting the coordinates of the two targets in the image to determine the relative position of the two targets; if the relative position of the two targets meets the position interval of the hook hanging on the ladle ear, it is determined that the hook is hanging the ladle ear, and the vertical coordinate Y of the ladle ear is recorded at this time; finally, if the Y coordinates of a plurality of continuous images are found to be continuously increasing, it is judged that the ladle is falling; Step 5: after detecting that the ladle is falling, first collecting the monitoring image, then calling the ladle ear / hook detection model to determine the ladle number recognition area, then processing the ladle number recognition area to make the image details clear, then detecting the numbers and their XY coordinates in the image through the single digit detection model, and then calculating the ladle number according to the number of numbers, the size of X between the numbers and the confidence: if it is a single number and the confidence of the number is greater than a set threshold, it is determined as a single number ladle number, and if it is a plurality of numbers and the confidence of each number is greater than a set threshold, the numbers are arranged in order of X coordinates from small to large to form a multi-digit ladle number. The method for calling the ladle ear / hook detection model to determine the ladle number recognition area is that the X and Y coordinates of the ladle ear / hook detected by the ladle ear / hook detection model are used as the horizontal and vertical coordinates of the upper left corner of the ladle number recognition area, V is used as the width and height of the ladle number recognition area, and the picture is cut to obtain the ladle number recognition area.

2. The method of claim 1, wherein the method further comprises: receiving a request for a list of the steel mill batch numbers; and transmitting the list of the steel mill batch numbers to the requesting device. The monitoring camera collects the arc-shaped part of the ladle number / hanging ear and hook in the image after the ladle falls on the ladle rack, and the total height of the ladle number / hanging ear and hook needs to account for more than half of the height of the whole image.

3. The method of claim 1, wherein the method further comprises: The specific method of step 2 is: first, record D days of video in the workshop, then manually find L falling processes in the video, then extract M images from each falling process, a total of P images; then, the ladle ear and hook of each image are labeled with a rectangular frame, and the ladle ear / hook detection model is obtained through the yolov3 target detection neural network. ​ 4. The method of claim 1, wherein the method further comprises: The specific method of step 3 is as follows: first, according to the recorded D-day videos of the workshop, find the video segment within T seconds after each drop process is completed, then extract N pictures from the video segment, call the bag ear / hook detection model to determine the bag number identification area, and cut out the bag number identification area image, then process the bag number identification area image to clarify the image details; then label the bag number identification area image with a rectangular frame to mark each digit in it; train through the yolov3 target detection neural network to obtain a single-digit detection model. ​ 5. The method of claim 1 or 4, wherein: The image processing method of the bag number identification area is as follows: first, bilateral filtering, and then limited contrast adaptive histogram equalization.

Images