Method and device for preprocessing for auto-labeling of data for slag removal automation

Abstract

The present disclosure provides a device for preprocessing for auto-labeling of data for slag removal automation. The device comprises: an image acquisition unit for acquiring an image of a ladle containing molten iron; a slag area determination unit for determining a slag area, which is an area in the image occupied by slag; and a coordinate adjustment unit for adjusting the number of coordinates constituting the slag area.

Description

Preprocessing method and device for data auto-labeling for slag smelting automation

[0001] The present disclosure relates to a technology for preprocessing for data auto-labeling for automation of slag removal.

[0002] If slag is mixed with molten steel during the tapping stage of the steelmaking process, the stable quality of the molten steel cannot be maintained, resulting in the production of steel with substandard quality components. Therefore, steelmaking processes typically include a slag removal device (e.g., a skimmer) to remove slag floating on top of the molten steel from the ladle.

[0003] In general, for slag removal, the slag removal device removes slag by repeatedly moving a scraping plate back and forth multiple times to scrape the slag floating on the top of the molten steel and remove it to the outside.

[0004] To automate this slag removal process, it is necessary to identify the slag area, which is the slag floating on the surface of the molten steel in the ladle. Therefore, a specific method is needed to identify the slag area and obtain accurate information about it more efficiently.

[0005] These embodiments aim to provide a technique for performing preprocessing for data auto-labeling of a slag area for automation of slag removal during a steelmaking process.

[0006] In order to solve the above-mentioned problem, one embodiment of the present disclosure is a preprocessing device for data auto-labeling for slag off automation, which includes an image acquisition unit for acquiring an image of a ladle containing molten iron, a slag area determination unit for determining a slag area, which is an area occupied by slag in the image, and a coordinate adjustment unit for adjusting the number of coordinates constituting the slag area.

[0007] In addition, one embodiment may provide a preprocessing method for data auto-labeling for slag off automation, including a step of acquiring an image of a ladle containing molten iron, a step of determining a slag area, which is an area occupied by slag in the image, and a step of adjusting the number of coordinates constituting the slag area.

[0008] According to the present embodiment, a device and method for performing preprocessing for data auto-labeling for a slag area for automating slag removal during a steelmaking process can be provided.

[0009] Additionally, by photographing the ladle area in steel mills and stainless steel (STS) mills and performing data preprocessing using computer vision-based artificial intelligence technology, we can reduce costs and perform accurate labeling in the data labeling process based on expensive semantic segmentation algorithms.

[0010] Additionally, it can provide the ability to acquire a large, more sophisticated data set by utilizing artificial intelligence from photos taken by cameras installed in steel mills or STS steel mills.

[0011] FIG. 1 is a drawing for explaining the configuration of a preprocessing device for data auto-labeling for slag removal automation according to one embodiment.

[0012] FIG. 2 is a drawing for explaining obtaining an image of molten steel contained in a ladle according to one embodiment.

[0013] FIG. 3 is a diagram for explaining determining a slag area by applying a semantic segmentation algorithm according to one embodiment.

[0014] FIGS. 4 and 5 are drawings for explaining adjusting and optimizing the number of coordinates constituting a slag region by applying a vector optimization algorithm according to one embodiment.

[0015] FIG. 6 is a diagram illustrating learning and distribution of an artificial intelligence-based algorithm applied to preprocessing for data auto-labeling for slag removal automation according to one embodiment.

[0016] FIG. 7 is a diagram illustrating a procedure of a preprocessing method for data auto-labeling for slag removal automation according to one embodiment.

[0017] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When adding reference numerals to components in each drawing, identical components may have the same numerals as much as possible even if they are shown in different drawings. In addition, when describing the present embodiments, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the technical idea of ​​the present invention, the detailed description may be omitted. When "includes," "has," "consists of," etc. are used in this specification, other parts may be added unless "only" is used. When a component is expressed in the singular, it may include a case in which the plural is included unless specifically stated otherwise.

[0018] Additionally, terms such as first, second, A, B, (a), (b), etc. may be used to describe components of the present disclosure. These terms are only intended to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by the terms.

[0019] In a description of the positional relationship of components, when it is described that two or more components are "connected," "combined," or "connected," it should be understood that the two or more components may be directly "connected," "combined," or "connected," but that the two or more components may also be further "interposed" with another component to be "connected," "combined," or "connected." Here, the other component may be included in one or more of the two or more components that are "connected," "combined," or "connected" to each other.

[0020] In the description of the temporal flow relationship related to components, operation methods, or manufacturing methods, for example, when the temporal or flow relationship is described as “after”, “following”, “next to”, “before”, etc., it may also include cases where it is not continuous, unless “immediately” or “directly” is used.

[0021] Meanwhile, when numerical values ​​or corresponding information (e.g., levels, etc.) for components are mentioned, even without separate explicit description, the numerical values ​​or corresponding information may be interpreted as including an error range that may occur due to various factors (e.g., process factors, internal or external impact, noise, etc.).

[0022] This disclosure presents a data labeling technique utilizing AI and image processing to segment steel slag. This technique captures images of a steel ladle via cameras installed in a steel mill or stainless steel mill, and uses AI to perform preprocessing before human labeling. Image processing allows segmentation of the molten iron and steel slag regions within the ladle, which can then be used for data labeling through learning.

[0023] Hereinafter, a preprocessing device for data auto-labeling for slag removal automation according to embodiments of the present disclosure will be described in detail with reference to related drawings.

[0024] FIG. 1 is a diagram illustrating the configuration of a preprocessing device for data auto-labeling for slag smelting automation according to one embodiment. FIG. 2 is a diagram illustrating obtaining an image of molten steel contained in a ladle according to one embodiment. FIG. 3 is a diagram illustrating determining a slag area by applying a semantic segmentation algorithm according to one embodiment. FIGS. 4 and 5 are diagrams illustrating adjusting and optimizing the number of coordinates constituting a slag area by applying a vector optimization algorithm according to one embodiment.

[0025] According to one embodiment, a preprocessing device (100) for data auto-labeling for slag shaving automation may include an image acquisition unit (110) that acquires an image of a ladle containing molten iron, a slag area determination unit (120) that determines a slag area, which is an area occupied by slag in the image, and a coordinate adjustment unit (130) that adjusts the number of coordinates constituting the slag area.

[0026] The image acquisition unit (110) can acquire an image of a ladle (10) including a molten steel surface (20) through an image capturing device, as illustrated in FIG. 2, during a steelmaking process in a steelmaking plant or an STS steelmaking plant. As an example, the image acquisition unit (110) can be implemented as an image capturing device such as a camera capable of acquiring image data. In this regard, as long as an image of a ladle in which a molten steel surface is visible can be acquired, the device is not limited to a specific device.

[0027] Images of a ladle including a molten steel surface acquired from an image acquisition unit (110) can be classified and stored in a database according to predetermined conditions. In addition, the images of the ladle can be applied as input data for an artificial intelligence-based algorithm used in a subsequent process.

[0028] Referring again to FIG. 1, the slag area determination unit (120) may be implemented in software, hardware, or a combination thereof in various devices capable of executing the method according to the technical idea of ​​the present disclosure, such as a processor, a computer, or other processing device.

[0029] The slag area determination unit (120) can segment the molten steel area and the slag area within the ladle in the image by applying a semantic segmentation algorithm. For example, any publicly known algorithm before or after the present disclosure can be applied to the semantic segmentation algorithm, as long as the technical concepts of the present disclosure can be substantially applied in the same manner, and is not limited to a specific algorithm.

[0030] The slag area determination unit (120) can divide each pixel of the image of the ladle including the molten steel surface into multiple sets, such as the molten steel class and the slag class, by applying a semantic segmentation algorithm. As illustrated in FIG. 3, the slag area determination unit (120) can divide the image into distinct areas, such as the skimmer head area, the slag area, the ladle spout area, and the slag runner area.

[0031] The slag area determination unit (120) can identify the location of the slag area within the ladle based on the segmented results. Furthermore, the slag area determination unit (120) can extract the coordinate values ​​of the coordinates constituting the slag area. Through this, the slag area determination unit (120) can identify the basic location of the slag.

[0032] Here, the coordinates constituting the slag region may correspond to the vertices that make up the segmented slag region. Directly labeling AI-based semantic segmentation data requires a significantly larger number of point labels than conventional object detection techniques. This is because, while data used for general object recognition requires coordinate labels for four vertices in the form of a rectangular box, semantic segmentation techniques utilize unstructured data, requiring coordinate labels for a significantly larger number of vertices. Consequently, adjustments to the number of vertices are necessary.

[0033] Referring back to FIG. 1, the coordinate adjustment unit (130) may be implemented in software, hardware, or a combination thereof in various devices capable of executing the method according to the technical idea of ​​the present disclosure, such as a processor, a computer, or other processing devices. In FIG. 1, according to an example, the slag area determination unit (120) and the coordinate adjustment unit (130) are illustrated as separate components, but this is not limited thereto. According to another example, the slag area determination unit (120) and the coordinate adjustment unit (130) may be implemented as a single component, such as a control unit.

[0034] The coordinate adjustment unit (130) can adjust the number of coordinates constituting the slag region by applying a vector optimization algorithm. In this case, according to an example, the vector optimization algorithm may include the Douglas-Peucker algorithm. However, this is merely an example, and various algorithms known before and after the present disclosure may be applied as the vector optimization algorithm, as long as the technical concepts of the present disclosure can be substantially equally applied.

[0035] The coordinate adjustment unit (130) can adjust the coordinates of the extracted slag region by utilizing the Douglas-Peucker algorithm for the processing result of the semantic segmentation algorithm. The coordinate adjustment unit (130) can approximate the line segments constituting the curve forming the segmented slag region with the minimum number of points by applying the Douglas-Peucker algorithm, as illustrated in FIG. 4. That is, the coordinate adjustment unit (130) can first find the two farthest points among all points on the curve and create a line segment including them. Thereafter, the coordinate adjustment unit (130) can find the point that is the farthest from the created line segment and divide the curve into two sub-curves based on this point. The coordinate adjustment unit (130) can approximate the curve by repeating the above process for the sub-curves thus divided. By repeating this process recursively, the line segments constituting the slag region can ultimately be approximated with the minimum number of points.

[0036] In addition, the coordinate adjustment unit (130) can optimize the number of coordinates constituting the slag region by adjusting the tolerance value of the Douglas-Peucker algorithm. That is, the coordinate adjustment unit (130) can find an optimal value capable of expressing the slag region, as illustrated in FIG. 5, by adjusting the tolerance value of the Douglas-Peucker algorithm, and can optimize the number of coordinates accordingly.

[0037] Here, the tolerance value refers to the maximum distance used when approximating the aforementioned curve. To optimize the number of coordinates for the slag region, the coordinate adjustment unit (130) can optimize the number of coordinates by merging the point cloud data of the coordinates while modifying the tolerance value. Accordingly, the optimal number of coordinates for representing the slag region can be derived.

[0038] For example, the preprocessing device (100) for data auto-labeling may further include a labeling unit that performs coordinate labeling with corresponding coordinate values ​​for each of an optimized number of coordinates. That is, the labeling unit may label each coordinate value for the optimized number of coordinates. Accordingly, the data labeler may provide the corresponding coordinate values ​​to a labeling tool for use.

[0039] FIG. 6 is a diagram illustrating learning and distribution of an artificial intelligence-based algorithm applied to preprocessing for data auto-labeling for slag removal automation according to one embodiment.

[0040] Referring to Figure 6, data manually labeled for slag regions in the acquired ladle image can be applied as input to the AI-based algorithm of the present disclosure. Learning can be performed based on these input values, and auto-labeled data can be acquired based on the learned results. The auto-labeled data can then be applied as input again to train the AI-based algorithm.

[0041] Based on these learning results, if the accuracy of the AI-based algorithm reaches the target level, the algorithm can be deployed.

[0042] Accordingly, a device and method for performing preprocessing for data auto-labeling of the slag area to automate slag removal during the steelmaking process can be provided. Furthermore, by utilizing computer vision-based artificial intelligence technology to perform data preprocessing on the slag area in the ladle area photographed during the steelmaking process, a more efficient data labeling method can be provided.

[0043] Hereinafter, a preprocessing method for data auto-labeling for slag removal automation, which can perform some or all of the embodiments described with reference to FIGS. 1 through 6, will be described with reference to the drawings. The above description may be omitted to avoid redundant explanation, and in this case, the omitted content may be substantially equally applied to the following description, as long as it does not contradict the technical spirit of the invention.

[0044] FIG. 7 is a diagram illustrating a procedure of a preprocessing method for data auto-labeling for slag removal automation according to one embodiment.

[0045] Referring to Fig. 7, a preprocessing device for data auto-labeling can obtain an image of a ladle containing molten iron (S710).

[0046] A preprocessing device for data auto-labeling can acquire images of a ladle containing the molten steel surface through an image capture device during the steelmaking process at a steel mill or stainless steel mill. The acquired images of the ladle containing the molten steel surface can be classified and stored in a database according to predetermined conditions. Furthermore, the ladle images can be used as input data for an artificial intelligence-based algorithm used in subsequent processes.

[0047] Referring again to FIG. 7, the preprocessing device for data auto-labeling can determine a slag area, which is an area occupied by slag in an image (S720).

[0048] A preprocessing device for data auto-labeling can segment the molten steel region and slag region within a ladle from an image using a semantic segmentation algorithm. The preprocessing device for data auto-labeling can segment each pixel in an image of a ladle containing a molten steel surface into multiple classes, such as a molten steel class and a slag class, using a semantic segmentation algorithm.

[0049] The preprocessing device for data auto-labeling can identify the location of the slag area within the ladle based on the segmented results. Furthermore, the preprocessing device for data auto-labeling can extract the coordinate values ​​of the coordinates constituting the slag area. Through this, the preprocessing device for data auto-labeling can identify the basic location of the slag.

[0050] Referring again to FIG. 7, the preprocessing device for data auto-labeling can adjust the number of coordinates constituting the slag area (S730).

[0051] A preprocessing device for data auto-labeling can adjust the number of coordinates constituting the slag region by applying a vector optimization algorithm. In this case, as an example, the vector optimization algorithm may include the Douglas-Peucker algorithm.

[0052] A preprocessing device for data auto-labeling can adjust the coordinates of the extracted slag region using the Douglas-Peucker algorithm based on the processing results of the semantic segmentation algorithm. The preprocessing device for data auto-labeling can approximate the line segments forming the curve forming the segmented slag region with a minimum number of points by applying the Douglas-Peucker algorithm. Specifically, the preprocessing device for data auto-labeling can first find the two most distant points among all points on the curve and create a line segment containing them. Then, the preprocessing device for data auto-labeling can find the point furthest from the generated line segment and split the curve into two sub-curves based on this point. The preprocessing device for data auto-labeling can approximate the curve by repeating the above process for the segmented sub-curves in this way. By recursively repeating this process, the line segments forming the slag region can ultimately be approximated with a minimum number of points.

[0053] In addition, the preprocessing device for data auto-labeling can optimize the number of coordinates constituting the slag region by adjusting the tolerance value of the Douglas-Peucker algorithm. In other words, the preprocessing device for data auto-labeling can find the optimal value for expressing the slag region by adjusting the tolerance value of the Douglas-Peucker algorithm, and accordingly optimize the number of coordinates.

[0054] Here, the tolerance value refers to the maximum distance used when approximating the aforementioned curve. To optimize the number of coordinates for the slag region, the preprocessing device for data auto-labeling can merge point cloud data while modifying the tolerance value to optimize the number of coordinates. Accordingly, the optimal number of coordinates for representing the slag region can be derived.

[0055] A preprocessing device for data auto-labeling can label each coordinate value for an optimized number of coordinates. Accordingly, the data labeler can provide the coordinate values ​​for use by the labeling tool.

[0056] Accordingly, a device and method for performing preprocessing for data auto-labeling of the slag area to automate slag removal during the steelmaking process can be provided. Furthermore, by utilizing computer vision-based artificial intelligence technology to perform data preprocessing on the slag area in the ladle area photographed during the steelmaking process, a more efficient data labeling method can be provided.

[0057] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0058] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0059] The above-described methods and / or various embodiments may be realized by digital electronic circuits, computer hardware, firmware, software, and / or a combination thereof. Various embodiments of the present disclosure may be implemented as a computer program that is executed by a data processing device, for example, one or more programmable processors and / or one or more computing devices, or stored on a computer-readable recording medium and / or a computer-readable recording medium. The above-described computer program may be written in any form of programming language, including a compiled language or an interpreted language, and may be distributed in any form, such as a standalone program, a module, a subroutine, etc. The computer program may be distributed through a single computing device, multiple computing devices connected through the same network, and / or multiple computing devices distributed to be connected through multiple different networks.

[0060] The methods and / or various embodiments described above may be performed by one or more processors configured to execute one or more computer programs that process, store, and / or manage any function, function, etc. by operating on the basis of input data or generating output data. For example, the methods and / or various embodiments of the present disclosure may be performed by special purpose logic circuits such as Field Programmable Gate Arrays (FPGAs) or Application Specific Integrated Circuits (ASICs), and an apparatus and / or system for performing the methods and / or embodiments of the present disclosure may be implemented as special purpose logic circuits such as FPGAs or ASICs.

[0061] The one or more processors executing the computer program may include a general-purpose or special-purpose microprocessor and / or one or more processors of any type of digital computing device. The processor may receive instructions and / or data from each of read-only memory and random-access memory, or may receive instructions and / or data from the read-only memory and the random-access memory. In the present disclosure, components of a computing device performing the methods and / or embodiments may include one or more processors for executing instructions, and one or more memory devices for storing instructions and / or data.

[0062] According to one embodiment, the computing device can transmit and receive data to and from one or more mass storage devices for storing data. For example, the computing device can receive and / or transfer data from a magnetic disc or an optical disc. A computer-readable storage medium suitable for storing instructions and / or data associated with a computer program may include, but is not limited to, any form of non-volatile memory, including semiconductor memory devices such as Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), flash memory devices, and the like. For example, the computer-readable storage medium may include a magnetic disk such as an internal hard disk or a removable disk, a magneto-optical disk, a CD-ROM, and a DVD-ROM disk.

[0063] To provide interaction with a user, a computing device may include, but is not limited to, a display device (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), etc.) for providing or displaying information to the user, and a pointing device (e.g., a keyboard, a mouse, a trackball, etc.) for allowing the user to provide input and / or commands to the computing device. That is, the computing device may further include any other types of devices for providing interaction with the user. For example, the computing device may provide any form of sensory feedback to the user, including visual feedback, auditory feedback, and / or tactile feedback, for interaction with the user. In this regard, the user may provide input to the computing device through various gestures, such as visual, vocal, or motion.

[0064] In the present disclosure, various embodiments may be implemented in a computing system that includes a backend component (e.g., a data server), a middleware component (e.g., an application server), and / or a front-end component. In this case, the components may be interconnected via any form or medium of digital data communication, such as a communications network. For example, the communications network may include a Local Area Network (LAN), a Wide Area Network (WAN), and the like.

[0065] A computing device based on the present embodiments may be implemented using hardware and / or software configured to interact with a user, including a user device, a user interface (UI) device, a user terminal, or a client device. For example, the computing device may include a portable computing device such as a laptop computer. Additionally or alternatively, the computing device may include, but is not limited to, Personal Digital Assistants (PDAs), tablet PCs, game consoles, wearable devices, Internet of Things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. The computing device may further include other types of devices configured to interact with a user. In addition, the computing device may include a portable communication device (e.g., a mobile phone, a smart phone, a wireless cellular phone, etc.) suitable for wireless communication over a network such as a mobile communication network. The computing device may be configured to communicate wirelessly with a network server using wireless communication technologies and / or protocols, such as Radio Frequency (RF), Microwave Frequency (MWF), and / or Infrared Ray Frequency (IRF).

[0066] The above description is merely an illustrative example of the technical idea of ​​the present disclosure, and those skilled in the art to which the present disclosure pertains will appreciate that various modifications and variations can be made without departing from the essential characteristics of the technical idea of ​​the present disclosure. In addition, the present embodiments are not intended to limit the technical idea of ​​the present disclosure but rather to explain it, and therefore the scope of the technical idea of ​​the present disclosure is not limited by these embodiments. The scope of protection of the present disclosure should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included within the scope of the rights of the present disclosure.

[0067]

[0068] CROSS-REFERENCE TO RELATED APPLICATION

[0069] This patent application claims priority under 35 USC § 119(a) to Korean Patent Application No. 10-2023-0183259, filed December 15, 2023, the entire contents of which are incorporated herein by reference. Furthermore, this patent application claims priority in countries other than the United States for the same reasons, the entire contents of which are incorporated herein by reference.

Claims

1. In a preprocessing device for data auto-labeling for slag off automation, An image acquisition unit for acquiring an image of a ladle containing molten iron; A slag area determination unit for determining the slag area, which is the area occupied by slag in the image above; and A coordinate adjustment unit for adjusting the number of coordinates constituting the above slag area; A preprocessing device for data auto-labeling including:

2. In paragraph 1, The above slag area judgment part is, A preprocessing device for data auto-labeling that segments the molten steel region and the slag region in the ladle in the image by applying a semantic segmentation algorithm.

3. In paragraph 2, The above slag area judgment part is, A preprocessing device for data auto-labeling that extracts coordinate values ​​of coordinates constituting the above slag area.

4. In paragraph 3, The above coordinate adjustment part, A preprocessing device for data auto-labeling that adjusts the number of coordinates constituting the slag area by applying a vector optimization algorithm.

5. In paragraph 4, The above vector optimization algorithm includes the Douglas-Peucker algorithm, The above coordinate adjustment part, A preprocessing device for data auto-labeling that optimizes the number of coordinates constituting the slag area by adjusting the tolerance value of the Douglas-Peucker algorithm.

6. In paragraph 1, A preprocessing device for data auto-labeling, further comprising a labeling unit that performs coordinate labeling with corresponding coordinate values ​​for each of the above-mentioned optimized number of coordinates.

7. In a preprocessing method for data auto-labeling for slag off automation, Step of obtaining an image of a ladle containing molten iron; A step of determining the slag area, which is the area occupied by slag in the above image; and A step of adjusting the number of coordinates constituting the above slag area; A preprocessing method for data auto-labeling including .

8. In paragraph 7, The step of determining the above slag area is: A preprocessing method for data auto-labeling that segments the molten steel region and the slag region in the ladle in the image by applying a semantic segmentation algorithm.

9. In paragraph 8, The step of determining the above slag area is: A preprocessing method for data auto-labeling that extracts coordinate values ​​of coordinates corresponding to the above slag area.

10. In paragraph 7, The step of adjusting the number of the above coordinates is: A preprocessing method for data auto-labeling that adjusts the number of coordinates corresponding to the slag area by applying a vector optimization algorithm.

11. In Article 10, The above vector optimization algorithm includes the Douglas-Peucker algorithm, A preprocessing method for data auto-labeling in which the step of adjusting the number of the above coordinates optimizes the number of coordinates corresponding to the slag area by adjusting the tolerance value of the Douglas-Peucker algorithm.

12. In paragraph 7, A preprocessing method for data auto-labeling, further comprising: a labeling step of performing coordinate labeling with corresponding coordinate values ​​for each of the above-mentioned optimized number of coordinates;

Images