How to operate an electric furnace

Abstract

To provide a moving object identification device and a moving object identification method capable of accurately identifying the shape of a moving object.SOLUTION: A moving object identification device according to the present invention includes: a preprocessing unit that converts video data of a moving object into a two-dimensional data matrix; an arithmetic unit that obtains a sparse matrix by performing robust principal component analysis on the two-dimensional data matrix; a binarized image processing unit that generates a binarized image by binarizing element values of the sparse matrix with a predetermined threshold; a first image modification unit that generates a modified binarized image by performing a morphological transform that expands a region of a predetermined pixel value with respect to the binarized image and then contracts the region of the predetermined pixel value; a second image modification unit that generates a mask image by performing a morphological transform that contracts the region of the predetermined pixel value with respect to the modified binarized image and then expands the region of the predetermined pixel value; and a mask processing unit that extracts, from the video data of the moving object, an image corresponding to the mask image as an image of the moving object.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a moving object identification device, a moving object identification method, and an electric furnace operation method. [Background technology]

[0002] In the steel industry, the use of scrap iron is expected to increase in order to reduce CO2 emissions. Scrap iron is steel-based products such as discarded automobiles, rails, and building materials, as well as iron generated during the steelmaking process, and it comes in various shapes such as plates, bars, and blocks. Most scrap iron is melted in an electric furnace using arc discharge, and various composition adjustments are made before it is processed into the desired shape and reused. In an electric furnace, since scrap iron comes in various shapes, it is expected that the operating state will change depending on the shape of the scrap iron. For example, if the scrap iron is thin and plate-shaped, it takes a short time to reach the melting temperature and melts early, so it is necessary to suppress the heat input. In contrast, if the scrap iron is block-shaped, it takes a long time for the heat to be transferred to the center, so it is necessary to heat it up for a long time. Scrap iron is sometimes separated by shape, but scrap iron of various shapes is often mixed together. For this reason, in order to operate more efficiently, a technology is needed to recognize the shape of the scrap iron charged into the electric furnace and control the operating state of the electric furnace according to the shape of the scrap iron. Here, as a method for identifying the shape of scrap iron, a method for identifying the shape of scrap iron from an image of the scrap iron charged into an electric furnace is effective. Specifically, as a method for identifying the shape of scrap iron from an image of the scrap iron, it is possible to use image segmentation as described in Patent Document 1. In image segmentation, a teacher image in which an object is shown and a label image in which a label according to the shape of the object is attached to the pixel in which the object is shown in the teacher image are prepared, and the shape of the object is model-learned using the teacher image and the label image. Then, the shape of the object is identified by using the learned model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-61065 [Non-patent literature]

[0004] [Non-Patent Document 1] EMMANUEL J. Candes, XIAODONG LI, YI MA, JOHN WRIGHT, Journal of the ACM, Vol.58, No3, Article 11 Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is difficult to identify the shape of scrap iron using image segmentation in images of scrap iron captured in factories with electric furnaces. More specifically, in factories with electric furnaces, a large amount of scrap iron is stored in a scrap yard. The vicinity of a charging cart that transports scrap iron to be loaded into an electric furnace, which is a desirable location for capturing images of scrap iron, is often close to the scrap yard, so the scrap iron in the scrap yard is reflected in the background image. Since the scrap iron reflected in the background image and the scrap iron in the charging cart are the same scrap iron, they cannot be distinguished by image segmentation. Therefore, even if image segmentation is used, the shape of the scrap iron cannot be properly identified, and there is a possibility that the shape of the scrap iron may be erroneously detected.

[0006] The present invention has been made to solve the above problems, and an object of the present invention is to provide a moving object identification device and a moving object identification method capable of identifying the shape of a moving object with high accuracy. Another object of the present invention is to provide an electric furnace operating method capable of operating the electric furnace efficiently. [Means for solving the problem]

[0007] The moving object identification device of the present invention is characterized in that it comprises a pre-processing unit that converts video data of a moving object into a two-dimensional data matrix, a calculation unit that obtains a sparse matrix by performing robust principal component analysis on the two-dimensional data matrix, a binary image processing unit that generates a binary image by binarizing element values ​​of the sparse matrix with a predetermined threshold, a first image correction unit that generates a modified binary image by performing a morphological transformation on the binary image to expand an area of ​​a predetermined pixel value and then contract the area of ​​the predetermined pixel value, a second image correction unit that generates a mask image by performing a morphological transformation on the modified binary image to contract an area of ​​a predetermined pixel value and then expand the area of ​​the predetermined pixel value, and a mask processing unit that extracts an image corresponding to the mask image from the video data of the moving object as an image of the moving object.

[0008] The moving object identification method of the present invention is characterized by including the steps of converting video data of a moving object into a two-dimensional data matrix, obtaining a sparse matrix by performing robust principal component analysis on the two-dimensional data matrix, generating a binary image by binarizing element values ​​of the sparse matrix with a predetermined threshold, generating a modified binary image by performing a morphological transformation on the binary image to expand an area of ​​a predetermined pixel value and then contract the area of ​​the predetermined pixel value, generating a mask image by performing a morphological transformation on the modified binary image to contract an area of ​​a predetermined pixel value and then expand the area of ​​the predetermined pixel value, and extracting an image corresponding to the mask image from the video data of the moving object as an image of the moving object.

[0009] The method for operating an electric furnace according to the present invention is characterized by including a step of identifying the shape of scrap iron using the moving object identification method according to the present invention and controlling the electric furnace in accordance with the identified shape. Effect of the Invention

[0010] According to the moving object identification device and the moving object identification method of the present invention, the shape of a moving object can be identified with high accuracy. Also, according to the electric furnace operation method of the present invention, the electric furnace can be operated efficiently. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a scrap iron recycling plant to which a shape identification system according to one embodiment of the present invention is applied. [Diagram 2] FIG. 2 is a block diagram showing the configuration of a shape identification system according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a flowchart showing the flow of the shape identification process according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram illustrating an example of a histogram of sparse matrix values. [Diagram 5] FIG. 5 is a diagram for explaining the shape identification process in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, a shape identification system according to an embodiment of the present invention will be described with reference to the drawings. Note that, although this embodiment is for identifying the shape of scrap iron, the object to be identified is not limited to scrap iron, and may be any moving object. In addition, the shape identification system may be installed anywhere as long as it can capture video of the moving object.

[0013] [Configuration of recycling plant] First, with reference to FIG. 1, a configuration of a scrap iron recycling plant to which a shape identification system according to an embodiment of the present invention is applied will be described.

[0014] Fig. 1 is a schematic diagram showing a scrap iron recycling plant to which a shape identification system according to one embodiment of the present invention is applied. As shown in Fig. 1, in the scrap iron recycling plant to which a shape identification system according to one embodiment of the present invention is applied, scrap iron S of various shapes is stored in a scrap yard 100, and the scrap iron S is transported into a charging cart 102 by a transport facility 101 such as a crane. The charging cart 102 then loads the scrap iron S therein into an electric furnace 103, which melts and reuses the scrap iron S loaded therein.

[0015] [Configuration of shape recognition system] Next, the configuration of a shape identification system according to an embodiment of the present invention will be described with reference to FIG.

[0016] Fig. 2 is a block diagram showing the configuration of a shape identification system according to one embodiment of the present invention. The shape identification system according to one embodiment of the present invention is an information processing system that captures a video of scrap iron S being transported into a charging cart 102 by a transport facility 101 and identifies the shape of the scrap iron S from the captured video. As shown in Fig. 2, the shape identification system 1 according to one embodiment of the present invention includes a photographing device 2, an input device 3, an output device 4, and an image processing device 5.

[0017] The photographing device 2 is composed of a video photographing device such as a video camera, and is installed in a position (see FIG. 1) where it can photograph video of the scrap iron S being transported into the loading cart 102 by the transport equipment 101. The photographing device 2 photographs video including images of the scrap iron S being transported into the loading cart 102 by the transport equipment 101, and outputs the photographed video data to the image processing device 5.

[0018] The input device 3 is constituted by input devices such as a keyboard, a mouse pointer, a touch pen, etc., and inputs operation input information to the image processing device 5.

[0019] The output device 4 is composed of output devices such as a liquid crystal display, a printer, and a speaker, and outputs various information in accordance with an output control signal from the image processing device 5.

[0020] The image processing device 5 is composed of an information processing device such as a personal computer or a workstation, and functions as a preprocessing unit 5a, a calculation unit 5b, a binary image processing unit 5c, a first image correction unit 5d, a second image correction unit 5e, and a mask processing unit 5f by executing a computer program with an arithmetic processing unit such as a CPU (Central Processing Unit) in the information processing device. The functions of each of these units will be described later.

[0021] Such a shape identification system 1 accurately identifies the shape of the scrap by executing the shape identification process described below. Hereinafter, the operation of the shape identification system 1 when executing the shape identification process will be described with reference to the flowchart shown in FIG.

[0022] [Shape Identification Processing] Fig. 3 is a flowchart showing the flow of the shape recognition process according to one embodiment of the present invention. The shape recognition process shown in Fig. 3 starts when an execution command for the shape recognition process is input to the image processing device 5 via the input device 3, and the shape recognition process proceeds to step S1.

[0023] In the process of step S1, the photographing device 2 photographs a video including an image of the scrap iron S being transported into the charging cart 102 by the transport equipment 101, and outputs the photographed video data to the image processing device 5. The video data includes image data of the scrap iron S, which is a moving object, and image data of a stationary background. This completes the process of step S1, and the shape identification process proceeds to the process of step S2.

[0024] In the process of step S2, the preprocessing unit 5a converts the video data captured in the process of step S1 into grayscale video data and creates a two-dimensional data matrix from the grayscale video data. Specifically, the video data captured in the process of step S1 may be color video data or grayscale video data. In the case of color video data, the preprocessing unit 5a converts the video data into grayscale video data in which each pixel has one luminance value. In addition, since the video data captured in the process of step S1 may include images that do not include scrap iron S, the preprocessing unit 5a extracts video data for a period specified by the input device 3 or the like as processing period video data. Since the processing period video data is a three-dimensional matrix of two-dimensional images arranged in a chronological order, the preprocessing unit 5a converts the two-dimensional image into a one-dimensional image and converts the processing period video data, which is a three-dimensional matrix, into a two-dimensional data matrix in which one-dimensional images are arranged in a chronological order. The method of converting a two-dimensional image into a one-dimensional image does not matter as long as it is converted according to a conversion rule with a restorable regularity, such as arranging pixel values ​​in the row direction from the top of the two-dimensional image, and then arranging the pixel values ​​below that in the row below. With this, the process of step S2 is completed, and the shape identification process proceeds to the process of step S3.

[0025] In the process of step S3, the calculation unit 5b performs robust principal component analysis described in Non-Patent Document 1 on the data matrix of the two-dimensional matrix obtained by the process of step S2 to obtain a sparse matrix. In detail, principal component analysis is a method for obtaining a direction (principal component vector) and principal components with maximum variance for a data matrix, and is a method for obtaining features from data. In the principal component analysis, for a data matrix containing outliers, the outliers increase the variance, so that principal component vectors and principal components that are greatly influenced by the outliers are obtained. In contrast, robust principal component analysis is a principal component analysis with enhanced resistance to outliers, and is a method for calculating a low-rank matrix including principal components by removing the outliers as a sparse matrix.

[0026] In the shape identification process of this embodiment, a sparse matrix indicating an outlier is used. That is, in a video in which scrap iron S appears in the background, the background appears in every image, so the background can be regarded as a principal component. On the other hand, since the scrap iron S moves relative to the background, it may or may not be present in each pixel. Therefore, when viewed from the background, the scrap iron S can be regarded as an outlier. That is, the sparse matrix obtained by robust principal component analysis of the data matrix indicates the scrap iron S. The robust principal component analysis algorithm used in this embodiment is a method called Principal Component Pursuit described in Non-Patent Document 1, but any other method may be used as long as it can detect outliers in the principal components. In the sparse matrix obtained by robust principal component analysis of the data matrix, the elements indicating the background are 0, and the elements indicating the scrap iron S are non-zero. This completes the process of step S3, and the shape identification process proceeds to the process of step S4.

[0027] In the process of step S4, the binary image processing unit 5c creates a binary image by binarizing the sparse matrix indicating the scrap iron S obtained in the process of step S3 using a predetermined threshold value. Specifically, when each element value of the sparse matrix is ​​plotted as a histogram, it has a mountain-shaped distribution with 0 at the apex as shown in FIG. 4. As described above, in the sparse matrix, 0 indicates the background, and values ​​other than 0 indicate the scrap iron S. Therefore, in order to clearly distinguish between the background and the scrap iron S, the element value of the sparse matrix corresponding to the i pixel of the captured image is binarized as S i The sparse matrix is ​​binarized according to the conditional expression shown in the following formula (1) with a threshold value a.

[0028]

number

[0029] For example, the binary image processing unit 5c creates a binary matrix by setting the element value of the sparse matrix that satisfies the condition shown in the formula (1) to 1 and the element value of the sparse matrix that does not satisfy the condition shown in the formula (1) to 0. In this embodiment, the element value of the sparse matrix indicating the scrap iron S is set to 1 and the element value of the background is set to 0, but as long as the scrap iron S and the background can be clearly distinguished from each other, any numerical value can be used for the distinction. In addition, the threshold value a in the formula (1) may be set within a range of 0.1σ to 3σ by calculating the standard deviation σ of the sparse matrix. Then, the binary image processing unit 5c converts the binary matrix into a binary image by performing the opposite operation to the conversion rule that converted the two-dimensional image into a one-dimensional image in the preprocessing unit 5a. This completes the process of step S4, and the shape identification process proceeds to the process of step S5.

[0030] In the process of step S5, the first image correction unit 5d performs morphological transformation on the binary image created by the process of step S4 to connect pixel values ​​1 indicating scrap iron S, thereby creating a corrected binary image. Specifically, in the binary image created by the process of step S4, pixel values ​​1 indicating scrap iron S may not be connected to each other and may be missing, or points that are not scrap iron S may have pixel value 1 as noise. For this reason, the first image correction unit 5d performs morphological transformation on the binary image. Here, morphological transformation is an image processing method that expands and contracts a certain pixel value area in an image to restore missing areas and remove noise. The first image correction unit 5d expands the pixel value 1 area based on the kernel matrix, and then contracts it to fill in missing areas of pixel values ​​1 indicating scrap iron S, thereby processing the area including scrap iron S to have pixel value 1. The kernel matrix is ​​a matrix that indicates the size at which dilation and contraction are performed, and if the kernel matrix is ​​large, regions with pixel value 1 in distant regions are connected by dilation. The size of the kernel matrix should be a size that compensates for missing parts and does not connect noise regions together. Specifically, the size of the kernel matrix should be between a 3×3 square matrix and an 11×11 square matrix. In this process, dilation and contraction operations are also performed on the parts with pixel value 1 that become noise, but in the case of noise, since the parts with pixel value 1 are each far away from each other, the noise regions are not connected to each other. This completes the process of step S5, and the shape identification process proceeds to the process of step S6.

[0031] In the process of step S6, the second image correction unit 5e performs morphological transformation on the corrected binary image created in the process of step S5, first contracting the pixel value 1 region and then expanding it, thereby creating a mask image. When the operation of contracting the pixel value 1 region based on the size of the kernel matrix is ​​performed, scattered pixel value 1 points are removed. When the region is then expanded, the region of large pixel value 1 other than noise is restored. The size of the kernel matrix is ​​preferably a size that does not remove the pixel value 1 region other than noise during contraction, and specifically, it is preferably set between the size of a 3×3 square matrix and the size of an 11×11 square matrix, similar to the size of the kernel matrix used in the process of step S5. This completes the process of step S6, and the shape identification process proceeds to the process of step S7.

[0032] In the process of step S7, the mask processing unit 5f extracts an image of the scrap iron S from the video data captured by the photographing device 2 using the mask image created in the process of step S6. Specifically, the mask processing unit 5f uses the mask image corresponding to the target frame of the photographed image to perform a masking operation in which the pixels of the photographed image corresponding to the positions of pixel value 1 in the mask image are filled with the pixel value 0 in the mask image, while the pixels of the photographed image corresponding to the positions of numerical value 0 in the mask image are filled with the pixel value 0, thereby extracting the image of the scrap iron S. Thereafter, if the extracted image is applied to a machine learning model that has learned the shape of the scrap iron in advance by semantic segmentation, instance segmentation, or the like, the shape of the scrap iron can be identified with high accuracy. In addition, the extracted image may be output to the output device 4, and the shape of the scrap iron may be identified visually. This completes the process of step S7, and a series of shape identification processes are terminated. EXAMPLES

[0033] An embodiment in which the method of the present invention is applied to a video of a simulated scrap iron falling from an experiment is shown. The frame rate of the captured video is 30 fps, and the number of frames is 40. FIG. 5(a) shows a grayscale image of the 20th frame, in which the simulated scrap iron falling in the center is captured. This captured image was preprocessed by the preprocessing unit 5a, and then robust principal component analysis was performed by the calculation unit 5b to obtain a sparse matrix. FIG. 5(b) shows an image of the sparse matrix. The standard deviation σ was calculated for the sparse matrix in FIG. 5(b), and a binarized image was created in the 0.5σ region, as shown in FIG. 5(c), and the white part indicating the simulated scrap iron was obtained. In FIG. 5(c), since there is a missing part in the region considered to be the simulated scrap iron, the first image correction unit 5d performed a morphological transformation to expand and contract the white part using a 5×5 kernel, and a corrected binarized image was created. The corrected binary image is as shown in FIG. 5(d), and the missing parts were filled and the white areas showing the simulated scrap iron were connected. Next, since the corrected binary image shown in FIG. 5(d) has scattered white parts which are noise, the second image correction unit 5e performs morphological transformation to shrink and expand the image, as shown in FIG. 5(e). The noise in the corrected binary image of FIG. 5(d) was removed, and a mask image was created. Then, the mask processing unit 5f used the mask image to extract the simulated scrap iron part of the photographed image, as shown in FIG. 5(f), and an extracted image showing only the fallen simulated scrap iron was created. Since this extracted image shows only the moving object and the simulated scrap iron with its characteristics, if it is applied to a machine learning model such as image segmentation that has been trained in advance, the accuracy of identifying the simulated scrap iron can be improved.

[0034] As is clear from the above description, in the shape identification process according to one embodiment of the present invention, the image processing device 5 converts the video data of the scrap iron S into a two-dimensional data matrix, performs robust principal component analysis on the two-dimensional data matrix to obtain a sparse matrix, binarizes the element values ​​of the sparse matrix with a predetermined threshold value to generate a binary image, performs morphological transformation on the binary image to expand a region of a predetermined pixel value and then contract the region of the predetermined pixel value to generate a modified binary image, performs morphological transformation on the modified binary image to contract a region of a predetermined pixel value and then expand the region of the predetermined pixel value to generate a mask image, and extracts an image corresponding to the mask image from the video data of the scrap iron S as an image of the scrap iron S. This allows the shape of the scrap iron S to be identified with high accuracy. In addition, the electric furnace can be operated efficiently by controlling the electric furnace according to the identified shape of the scrap iron S.

[0035] In addition, when a modified binary image is generated by performing a morphological transformation to expand a region of a predetermined pixel value on a binary image and then expand the region of the predetermined pixel value on the binary image, and a mask image is generated by performing a morphological transformation to expand a region of a predetermined pixel value on the modified binary image and then contract the region of the predetermined pixel value on the modified binary image, since there is a missing region in the binary image, a necessary part may be recognized as noise and removed when the modified binary image is generated. Therefore, it is possible that the missing region cannot be complemented when the mask image is generated, and a mask image in which a moving object part is missing is generated. In contrast, in this embodiment, a modified binary image is generated by performing a morphological transformation to expand a region of a predetermined pixel value on a binary image and then contract the region of the predetermined pixel value on the modified binary image, and a mask image is generated by performing a morphological transformation to contract a region of a predetermined pixel value on the modified binary image and then expand the region of the predetermined pixel value on the modified binary image. That is, in this embodiment, the missing region is complemented by the first morphological transformation, and noise is removed by the second morphological transformation. This makes it possible to remove noise without missing necessary image information.

[0036] Although the embodiments of the invention made by the present inventors have been described above, the present invention is not limited by the descriptions and drawings that form part of the disclosure of the present invention according to the present embodiments. In other words, other embodiments, examples, operation techniques, etc. made by those skilled in the art based on the present embodiments are all included in the scope of the present invention. [Explanation of symbols]

[0037] 1. Shape Identification System 2. Imaging Device 3 Input Devices 4 Output Device 5. Image Processing Device 5a Pretreatment section 5b Arithmetic section 5c Binary image processing section 5d First image correction section 5e 2nd Image Correction Section 5f Mask processing section 100 Scrap Yard 101 Transport Equipment 102 Charging trolley 103 Electric furnace S Scrap iron

Claims

[Claim 1] a step of converting video data of a moving object into a two-dimensional data matrix, a step of performing robust principal component analysis on the two-dimensional data matrix to obtain a sparse matrix, a step of binarizing element values ​​of the sparse matrix with a predetermined threshold to generate a binary image, a step of performing a morphological transformation on the binary image to expand a region of a predetermined pixel value and then contract the region of the predetermined pixel value to generate a modified binary image, a step of performing a morphological transformation on the modified binary image to contract a region of a predetermined pixel value and then expand the region of the predetermined pixel value to generate a mask image, and a step of extracting an image corresponding to the mask image from the video data of the moving object as an image of the moving object.

Images