Iron scrap classification method and iron scrap classification apparatus through image analysis

Abstract

To provide iron scrap classification information by image analysis for a region including one or more iron scraps.SOLUTION: There are provided a method which includes the steps of: allowing a reception part to acquire a loaded state image imaged in a state in which a plurality of iron scraps is loaded on a loading facility; allowing a processor to acquire a divided image including an object iron scrap which is one of the plurality of iron scraps by a loaded state image using a segmentation model performing segmentation for the object iron scrap; allowing a processor to acquire item information and grade information corresponding to the object iron scrap using a classification model performing classification for the divided image, and performing analysis by image unit; and providing iron scrap classification information including the item information and the grade information, and an iron scrap classification device and a recording medium for performing the method.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The technical field of the present disclosure relates to a method for providing image classification result information for an article loaded on a loading equipment or a single article, etc., and relates to a technical field of a method for providing iron scrap classification information by image analysis for an area including one or more iron scraps.

Background Art

[0002] Due to the recent increase in the logistics industry, in fact, loading equipment for loading various articles or for loading articles to move from one place to another is widely used. However, although a plurality of articles loaded on the loading equipment can be managed or loaded by the operator's judgment according to product type and specification position, it may be difficult to check this every time. Therefore, a segmentation method may be used to monitor the images of each area by dividing the entire area of the loading equipment on which a plurality of articles are loaded. Generally, in the case of a segmentation method for iron scrap-related image analysis, the optical system (CCTV and mechanical parts) is easily installed at the loading (unloading) location without H / W engineering to set the position and angle of the optical system so that AI can exhibit optimal performance, and images are acquired. In such a case, there are few consistent features (various shapes by utilization method, plain texture (quality), various color senses (painting, rust, etc.)), and an irregular iron scrap due to various cutting methods, warping, etc. is classified by instance segmentation, and a method for determining the grade of the iron scrap is adopted. Therefore, in fact, there is a limitation that a large amount of image collection and labeling are required. Therefore, in order to solve the limitation of such an analysis method that requires a large amount of image collection, it is necessary to provide an image analysis method and a system that can improve performance with only a smaller amount of image collection and labeling.

Prior Art Documents

Patent Documents

[0003] Korean Published Patent No. 10-2011-0078566 (2011.07.07) Efficient Item Loading Position Detection System Using Digital Image Recognition [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] The problem that this disclosure aims to solve is to improve the accuracy of image analysis and classification of one or more pieces of scrap iron included in a loading image when providing image classification information for scrap iron, and to provide a service that uses an AI model to collect a minimum number of images and provides highly accurate image classification information as a result.

[0005] The problems that this disclosure aims to solve are not limited to the technical problems described above, and other technical problems may exist. [Means for solving the problem]

[0006] As a technical means for achieving the aforementioned technical challenges, the scrap iron classification method through image analysis relating to the first aspect of this disclosure may include the steps of: a receiving unit acquiring a loading state image captured with multiple scrap iron pieces loaded on a loading device; a processor performing segmentation on a target scrap iron piece, which is one of the multiple scrap iron pieces, and using a segmentation model to acquire a segmented image in the loading state image that includes the target scrap iron piece; a processor performing classification on the segmented image and using a classification model that performs analysis on an image-by-image basis to acquire item information and grade information corresponding to the target scrap iron piece; and a processor providing scrap iron classification information including the item information and grade information.

[0007] Furthermore, the steps for acquiring the item information and the grade information may include: a step in which the processor acquires a target scrap iron image showing the target scrap iron by removing the background area from the divided image; and a step in which the processor performs the classification on the target scrap iron image to acquire the item information and the grade information.

[0008] The process may further include: a step in which the receiving unit acquires a correct image representing the target iron scrap; a step in which the processor determines the ratio of the overlapping region between the correct image and the target iron scrap image; a step in which, if the ratio of the overlapping region exceeds a critical overlap ratio, the processor acquires an iron scrap determination accuracy indicating whether the target iron scrap image is actually an image of iron scrap; and a step in which the processor provides the iron scrap determination accuracy as a performance indicator.

[0009] The process may further include: a step in which the processor determines a target weight value determined by the size of the region of the target iron scrap image; a step in which the processor determines the target accuracy for the target iron scrap image; and a step in which the processor applies the target weight value to the target accuracy to determine the accuracy for the classification model.

[0010] The process may further include the steps of: the receiving unit acquiring a single segmented image captured for a single piece of scrap iron; the processor acquiring a composite image using the loading state image and the single segmented image; and the processor applying the segmentation model and the classification model to the composite image to provide additional scrap iron classification information.

[0011] Furthermore, the step of acquiring the composite image may include the step of the processor acquiring a single iron scrap image by removing the background region from the single divided image; and the step of the processor combining the loading state image and the single iron scrap image to acquire the composite image.

[0012] Furthermore, the step of combining the loading state image and the single iron scrap image to obtain the composite image may include the step of the processor determining the number of single iron scraps that can be synthesized into the loading state image based on the size of the single iron scrap image region; and the step of the processor obtaining the composite image based on the number of synthesizable items.

[0013] Furthermore, the step of acquiring the loading state image may include the step of the receiving unit acquiring an updated loading state image captured in the loading equipment on which the same plurality of iron scraps are loaded, with the positions of the plurality of iron scraps updated; and the step of acquiring the segmented image may include the step of the processor using the segmentation model to acquire a segmented image in the updated loading state image that includes the target iron scrap.

[0014] Furthermore, if the number of pixels included in the target iron scrap image is less than the first number, the target weighting increases in proportion to a linear function corresponding to the first slope; if the number of pixels is greater than or equal to the first number but less than the second number, the target weighting increases in proportion to an exponential function having a base greater than the first slope; and if the number of pixels is greater than or equal to the second number, the target weighting increases in proportion to a linear function corresponding to a second slope that exhibits a slope smaller than the first slope, and the first and second slopes can be positive numbers.

[0015] Furthermore, the step of providing the iron scrap classification information may include the step of the processor acquiring average weight information indicating the cumulative area and / or cumulative number of items for each item and grade, respectively, corresponding to the target iron scrap among the plurality of iron scraps; and the step of the processor providing a pie chart in the loading state image showing the cumulative area ratio and / or cumulative number ratio for each item and grade with respect to the target iron scrap, based on the average weight information.

[0016] The scrap iron sorting apparatus through image analysis relating to the second aspect of this disclosure may include: a receiving unit that acquires a loading state image captured when multiple pieces of scrap iron are loaded on a loading device; and a processor that uses a segmentation model to perform segmentation for a target scrap iron which is one of the multiple pieces of scrap iron, to acquire a segmented image including the target scrap iron in the loading state image, uses a classification model to perform classification on the segmented image and perform analysis on an image-by-image basis to acquire item information and grade information corresponding to the target scrap iron, and provides scrap iron sorting information including the item information and grade information.

[0017] Furthermore, the processor can obtain a target scrap iron image showing the target scrap iron by removing the background area from the divided image, and perform the classification on the target scrap iron image to obtain the item information and the grade information.

[0018] Furthermore, the receiving unit acquires a correct image representing the target iron scrap, the processor determines the ratio of the overlapping region between the correct image and the target iron scrap image, and if the ratio of the overlapping region exceeds a critical overlap ratio, it acquires an iron scrap determination accuracy indicating whether the target iron scrap image is actually an image of iron scrap, provides the iron scrap determination accuracy as a performance indicator, determines a target weighting value determined by the size of the region of the target iron scrap image, determines the target accuracy for the target iron scrap image, and applies the target weighting value to the target accuracy to determine the accuracy for the classification model.

[0019] Furthermore, the receiving unit acquires a single segmented image captured for a single piece of iron scrap, the processor acquires a composite image using the loading state image and the single segmented image, and can provide additional iron scrap classification information by applying the segmentation model and the classification model to the composite image.

[0020] According to the third aspect of this disclosure, a computer-readable non-temporary recording medium can be provided on which a program for embodying the method of the first aspect is recorded. [Effects of the Invention]

[0021] According to one embodiment of this disclosure, by performing image analysis and image classification using a segmentation model and a classification model, it is possible to provide a high-performance iron scrap classification process with a small number of image collections.

[0022] Furthermore, in providing iron scrap classification information, the fact that segmentation and classification are carried out separately has the advantage of enabling effective data augmentation of iron scrap images.

[0023] Also, in terms of performing an evaluation process for the segmentation model and the classification model to obtain an image of iron scrap and obtain image classification information, there is an effect that the accuracy of the classification result can be improved.

[0024] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

Brief Description of Drawings

[0025] [Figure 1] It is a block diagram schematically showing the configuration of an iron scrap classification device through image analysis according to an embodiment of the present disclosure. [Figure 2] It is a flowchart showing each stage in which an iron scrap classification device according to an embodiment of the present disclosure provides iron scrap classification information. [Figure 3] It is a flowchart schematically showing each stage in which an iron scrap classification device according to an embodiment of the present disclosure performs image analysis on iron scrap by changing layers. [Figure 4] It is a drawing schematically showing an example in which an iron scrap classification device according to an embodiment of the present disclosure performs classification after performing segmentation. [Figure 5] It is a drawing for explaining an example in which an iron scrap classification device according to an embodiment of the present disclosure performs accuracy evaluation on a classification model. [Figure 6] It is a drawing for explaining an example in which an iron scrap classification device according to an embodiment of the present disclosure obtains item information and grade information of each iron scrap based on the imaging results of a plurality of iron scraps or a single iron scrap included in the loading equipment. [Figure 7] It is a drawing for explaining an example in which an iron scrap classification device according to an embodiment of the present disclosure enhances data based on the imaging results of one iron scrap or a single iron scrap included in the loading equipment. [Figure 8]This is a drawing illustrating an example of how an iron scrap sorting apparatus according to one embodiment of the present disclosure performs segmentation on a composite image. [Figure 9] This drawing illustrates an example of how an iron scrap sorting device according to one embodiment of the present disclosure performs segmentation on an updated loading state image. [Figure 10] This diagram schematically illustrates an example of how an iron scrap sorting device according to one embodiment of the present disclosure provides iron scrap sorting information. [Modes for carrying out the invention]

[0026] The advantages and features of this disclosure, and how they are achieved, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, this disclosure is not limited to the embodiments disclosed below and may be embodied in a variety of different forms, although these embodiments are provided to complete the disclosure and to fully inform a person of the ordinary skill in the art of the subject of this disclosure of its scope.

[0027] The terms used herein are for illustrative purposes only and are not intended to limit the disclosure. In this specification, singular terms include plural terms unless otherwise specified in the text. The terms “comprises” and / or “comprising” used in the specification do not exclude the existence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same component, and “and / or” includes each of the components mentioned and all combinations of one or more of them. Although terms such as “first,” “second,” etc., are used to describe a variety of components, these components are not limited by these terms. These terms are simply used to distinguish one component from another. Therefore, the first component mentioned below may, of course, be the second component within the technical concept of this disclosure.

[0028] Unless otherwise defined, all terms used herein (including technical and scientific terms) should be used in a sense that can be commonly understood by an ordinary technician in the relevant technical field. Furthermore, terms defined in commonly used dictionaries should not be interpreted ideally or excessively unless explicitly defined otherwise.

[0029] Spatially relative terms such as "below," "beneath," "lower," "above," and "upper" can be used to easily describe the correlation between one component and another, as illustrated in the drawing. Spatially relative terms should be understood to include not only the directions illustrated in the drawing but also the different directions of components during use or operation. For example, if a component illustrated in the drawing is flipped over, a component described as "below" or "beneath" of another component may be placed "above" of that component. Therefore, the exemplary term "below" can include both the downward and upward directions. Components can also be oriented in other directions, and accordingly, spatially relative terms can be interpreted in terms of orientation.

[0030] The following describes an embodiment in detail with reference to the drawings.

[0031] Figure 1 is a block diagram schematically illustrating the configuration of an iron scrap sorting device 100 through image analysis according to one embodiment of the present disclosure.

[0032] Referring to Figure 1, the iron scrap sorting device 100 can include a receiving unit 110 and a processor 120.

[0033] According to one embodiment, the receiving unit 110 can acquire a loading state image captured when multiple pieces of scrap iron are loaded onto the loading equipment.

[0034] In one embodiment, the processor 120 can obtain a segmented image containing the target scrap iron in a loading state image by utilizing a segmentation model that performs segmentation on one of several scrap iron pieces. Furthermore, the processor 120 can obtain item information and grade information corresponding to the target scrap iron by utilizing a classification model that performs classification on the segmented image and performs analysis on an image-by-image basis. In addition, the processor 120 can provide scrap iron classification information including item information and grade information.

[0035] Furthermore, it should be noted that the iron scrap classification device 100, through image analysis, acquires a loading state image with a receiving unit 110, performs segmentation with a processor 120 to acquire segmented images, performs classification to acquire item information and grade information corresponding to the segmented images, and provides iron scrap classification information including item information and grade information. This process can be connected by various conventional network combinations such as the Internet or mobile communication networks, and there are no special restrictions on this.

[0036] In addition to the components shown in Figure 1, a person with ordinary skill in the relevant art will understand that other general-purpose components may be further included in the scrap iron sorting device 100 through image analysis. For example, the scrap iron sorting device 100 through image analysis may further include a memory (not shown) for storing loading state images, divided images, item information and grade information corresponding to the divided images, etc., and may further include a transmission unit (not shown) for providing scrap iron sorting information or a display unit (not shown) for displaying scrap iron sorting information. Alternatively, a person with ordinary skill in the relevant art will understand that some of the components shown in Figure 1 may be omitted if other embodiments are followed.

[0037] The iron scrap sorting device 100, which uses image analysis according to one embodiment, can be used by users and can be linked with all types of handheld wireless communication devices equipped with touchscreen panels, such as mobile phones, smartphones, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), and tablet PCs. In addition, it can be included in or linked with devices that have a platform on which applications can be installed and run, such as desktop PCs, tablet PCs, laptop PCs, and IPTVs including set-top boxes.

[0038] The iron scrap sorting device 100, which uses image analysis, can be implemented on a terminal such as a computer that operates through a computer program for implementing the functions described herein.

[0039] The iron scrap classification device 100 through image analysis according to one embodiment may include, but is not limited to, a system (not shown) and an associated server (not shown) that provide classification information for iron scrap. The server according to one embodiment can support an application that provides a service for providing classification information for iron scrap.

[0040] The following description will focus on an embodiment in which the iron scrap classification device 100, based on image analysis, independently acquires and provides classification information results using a pre-set iron scrap classification method. However, as mentioned above, this may also be performed in conjunction with a server. In other words, the iron scrap classification device 100 based on image analysis and the server, based on an embodiment, can be integrated and implemented in terms of their functions, the server may be omitted, and the system is not limited to just one embodiment.

[0041] In one embodiment, the iron scrap sorting device 100 and the server may be linked, and the configuration for providing iron scrap classification information by performing the iron scrap classification process and the classification result information provision process may be performed by the server or by the iron scrap sorting device 100. For example, the iron scrap sorting device 100 can operate as a server, and in the following description, we will refer to it simply as the iron scrap sorting device 100.

[0042] Figure 2 is a flowchart showing each step in which the iron scrap sorting device 100 according to one embodiment of the present disclosure provides iron scrap sorting information.

[0043] Referring to step S210, the iron scrap sorting device 100 according to one embodiment can acquire a loading state image captured when multiple pieces of iron scrap are loaded on the loading equipment. In one embodiment, the iron scrap sorting device 100 can acquire a loading state image captured from the upper side of the loading equipment downwards, at which time the loading equipment may be loaded with multiple pieces of iron scrap. Therefore, the iron scrap sorting device 100 can acquire a loading state image that includes multiple pieces of iron scrap.

[0044] Referring to step S220, the iron scrap sorting device 100 according to one embodiment can obtain a segmented image containing the target iron scrap in the loading state image by utilizing a segmentation model that performs segmentation on a target iron scrap which is one of a plurality of iron scraps. In one embodiment, the segmentation model can include semantic segmentation and instance segmentation. In one embodiment, the iron scrap sorting device 100 can obtain a scrap region corresponding to each of the plurality of iron scraps by performing segmentation on the plurality of iron scraps included in the loading state image. When the iron scrap sorting device 100 performs segmentation using semantic segmentation, it can classify the plurality of iron scraps individually by classifying the pixels of the loading state image in physical units. That is, in order to classify the plurality of iron scraps included in the loading state image into individual iron scraps, the iron scrap sorting device 100 can perform segmentation on a pixel-by-pixel basis for the iron scrap in any one of the plurality of iron scrap regions. Analysis of scrap regions based on the results of semantic segmentation can be performed at the pixel level. For example, the iron scrap sorting device 100 can acquire a scrap region as a rectangular area containing at least one pixel corresponding to each of multiple iron scraps, based on the pixel level. Therefore, the iron scrap sorting device 100 can acquire segmented images, which are images corresponding to each scrap region acquired in the loading state image. That is, in one embodiment, each scrap region representing a target iron scrap that is one of multiple iron scraps can correspond to a segmented image. Therefore, the iron scrap sorting device 100 can acquire multiple segmented images for a target iron scrap that is one of multiple iron scraps by utilizing the segmentation model.

[0045] Referring to step S230, the iron scrap classification device 100 according to one embodiment can acquire item information and grade information corresponding to the target iron scrap by performing classification on divided images and performing analysis on an image-by-image basis. In one embodiment, classification may be a process of analyzing or classifying iron scrap for each divided image based on the characteristic information acquired from each image. The iron scrap classification device 100 can perform analysis on an image-by-image basis for each of the multiple divided images acquired in step S220 by utilizing the classification model. Therefore, the iron scrap classification device 100 can acquire item information and grade information corresponding to each analysis image. In one embodiment, the item information corresponds to characteristic information for the iron scrap and may include, for example, weight information, lightweight information, etc. Also, in one embodiment, the iron scrap classification device 100 can acquire grade information along with item information corresponding to the target iron scrap by performing classification on the target iron scrap image. Furthermore, the iron scrap classification device 100 according to one embodiment can acquire a target iron scrap image showing the target iron scrap, excluding the background area in the segmented image. The segmented image acquired in step S220 may be an image that includes background areas other than the target iron scrap. In one embodiment, the background area may include the wall area of ​​the loading equipment, the edge area of ​​the loading equipment, etc. Therefore, the iron scrap classification device 100 can acquire a target iron scrap image showing the area corresponding to the target iron scrap, excluding the background area, and can perform classification on the target iron scrap image to acquire item information and grade information for the target iron scrap. Therefore, when acquiring item information and grade information for the target iron scrap, it is possible to prevent errors in the analysis results that may occur due to the background area. Furthermore, in one embodiment, the iron scrap classification device 100 can perform an evaluation process for the segmentation model. The iron scrap classification device 100 according to one embodiment can acquire a correct image showing the target iron scrap.In one embodiment, the correct answer image may mean an image representing the correct answer for scrap iron obtained from a user terminal or an external server. That is, the correct answer image may be an image corresponding to an actual scrap iron region that can be verified to determine whether it corresponds to an actual scrap iron region obtained from a user terminal or an external server for multiple scrap iron items. The scrap iron classification device 100 can determine the ratio of the overlapping region between the correct answer image and the target scrap iron image. For example, the target scrap iron image may be a segmented image obtained using a segmentation model with the background region excluded. Therefore, the target scrap iron image may be identical to the correct answer image representing the actual scrap iron, but it may also differ depending on the segmentation results. Thus, the scrap iron classification device 100 can determine the ratio of the overlapping region between the correct answer image and the target scrap iron image, and if the ratio of the overlapping region exceeds a critical overlap ratio, it can obtain a scrap iron determination accuracy indicating whether the target scrap iron image is actually an image of scrap iron. In one embodiment, the critical overlap ratio can represent the minimum overlap ratio that can be predicted to correspond to actual scrap iron when the acquired target scrap iron image is predicted to correspond to actual scrap iron. That is, the scrap iron sorting device 100 can determine that the target scrap iron image corresponds to actual scrap iron when the ratio of the overlapping area between the correct image and the target scrap iron image exceeds the critical overlap ratio (e.g., 50 percent), and in this case, it can obtain scrap iron determination accuracy regarding whether the corresponding target scrap iron image is actually scrap iron. Therefore, the scrap iron sorting device 100 can provide scrap iron determination accuracy as a performance indicator. For example, in one embodiment, the critical overlap ratio is the ratio that can be predicted to correspond to actual scrap iron and can correspond to a criterion for primary sorting.In other words, the iron scrap sorting device 100 can determine that an object is not iron scrap by predicting that it does not correspond to iron scrap when the ratio of overlapping areas is less than or equal to the critical overlap ratio, and can temporarily determine that the iron scrap in question is predicted iron scrap by predicting that it corresponds to iron scrap when the ratio of overlapping areas exceeds the critical overlap ratio. Therefore, it is possible to obtain an iron scrap determination accuracy that indicates whether the predicted iron scrap image, which has been temporarily determined to be predicted iron scrap, is actually an image of iron scrap. For example, the iron scrap sorting device 100 can perform verification of whether the predicted iron scrap image is an image of complete iron scrap. That is, if the area in the predicted iron scrap image that includes images of the wall surface of loading equipment, images of the edges of loading equipment, etc., which are not iron scrap, is greater than or equal to a predetermined percentage, or if the degree to which the predicted iron scrap image corresponds to actual iron scrap is less than a predetermined percentage, it can be determined that it is not actually an image of iron scrap. By obtaining an iron scrap determination accuracy that indicates whether or not an image is of iron scrap, the iron scrap sorting device 100 can provide a performance indicator for the segmentation model. Therefore, the iron scrap sorting device 100 can perform segmentation using a segmentation model that corresponds to a pre-set percentage (e.g., 90 percent to 95 percent or more) when the iron scrap determination accuracy is above that percentage. In other words, the iron scrap sorting device 100 can decide that the performance of the segmentation model corresponding to a case where the iron scrap determination accuracy is below the pre-set percentage is suboptimal and therefore not apply it as the model for performing segmentation. In other embodiments, the iron scrap sorting device 100 may update the pre-set percentage corresponding to the iron scrap determination accuracy using a critical overlap ratio.For example, the scrap iron sorting device 100 may perform an evaluation of the segmentation model only to the extent that it corresponds to a percentage adjacent to the critical overlap ratio, by increasing the pre-set percentage corresponding to the accuracy of scrap iron determination by a certain level (for example, from 90 percent to 95 percent or more to 93 percent to 98 percent or more) when the critical overlap ratio corresponds to a percentage adjacent to a pre-set percentage (for example, when it is between 48 percent and 50 percent, adjacent to 50 percent). Furthermore, the scrap iron sorting device 100 according to one embodiment can perform an evaluation process for the classification model. The scrap iron sorting device 100 can determine the target weight value, which is determined by the size of the region of the target scrap iron image. The scrap iron sorting device 100 can determine the target accuracy for the target scrap iron image. Furthermore, the scrap iron sorting device 100 can determine the accuracy for the classification model by applying the target weight value to the target accuracy. In one embodiment, the target accuracy for a target scrap iron image can include, as described above, the accuracy of determining whether it is actual scrap iron, and can further include the item accuracy and grade accuracy for the item information and grade information corresponding to each target scrap iron obtained in step S230. That is, the scrap iron classification device 100 can determine the target accuracy indicating whether the acquired item information and grade information corresponds to the actual item and grade of the target scrap iron. Therefore, after determining the target accuracy, the scrap iron classification device 100 can update the target accuracy based on the size of the target scrap iron image area. For example, the scrap iron classification device 100 can determine different target weighting values ​​to be assigned to each of multiple target scrap iron images. The scrap iron classification device 100 can assign even higher weighting values ​​to scrap iron with a larger target scrap iron image area.In one embodiment, the iron scrap classification device 100 can determine the accuracy of the classification model based on the updated target accuracy, which is achieved by assigning higher weighting values ​​to iron scrap with larger area sizes in the target iron scrap image, in such a model that is more suitable for accurately determining iron scrap with larger area sizes in the target iron scrap image. For example, the iron scrap classification device 100 can determine the target weighting values ​​to increase linearly in proportion to the size of the target iron scrap image area. In other embodiments, when the iron scrap classification device 100 performs semantic segmentation, it may determine different degrees to which the weighting increases proportionally based on the size range of a previously set area. For example, if the number of pixels in the target scrap iron image is less than the first number, the target weighting value increases proportionally to a linear function corresponding to the first slope; if the number of pixels is greater than or equal to the first number but less than the second number, the target weighting value increases proportionally to an exponential function with a base greater than the first slope; and if the number of pixels is greater than or equal to the second number, the target weighting value may increase proportionally to a linear function corresponding to a second slope that exhibits a slope smaller than the first slope, and the first and second slopes may be positive numbers. For example, the scrap iron sorting device 100 according to one embodiment can determine the size of the area of ​​the target scrap iron image based on the number of pixels. Therefore, if the number of pixels in the target scrap iron image is less than the first number which is a previously set number, the target weighting value can be determined to increase proportionally to a linear function corresponding to the first slope. Also, if the number of pixels in the target scrap iron image is greater than or equal to the first number but less than the second number, the target weighting value can be determined to increase proportionally to an exponential function. In one embodiment, the first slope may correspond to a constant corresponding to a linear function. The iron scrap sorting device 100 can be configured such that an exponential function with a base greater than a constant corresponding to the first slope is applied to the range of the number of pixels from the first number to less than the second number. That is, the degree to which the target weighting value increases when the number of pixels is from the first number to less than the second number may be greater than the degree to which the target weighting value increases when the number of pixels is less than the first number.Furthermore, in determining the target weight value, the iron scrap sorting device 100 can be configured to apply a linear function corresponding to a second slope smaller than the first slope when the number of pixels included in the target iron scrap image is two or more. In one embodiment, the second slope can correspond to a constant corresponding to the linear function, and can correspond to a constant smaller than the constant corresponding to the first slope. That is, the degree to which the target weight value increases when the number of pixels is two or more can be made smaller again than the degree to which the target weight value increases when the number of pixels is between one and two. Also, the constants corresponding to the first slope, the base of the exponential function, and the second slope can each be positive numbers. Therefore, the degree to which the target weight value increases can be determined and applied differently depending on the range in which the size of the area of ​​the target iron scrap image is included. In such a case, the degree to which the target weight value increases can be smallest when the number of pixels is two or more, and the degree to which the target weight value increases can be largest when the number of pixels is between one and two. Therefore, when the iron scrap sorting device 100 performs semantic segmentation, it has the effect of more accurately determining the accuracy of the classification model by determining different weighting values ​​based on the size of the region based on the number of pixels. In one embodiment, the first... The region between the second and third pixel counts represents the broadest range and may contain the largest amount of scrap iron. In this case, the scrap iron sorting device 100 can further apply weighting based on the size (area) of the region by applying an exponential function to the extent that the target weighting increases by determining that the size of the region is of greater importance. In one embodiment, the larger the region, the greater the impact on the classification accuracy. However, since the sizes of multiple scrap iron corresponding to the second or third pixel count can all be large, it may not be very meaningful to create a large difference between them. Also, since the sizes of multiple scrap iron corresponding to the first pixel count can be smaller than the size of the reference range, it may be meaningful to create a larger difference between them within that range than when the second or third pixel count is greater. Therefore, the iron scrap sorting device 100 can determine the accuracy of the classification model by determining the degree to which the target weighting value increases in the range of the number of pixels being between the first number and less than the second number, in the range of the number of pixels being less than the first number, and in the range of the number of pixels being the second number or more. Therefore, by obtaining accuracy for the classification model, the iron scrap sorting device 100 can perform classification using the corresponding classification model when the classification model accuracy is above a predetermined percentage. In other words, the iron scrap sorting device 100 can perform classification using a classification model that shows high target accuracy for iron scrap with a large area of ​​the target iron scrap image.

[0046] Referring to step S240, the iron scrap classification device 100 according to one embodiment can provide iron scrap classification information including item information and grade information. As described above, the iron scrap classification device 100 can provide item information and grade information for target iron scrap obtained using a segmentation model and a classification model in the iron scrap classification information.

[0047] Figure 3 is a flowchart illustrating the steps involved in the process by which the iron scrap sorting apparatus 100 according to one embodiment of the present disclosure performs image analysis on iron scrap by changing the layer.

[0048] Referring to Figure 3, the iron scrap sorting device 100 according to one embodiment can determine the presence or absence of loading equipment (cargo bed) and a grapple. Therefore, the Layer can be measured when loading equipment is present but a grapple is absent. Furthermore, image matching can be performed only if a Layer change is performed after the Layer has been measured, and as explained in step S220, segmentation can be performed to obtain a segmented image through individual extraction of iron scrap dogs. Subsequently, as explained in step S230, classification can be performed to obtain item information and grade information for each piece of iron scrap, and the final grade of each piece of iron scrap can be determined.

[0049] Figure 4 is a schematic diagram illustrating an example in which an iron scrap sorting apparatus 100 according to one embodiment of the present disclosure performs classification after performing segmentation.

[0050] Referring to Figure 4, the iron scrap classification device 100 according to one embodiment can acquire a segmented image by using a segmentation model to individually extract multiple iron scraps into their respective scrap areas based on pixel units. Furthermore, the iron scrap classification device 100 can acquire a target iron scrap image showing the target iron scrap by removing the background area from the segmented image. Accordingly, as shown at the bottom of Figure 4, classification can be performed on the target iron scrap image with the background area removed to acquire item information and grade information for the target iron scrap.

[0051] Figure 5 is a diagram illustrating an example of how an iron scrap sorting device 100 according to one embodiment of the present disclosure performs accuracy evaluation against a classification model.

[0052] Referring to Figure 5, the iron scrap classification device 100 according to one embodiment can assign different target weighting values ​​depending on the size of the area of ​​the target iron scrap image. The AI ​​performance accuracy measurement method shown on the left side of Figure 5 is a general accuracy measurement method (Acc-Accuracy), in which equivalent evaluation is performed by assigning the same weighting value to the analysis results for large and small iron scrap. In the case of the general accuracy measurement method, the formula number of correct answers / total number may be applied. In contrast, according to one embodiment, the AWA (Area-Weighted Accuracy) area-weighted accuracy method can be applied. In one embodiment, the iron scrap classification device 100 can measure accuracy by assigning target weighting values ​​differently depending on the size of the area. In the case of the area-weighted accuracy method, the formula number of correct area pixels / total area pixels may be applied. Therefore, since the iron scrap sorting device 100 uses an area-weighted accuracy method to determine the accuracy of the classification model, it can perform classification using a model that gives higher importance to larger pieces of iron scrap.

[0053] Figure 6 is a diagram illustrating an example in which an iron scrap sorting device 100 according to one embodiment of the present disclosure acquires item information and grade information for each iron scrap based on imaging results of multiple iron scraps or a single iron scrap included in the loading equipment.

[0054] Referring to Figure 6, the iron scrap sorting device 100 according to one embodiment may acquire a loading state image captured when multiple iron scraps are loaded, as explained in step S210, or it may acquire a single segmented image captured from multiple angles of a single iron scrap, as shown at the bottom of Figure 6. Therefore, the iron scrap sorting device 100 can acquire item information and grade information corresponding to the target iron scrap by performing segmentation using a segmentation model to extract individual iron scraps from the loading state image and acquire segmented images, and then performing classification on the segmented images using a classification model. Furthermore, when the iron scrap sorting device 100 acquires a single segmented image, it can acquire item information and grade information corresponding to the single iron scrap by performing classification on the single segmented image using a classification model. Therefore, item matching information, in which the image and item information for each iron scrap are matched, can be acquired and stored in a database.

[0055] Figure 7 is a diagram illustrating an example in which a scrap iron sorting device 100 according to one embodiment of the present disclosure augments data based on the imaging results of one or a single piece of scrap iron included in the loading equipment.

[0056] Referring to Figure 7, the iron scrap sorting device 100 according to one embodiment can extract one of the target iron scraps, which is one of several iron scraps loaded on the loading equipment, and perform individual classification to acquire item information and grade information corresponding to each target iron scrap, which can then be stored in a database. Furthermore, the iron scrap sorting device 100 does not immediately perform classification on a single segmented image captured for a single iron scrap, but rather applies it to the loading state image, thereby performing segmentation and classification on newly acquired images, and consequently, the multiple pieces of information acquired can be further stored in the database. This can be explained with reference to Figure 8.

[0057] Figure 8 is a diagram illustrating an example of how an iron scrap sorting apparatus 100 according to one embodiment of the present disclosure performs segmentation on a composite image.

[0058] Referring to Figure 8, the iron scrap sorting device 100 according to one embodiment can acquire a composite image using a loading state image and a single-part image. The iron scrap sorting device 100 can acquire a single iron scrap image by removing the background area from the single-part image. Furthermore, the iron scrap sorting device 100 can acquire a composite image by combining the loading state image and the single iron scrap image. As shown in Figure 8, a single iron scrap corresponding to the hatched area can be combined with the loading state image. Therefore, the iron scrap sorting device 100 can perform segmentation based on the newly acquired composite image, such that the single-part image is accumulated in the loading state image. The iron scrap sorting device 100 can apply a segmentation model and a classification model to the composite image to provide additional iron scrap classification information, including item information and grade information corresponding to the target iron scrap acquired. In another embodiment, the iron scrap sorting device 100 can determine the number of single iron scraps that can be combined with the loading state image based on the size of the single iron scrap image area. Furthermore, the scrap iron sorting device 100 can acquire a composite image based on the number of possible composite images. For example, the scrap iron sorting device 100 according to one embodiment can preferentially acquire a composite image in which one single scrap iron image is combined with a loading state image, and thereafter may acquire a composite image in which multiple single scrap iron images are combined. For example, if the ratio of the number of pixels included in a single scrap iron image area to the total number of pixels in the cross-sectional area corresponding to the loading equipment is less than 1 percent (e.g., 10 percent), the scrap iron sorting device 100 can determine the number of possible composite images so that single scrap iron images can be combined in two or more areas for each area obtained by dividing the horizontal length of the loading equipment by a first value (e.g., 5). For example, the scrap iron sorting device 100 can make it possible for single scrap iron images to be overlapped and combined in the first area (e.g., 5 areas) obtained by dividing the horizontal length of the loading equipment by a first value.In other words, in this case, the scrap iron sorting device 100 can acquire multiple composite images by determining the number of synthesizable elements to be one of 2 to 5. Furthermore, if the ratio of the number of pixels in a single scrap iron image area to the total number of pixels in the cross-sectional area corresponding to the loaded equipment is between 1 percent and 2 percent (e.g., 20 percent), the scrap iron sorting device 100 can determine the number of synthesizable elements by dividing the lateral length of the loaded equipment by a second value smaller than the first value (e.g., 3) so that a single scrap iron image can be combined into two or more areas for each area. For example, the scrap iron sorting device 100 can ensure that a single scrap iron image is superimposed and combined into a second area (e.g., three areas) obtained by dividing the lateral length of the loaded equipment by the second value. In other words, in this case, the scrap iron sorting device 100 can acquire multiple composite images by determining the number of synthesizable elements to be 2 or 3. If the ratio of the number of pixels in a single scrap iron image area to the total number of pixels in the cross-sectional area corresponding to the loading equipment is 2 percent or more, the scrap iron sorter 100 can determine the number of composite images so that a single scrap iron image can be combined into two areas for each area obtained by dividing the lateral length of the loading equipment by a third value smaller than the second value (e.g., 2). For example, the scrap iron sorter 100 can ensure that a single scrap iron image can be superimposed and combined into a third area (e.g., a 2-pixel area) obtained by dividing the lateral length of the loading equipment by the third value. In this case, the scrap iron sorter 100 can obtain an additional composite image by determining the number of composite images to be 2. Therefore, the scrap iron sorter 100 can perform more data augmentation by utilizing the data augmentation process described above.

[0059] Figure 9 is a diagram illustrating an example in which an iron scrap sorting device 100 according to one embodiment of the present disclosure performs segmentation on an updated loading state image.

[0060] Referring to Figure 9, the iron scrap sorting device 100 according to one embodiment is a loading device on which multiple identical pieces of iron scrap are loaded, and it can acquire an updated loading state image in which the positions of the multiple pieces of iron scrap have been updated. That is, the iron scrap sorting device 100 can acquire an updated loading state image in which the positions of the multiple pieces of iron scrap have been changed via a grapple. In addition, the iron scrap sorting device 100 can acquire a segmented image that includes the target iron scrap in the updated loading state image by utilizing a segmentation model. Referring to Figure 9, the iron scrap sorting device 100 may acquire an updated loading state image in which the positions of the multiple pieces of iron scrap have been updated via a grapple, or it may acquire an updated loading state image in which the positions of the multiple pieces of iron scrap have been updated by dividing the region of the loading state image and updating the order of each image region differently. The scrap iron sorting device 100 can perform segmentation on an updated loading state image by acquiring a new image in which the positions of multiple scrap iron pieces have been updated using loading equipment in which the same multiple scrap iron pieces are loaded, and consequently, it can perform a greater data augmentation.

[0061] Figure 10 is a schematic diagram illustrating an example in which an iron scrap sorting device 100 according to one embodiment of the present disclosure provides iron scrap sorting information.

[0062] Referring to Figure 10, the iron scrap sorting device 100 according to one embodiment can acquire average weight information showing the cumulative area and / or cumulative number of items for each item and grade corresponding to the target iron scrap among multiple iron scraps. Furthermore, the iron scrap sorting device 100 can provide a pie chart showing the cumulative area ratio and / or cumulative number ratio for each item and grade relative to the target iron scrap in a loading state image based on the average weight information. The iron scrap sorting device 100 according to one embodiment can provide not only grade information but also item information for each target iron scrap. For example, the iron scrap sorting device 100 can provide information such as weight and light weight, which indicate the item, in addition to grades such as A grade and B grade, which indicate the grade of the iron scrap. The iron scrap sorting device 100 can accumulate all the results determined for the entire loading equipment and finally calculate the area (or number). Furthermore, if the iron scrap sorting device 100 creates a table of the average weight information by area for each grade / item, it can measure not only the area ratio but also the overall weight by grade. According to one embodiment, unlike existing conventional technologies, it acquires and provides not only grade information but also item information, which has the effect of making it easily applicable to countries (by country or by steel company) where the same item has different grades.

[0063] According to one embodiment, by performing image analysis and image classification using a segmentation model and a classification model, it is possible to provide a high-performance iron scrap classification process with a small number of image collections. Furthermore, by performing segmentation and classification separately in the process of providing iron scrap classification information, it is possible to effectively augment the data for iron scrap images. Additionally, by performing an evaluation process for the segmentation model and classification model to acquire images of iron scrap and obtain image classification information, the accuracy of the classification results can be improved.

[0064] Various embodiments of this disclosure may be embodied in software that includes one or more instructions stored in a storage medium (e.g., memory) readable by a machine (e.g., a display device or a computer). For example, the machine's processor (e.g., processor 220) may call and execute at least one of the one or more instructions stored in the storage medium. This allows the machine to be operated to perform at least one function by the one or more instructions called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, “non-transitory” simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily on the storage medium.

[0065] According to one embodiment, the methods relating to the various embodiments disclosed herein may be provided in a computer program product. The computer program product may be traded as a commodity between sellers and buyers. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or online (e.g., downloaded or uploaded) through an application store (e.g., PlayStore™), or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily generated on a device-readable storage medium such as the memory of a manufacturer's server, an application store server, or an intermediary server.

[0066] Although the present invention has been described with reference to the illustrated drawings, it is not limited by the disclosed embodiments and drawings, and a person with ordinary skill in the art relating to these embodiments will understand that it can be embodied in modified forms without departing from the essential characteristics described above. Therefore, the disclosed methods should be considered in an explanatory rather than restrictive manner. Even if the effects of the configuration of the present invention are not explicitly described when describing the embodiments, predictable effects may be recognized by the configuration in question. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within an equivalent scope should be interpreted as being included in the present invention. [Explanation of Symbols]

[0067] 100: Iron scrap sorting machine 110: Receiver 120: Processor

Claims

1. In a method for providing iron scrap classification information through image analysis, The receiving unit acquires an image of the loading state, which is captured when multiple pieces of scrap iron are loaded onto the loading equipment; A step in which the processor uses a segmentation model to perform segmentation on a target scrap iron that is one of the plurality of scrap iron pieces to obtain a segmented image that includes the target scrap iron in the loading state image; The process involves the processor performing classification on the segmented images and using a classification model that performs analysis on an image-by-image basis to obtain item information and grade information corresponding to the target iron scrap; and The process includes the step of the processor providing iron scrap classification information including the item information and the grade information; further, The step in which the processor obtains a target scrap iron image showing the target scrap iron by removing the background region from the divided image; and The process includes the step of the processor determining a target weight value determined by the size of the region of the target iron scrap image; If the number of pixels included in the target iron scrap image is less than the first number, the target weighting value increases in proportion to the linear function corresponding to the first slope. If the number of pixels is greater than or equal to the first number but less than the second number, the target weighting value increases in proportion to an exponential function having a base greater than the first slope. If the number of pixels is equal to or greater than the second number, the target weighting value increases in proportion to a linear function corresponding to a second slope that exhibits a slope smaller than the first slope. The first slope and the second slope are positive numbers, method.

2. The method according to claim 1, wherein the step of obtaining the item information and grade information includes the step of the processor performing the classification on the target iron scrap image to obtain the item information and grade information.

3. The receiving unit acquires a correct image indicating the target iron scrap; The step in which the processor determines the ratio of the overlapping region between the correct image and the target scrap iron image; If the ratio of the overlapping regions exceeds the critical overlap ratio, the processor obtains an accuracy in determining whether the target scrap iron image is actually an image of scrap iron; and The method according to claim 2, further comprising the step of the processor providing the iron scrap determination accuracy as a performance indicator.

4. The step of the processor determining the accuracy of the target iron scrap image; and The method according to claim 2, further comprising the step of the processor applying the target weighting value to the target accuracy to determine the accuracy for the classification model.

5. The receiving unit acquires a single segmented image captured for a single piece of scrap iron; The process involves the processor obtaining a composite image using the loading state image and the single-partition image; and The method according to claim 1, further comprising the step of the processor applying the segmentation model and the classification model to the composite image to provide additional iron scrap classification information.

6. The step of obtaining the aforementioned composite image is The process involves the processor obtaining a single scrap metal image by removing the background region from the single partitioned image; and The method according to claim 5, further comprising the step of the processor combining the loading state image and the single iron scrap image to obtain the composite image.

7. The step of obtaining the composite image by combining the aforementioned loading state image and the aforementioned single iron scrap image is The processor determines the number of single scrap iron pieces that can be synthesized into the loading state image based on the size of the single scrap iron image region; and The method according to claim 6, further comprising the step of the processor acquiring the composite image based on the number of synthesizable elements.

8. The step of obtaining the aforementioned image of the loaded state is The receiving unit includes the step of acquiring an updated loading state image, which is captured in the loading equipment on which the same plurality of iron scraps are loaded, with the positions of the plurality of iron scraps updated; The step of obtaining the aforementioned segmented image is The method according to claim 1, further comprising the step of the processor using the segmentation model to obtain a segmented image in the updated loading state image that includes the target iron scrap.

9. The step of providing the aforementioned iron scrap classification information is The processor obtains average weight information indicating the cumulative area and / or cumulative number of items for each item and grade, respectively, for the item information and grade information corresponding to the target scrap iron among the plurality of scrap iron pieces; and The method according to claim 1, further comprising the step of the processor providing a pie chart showing the cumulative area ratio and / or cumulative number ratio by item and grade for the target iron scrap in the loading state image based on the average weight information.

10. In an iron scrap classification device that provides iron scrap classification information through image analysis, A receiving unit that acquires a loading state image captured when multiple pieces of scrap iron are loaded onto the loading equipment; and Using a segmentation model that performs segmentation on a target scrap iron that is one of the aforementioned multiple scrap iron pieces, a segmented image including the target scrap iron is obtained in the loading state image. Using a classification model that performs classification on the aforementioned segmented images and analyzes them on an image-by-image basis, item information and grade information corresponding to the target iron scrap are obtained. A processor that provides iron scrap classification information including the item information and the grade information; The aforementioned processor By removing the background region from the aforementioned segmented image, an image of the target iron scrap showing the target iron scrap is obtained. The target weight value is determined by the size of the area of ​​the target iron scrap image, If the number of pixels included in the target iron scrap image is less than the first number, the target weighting value increases in proportion to the linear function corresponding to the first slope. If the number of pixels is greater than or equal to the first number but less than the second number, the target weighting value increases in proportion to an exponential function having a base greater than the first slope. If the number of pixels is equal to or greater than the second number, the target weighting value increases in proportion to a linear function corresponding to a second slope that exhibits a slope smaller than the first slope. Iron scrap sorting device in which the first and second inclines are positive numbers.

11. The iron scrap classification apparatus according to claim 10, wherein the processor performs the classification on the target iron scrap image to obtain the item information and the grade information.

12. The receiving unit Obtain the correct image showing the aforementioned target iron scrap, The aforementioned processor The ratio of the overlapping region between the aforementioned correct image and the aforementioned target iron scrap image is determined. If the ratio of the overlapping regions exceeds the critical overlap ratio, the iron scrap determination accuracy is obtained to indicate whether the target iron scrap image is actually an image of iron scrap. The accuracy of the aforementioned iron scrap determination is provided as a performance indicator. The accuracy of the target iron scrap image is determined, The iron scrap sorting apparatus according to claim 11, wherein the accuracy with respect to the classification model is determined by applying the target weight value to the target accuracy.

13. The receiving unit A single segmented image is obtained from a single piece of iron scrap, The aforementioned processor A composite image is obtained using the aforementioned loading state image and the aforementioned single-part image. The iron scrap classification apparatus according to claim 10, which provides additional iron scrap classification information by applying the segmentation model and the classification model to the composite image.

14. A computer-readable recording medium that stores a program for causing a computer to perform the method described in any one of claims 1 to 9.

Images