Method, device, system, electronic device and medium for collecting images of iron scrap

Abstract

To provide an iron scrap image collection method, an iron scrap image collection system, an iron scrap image collection device, an electronic device, a storage medium, and a computer program.SOLUTION: An iron scrap image collection method includes the steps of: identifying one sub-region in an iron scrap down region where an iron scrap is taken down by an iron griping device and a photographing position corresponding to the sub-region, the photographing position being a position in the sub-region where the iron gripping device in the sub-region takes down the iron scrap; acquiring three-dimensional point group data of the sub-region in response to the confirmation that the iron griping device departed from the down region; determining the magnification when the sub-region is photographed on the basis of the three-dimensional point group data; and starting image collection of iron scraps of a lot on the basis of the photographing position and the magnification. An iron scrap image collection system includes an image collection device for collecting down region images, a three-dimensional scanning device which obtains three-dimensional point group data of the region by scanning the down region, and a processing device which executes the iron scrap image collection method.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to the field of machine vision, and more particularly to a method, an apparatus, a system for collecting images of ferrous scrap, an electronic device, a computer-readable storage medium and a computer program product. [Background technology]

[0002] In response to the demands of green economic development, many domestic and international steel mills are turning to scrap steel as an important raw material for steelmaking, aiming to achieve carbon peaking and carbon neutrality as soon as possible and reduce their reliance on iron ore. Steel mills must purchase large quantities of scrap steel every year, which comes in a vast variety of materials, complex shapes, and sizes, often containing impurities such as soil and oil. Traditionally, steel mills rely on manual quality inspection to grade the purchased scrap steel. After the scrap steel is collected, stored, and transshipped, the grading results are manually entered and uploaded into an ERP or MES system. Summary of the Invention

[0003] The present disclosure provides a method, an apparatus, a ferrous scrap image collection system, an electronic device, a computer-readable storage medium, and a computer program product for collecting images of ferrous scrap.

[0004] According to one aspect of the present disclosure, there is provided a method for collecting images of iron scrap, the method including the steps of: identifying a sub-area in an iron scrap unloading area where the iron scrap of the lot is unloaded by an iron gripping device each time the iron gripping device transfers the iron scrap to the iron scrap unloading area; and a photographing position corresponding to the sub-area, wherein the photographing position represents a position in the sub-area where the iron gripping device unloads the iron scrap of the lot; acquiring three-dimensional point cloud data for the sub-area in response to determining that the iron gripping device has left the iron scrap unloading area; determining a magnification ratio for photographing the sub-area based on the three-dimensional point cloud data; and starting to collect images of the iron scrap of the lot based on the photographing position and the magnification ratio.

[0005] According to another aspect of the present disclosure, there is provided an apparatus for collecting images of iron scrap, the apparatus including: a first module for identifying a sub-area in an iron scrap unloading area where an iron gripping device unloads the iron scrap of the lot each time the iron gripping device transfers the iron scrap to the iron scrap unloading area and a photographing position corresponding to the sub-area, the photographing position representing a position in the sub-area where the iron gripping device unloads the iron scrap of the lot; a second module for acquiring three-dimensional point cloud data for the sub-area in response to determining that the iron gripping device has left the iron scrap unloading area; a third module for determining a magnification ratio for photographing the sub-area based on the three-dimensional point cloud data; and a fourth module for starting to collect images of the iron scrap of the lot based on the photographing position and the magnification ratio.

[0006] According to another aspect of the present disclosure, there is provided a system for collecting images of ferrous scrap, the system including: an image collection device for collecting images including at least a portion of a ferrous scrap unloading area; a three-dimensional scanning device for scanning at least the portion of the ferrous scrap unloading area to obtain three-dimensional point cloud data of a corresponding area; and a processing device operably connected to the image collection device and the three-dimensional scanning device for performing the method for collecting images of ferrous scrap of the present disclosure.

[0007] According to another aspect of the present disclosure, there is provided an electronic device including at least one processor and a memory communicatively coupled to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method for collecting images of iron scrap of the present disclosure.

[0008] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions, the computer instructions being used to cause a computer to perform the method for collecting images of ferrous scrap of the present disclosure.

[0009] According to another aspect of the present disclosure, there is provided a computer program product, the computer program product including a computer program that, when executed by a processor, implements the method for collecting images of iron scrap of the present disclosure.

[0010] According to one or more embodiments of the present disclosure, the unloading position of the iron gripping device can be accurately positioned to ensure that each lot of iron scrap being unloaded is photographed comprehensively and the overlap of the photographed areas can be avoided, and the magnification of the corresponding photographed area can be automatically adjusted to ensure that the sizes of the photographed iron scrap of each lot are relatively consistent, thereby improving the accuracy of the subsequent iron scrap grading.

[0011] It should be understood that the material described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily apparent from the following specification. [Brief explanation of the drawings]

[0012] The drawings illustrate examples in examples, constitute a part of the specification, and together with the written description serve to explain exemplary embodiments of the examples. The examples shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar, but not necessarily identical, elements.

[0013] [Figure 1] 1 is a schematic diagram of an exemplary system capable of implementing the methods described herein, according to an embodiment of the present disclosure. [Figure 2] 1 is an exemplary flowchart of a method for collecting images of ferrous scrap, according to an embodiment of the present disclosure. [Figure 3] 10 is an exemplary flowchart of additional steps of a method for collecting images of ferrous scrap, according to an embodiment of the present disclosure. [Figure 4] 10 is an exemplary flowchart of additional steps of a method for collecting images of ferrous scrap, according to an embodiment of the present disclosure. [Figure 5] 10 is an exemplary flowchart of additional steps of a method for collecting images of ferrous scrap, according to an embodiment of the present disclosure. [Figure 6] 10 is an exemplary flowchart of additional steps of a method for collecting images of ferrous scrap, according to an embodiment of the present disclosure. [Figure 7] 10 is an exemplary flowchart of additional steps of a method for collecting images of ferrous scrap, according to an embodiment of the present disclosure. [Figure 8] FIG. 1 is a schematic diagram of an exemplary image collection scenario for steel scrap according to an embodiment of the present disclosure. [Figure 9] 1 is a block diagram of an apparatus for collecting images of ferrous scrap, according to an embodiment of the present disclosure; [Figure 10] 1 is a block diagram of a system for collecting images of iron scrap according to an embodiment of the present disclosure. [Figure 11] FIG. 1 is a block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014]

[0023] The following describes exemplary embodiments of the present disclosure in conjunction with the drawings. For ease of understanding, various details of the embodiments of the present disclosure are included therein, but they should be considered merely as explanatory. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for clarity and conciseness, the following description omits descriptions of known functions and structures.

[0015] In this disclosure, unless otherwise specified, the use of terms such as "first," "second," etc. to describe various elements is not intended to limit the location, timing, or importance of these elements. Such terms are used only to distinguish one element from another. In some instances, a first element and a second element may refer to the same instance of the element, or in some cases, may refer to different instances based on the context.

[0016] The terms used in the description of various examples of the present disclosure are intended only to describe particular examples and are not intended to be limiting. Unless the context clearly indicates otherwise, unless the number of elements is specifically limited, the elements may be one or more. Furthermore, as used in this disclosure, the term "and / or" covers any one of the listed items and all possible combinations.

[0017] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0018] FIG. 1 is a schematic diagram of an example system 100 capable of implementing the methods described herein, according to an example embodiment.

[0019] Referring to FIG. 1, the system 100 includes a client device 110, a server 120, and a network 130 communicatively coupling the client device 110 to the server 120.

[0020] The client device 110 includes a display 114 and a client application (APP) 112 that can be displayed via the display 114. The client application 112 may be an application program that requires downloading and installation before it can be run, or a lightweight application program such as a mini-APP (lite app). If the client application 112 is an application program that requires downloading and installation before it can be run, the client application 112 may be pre-installed and activated on the client device 110. If the client application 112 is a mini-APP, the client application 112 does not need to be installed, and the user 102 can run the client application 112 directly on the client device 110 by searching for the client application 112 (e.g., by name) in a host application or by scanning a graphic code (e.g., a barcode, QR code, etc.) of the client application 112. In some embodiments, the client device 110 may be any type of mobile computing device, including a mobile computer, a mobile phone, a wearable computing device (e.g., a head-mounted device such as a smart watch or smart glasses), or another type of mobile device. In some embodiments, client device 110 may alternatively be a stationary computing device, such as a desktop computer, a server computer, or other type of stationary computing device.

[0021] Server 120 is typically a server deployed by an Internet Service Provider (ISP) or Internet Content Provider (ICP). Server 120 can represent a single server, a cluster of servers, a distributed system, or a cloud server providing underlying cloud services (e.g., cloud database, cloud computing, cloud storage, cloud communication, etc.). It will be appreciated that although server 120 is shown in FIG. 1 communicating with only one client device 110, server 120 can provide background services to multiple client devices simultaneously.

[0022] Examples of network 130 include a local area network (LAN), a wide area network (WAN), a personal area network (PAN), and / or a combination of communication networks, such as the Internet. Network 130 may be a wired or wireless network. In some embodiments, technologies and / or formats including HyperText Markup Language (HTML), Extensible Markup Language (XML), etc. may be used to process data exchanged over network 130. Note that all or some links may be encrypted using encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. In some embodiments, custom and / or proprietary data communication technologies may be used to replace or supplement the above data communication technologies.

[0023] 1 , the client application 112 may be a scrap iron image collection application program, which may provide various functions related to the scrap iron image collection process, such as segmenting the scrap iron unloading area, selecting a reference shooting position, and setting parameters for the image collection device and / or the 3D scanning device. Accordingly, the server 120 may be a server used in conjunction with the scrap iron image collection application program. The server 120 may provide an online scrap iron image collection service to the client application 112 running on the client device 110. Alternatively, the server 120 may provide a local scrap iron image collection service to the client application 112 running on the client device 110.

[0024] The system 100 of FIG. 1 may be configured and operated in a variety of ways to accommodate the various methods and apparatus described in this disclosure.

[0025] In related technology, steel mills often use manual quality inspection to grade the scrap they purchase. After the scrap is collected, stored, and transshipped, the grading results are manually entered and uploaded into an ERP or MES system. However, manual quality inspection has the following drawbacks: 1) During the manual grading process for scrap, the grader must climb the cargo ship or transshipment vehicle carrying the scrap. This creates blind spots, making it difficult to measure the entire scrap unloading area by eyeballing, posing a significant safety risk. 2) The grading process relies entirely on the grader's subjective experience, making it prone to misjudgments, which can easily lead to disputes with scrap suppliers and lacks a means for post-verification. 3) Manual grading does not allow for all-weather automatic grading, resulting in long workflow times and impacting the efficiency of scrap transshipment.

[0026] With the continuous advancement of technology, some scrap ferrous grading methods based on machine vision have also begun to participate in the field of scrap ferrous grading, but there is a large overlap in the areas photographed by these scrap ferrous grading methods based on machine vision, and the images of each scrap ferrous material are large or small, which affects the accuracy of grading the scrap ferrous material of the entire ship or vehicle.

[0027] Therefore, the present disclosure provides a more efficient and versatile machine vision-based method and system for collecting images of scrap iron, which can replace the traditional method of manual quality inspection for scrap iron grading, realize continuous and all-weather smart collection of images of scrap iron both indoors and outdoors, and improve the efficiency and accuracy of scrap iron grading.

[0028] 2 is an exemplary flowchart of a method 200 for collecting images of steel scrap according to an embodiment of the present disclosure. The client or server shown in FIG. 1 may be utilized to implement the method 200 in FIG.

[0029] As shown in FIG. 2, an embodiment of the present disclosure provides a method 200 for collecting images of iron scrap, which includes the following steps 210 to 240.

[0030] In some embodiments, steps S210 to S240 may be performed in the process of the iron gripping device transferring one lot of iron scrap to the iron scrap unloading area each time.

[0031] Step S210: Identify a sub-area in the iron scrap unloading area where one lot of iron scrap is unloaded by the iron gripping device and a photographing position corresponding to the sub-area, where the photographing position represents the position in the sub-area where the iron gripping device unloads the lot of iron scrap.

[0032] In examples, the ferrous gripping device may be an ferrous gripper or an electromagnetic suction cup, etc., although the present disclosure is not limited thereto.

[0033] In examples, the scrap iron unloading area may be the deck of a cargo ship or the loading platform of a transfer vehicle capable of loading scrap iron, and the present disclosure is not limited thereto.

[0034] As used in this disclosure, the term "one lot of scrap iron" may refer to a collection of scrap iron that is transferred in one go from a scrap iron transport side to a scrap iron recovery side by a scrap iron gripping device (e.g., transferred from a cargo ship loaded with scrap iron that is docked at a wharf to a scrap iron unloading area (e.g., a scrap iron transport truck) on the wharf, or vice versa).

[0035] The term "shooting position" as used in this disclosure may refer to a focal point in an image to be captured monitored by a finder (e.g., a hardware finder or a software finder) of an image acquisition device, i.e., an imaging focus point at which a lens of an image acquisition device focuses in an image to be captured or a sharpest focus point in an image to be captured. As can be understood, the shooting position does not have to be located at the geometric center of the image to be captured, and the number of shooting positions in an image to be captured may be one or more, and the present disclosure is not limited thereto.

[0036] Step S220: In response to determining that the iron gripping device has left the iron scrap unloading area, obtain three-dimensional point cloud data for the sub-area.

[0037] As can be understood, since the volume of the iron gripping device is generally very large (e.g., occupies multiple sub-areas within the iron scrap unloading area), in order to increase the transfer efficiency and maximize the weight of each lot of iron scrap that is grabbed (e.g., thereby reducing the workload of the iron gripping device traveling back and forth to transfer the iron scrap), acquiring 3D point cloud data for the sub-areas where the unloaded lot of iron scrap is located after the iron gripping device leaves the iron scrap unloading area can prevent the acquired 3D point cloud data from including 3D point cloud information about the iron gripping device, and further effectively reduce or eliminate noise in the 3D point cloud data for the sub-areas.

[0038] The term "three-dimensional point cloud data" as used in this disclosure may refer to a set of points in the form of spatial coordinates for each sampling point on an object surface acquired by scanning the object with a three-dimensional scanning device (e.g., a laser radar). The three-dimensional point cloud data may include not only coordinate information but also intensity information and / or angle information, and the present disclosure is not limited thereto.

[0039] In an example, three-dimensional point cloud data for a sub-region may be acquired by a three-dimensional scanning device.

[0040] In an example, to save costs, two-dimensional image data for a sub-region (e.g., a photographed image of the sub-region) may be acquired by an image acquisition device, and then three-dimensional point cloud data may be generated based on the two-dimensional image data by neural radiation fields (NeRF) technology. The present disclosure does not limit the acquisition method of the three-dimensional point cloud data.

[0041] Step S230: Determine the magnification ratio for capturing the sub-region based on the three-dimensional point cloud data.

[0042] In practice, each lot of scrap ferrous metal is typically unloaded into a scrap ferrous metal unloading area in a fixed sequence.

[0043] By way of example and not limitation, assume that the scrap iron unloading area is initially empty and does not contain any lots of scrap iron to be transferred therein. Assuming that the scrap iron unloading area is divided into N non-overlapping sub-areas, where the N sub-areas completely cover the area occupied by the scrap iron unloading area, the first N lots of scrap iron are unloaded into the corresponding N sub-areas, respectively, and the N lots of scrap iron form a first layer of scrap iron stacks in the scrap iron unloading area. Subsequently, the N+1th through 2Nth lots of scrap iron are similarly unloaded into the corresponding N sub-areas, respectively, and the N+1th through 2Nth lots of scrap iron form a second layer of scrap iron stacks in the scrap iron unloading area. Similar analogy applies.

[0044] As a result, the acquired three-dimensional point cloud data for the sub-region may be used as feature information characterizing a lot of iron scrap corresponding to the sub-region, and information about the volume, weight, height, distance (e.g., the distance between the unloading position of the iron scrap of the lot and a fixed point such as an image collection device) of the lot of iron scrap corresponding to the sub-region can be directly or indirectly derived from such three-dimensional point cloud data.

[0045] In an example, a magnification factor for photographing a sub-region corresponding to a lot of iron scrap may be determined based on the distance between a drop-off location of the iron scrap included in the 3D point cloud data and a fixed point such as an image collection device. As can be understood, the greater the distance between the drop-off location of the iron scrap and a fixed point such as an image collection device, the greater the magnification factor required to photograph the corresponding sub-region. Of course, the present disclosure does not limit the relationship (e.g., functional relationship) between the distance between the drop-off location of the iron scrap and a fixed point such as an image collection device and the magnification factor required to photograph the corresponding sub-region.

[0046] In an example, when the height of each point of a given layer of scrap iron stack does not vary significantly, the acquired three-dimensional point cloud data for a sub-area may be three-dimensional point cloud data for the entire scrap iron stack in which one lot of scrap iron corresponding to the sub-area is located, thereby reducing the workload of the three-dimensional scanning device for each sub-area in the layer of scrap iron stack and improving efficiency, so that the three-dimensional point cloud data of the scrap iron stack for that layer may be acquired as three-dimensional point cloud data for each sub-area.

[0047] For example, continuing with the non-limiting example of the iron scrap stack described above, assuming that the height of each point in a given layer of the iron scrap stack does not vary significantly, for at most N sub-areas in the iron scrap stack in that layer, three-dimensional point cloud data of the iron scrap stack in that layer can be obtained as three-dimensional point cloud data for each of these at most N sub-areas, and from such three-dimensional point cloud data, it can be concluded that the height information of the shooting positions corresponding to each of these at most N sub-areas may be consistent.

[0048] Step S240: start collecting images of the iron scrap of the lot based on the photographing position and magnification.

[0049] In an example, starting to collect images of a lot of iron scrap based on a photographing position and a magnification may include sending an instruction to an image collection device to focus on a corresponding sub-region where the iron scrap is located based on the photographing position included in the instruction, and photographing the sub-region based on the magnification included in the instruction to obtain an image of the sub-region, i.e., an image of the iron scrap of the lot corresponding to the sub-region.

[0050] In the related art, the machine vision-based grading method for iron scrap has many shortcomings, such as large overlaps between photographed areas and the images of each iron scrap being large or small, all of which affect the accuracy of grading the iron scrap for the entire ship or vehicle. Method 200 not only accurately positions the iron gripping device to unload a lot of iron scrap, ensuring that each lot of iron scrap being unloaded is comprehensively photographed and avoiding overlaps in the photographed areas, but also automatically adjusts the magnification of the photograph for the corresponding photographed area to ensure that the sizes of the photographed images of each lot of iron scrap are relatively consistent, thereby improving the accuracy of subsequent iron scrap grading.

[0051] 3 is an exemplary flowchart of additional steps of a method for collecting images of ferrous scrap according to an embodiment of the present disclosure. As shown in FIG. 3, before the ferrous gripping device transfers each lot of ferrous scrap to the ferrous scrap unloading area, the method 200 further includes steps S201-S202.

[0052] Step S201: Acquire an initial image including the iron scrap unloading area.

[0053] In an example, the initial image may be obtained from a data file containing previously collected images of the scrap ferrous unloading area.

[0054] By way of example and not limitation, if the scrap iron unloading area is the loading platform of a transfer vehicle (e.g., a truck) capable of loading scrap iron, such data file may include previously collected overhead or near-overhead images of the loading platform of the transfer vehicle (e.g., a truck hopper) of various vehicle types.

[0055] In another example, before the iron gripping device grips the first lot of iron scrap to be transferred, an initial image including the iron scrap unloading area may be acquired by the image acquisition device.

[0056] As used in this disclosure, when referring to an image including a scrap ferrous drop-off area, it means that the image includes the scrap ferrous drop-off area, whereby an image of the scrap ferrous drop-off area can be obtained by a cropping operation or the like.

[0057] Step S202: Based on the initial image, the iron scrap unloading area is divided to obtain a plurality of sub-areas of the iron scrap unloading area.

[0058] In an example, the scrap ferrous unloading area may be segmented based on the initial image based on factors such as the average volume of scrap ferrous to be transferred for each lot grasped by the scrap ferrous gripping device, the operating mode of the arms of the scrap ferrous gripping device, and the aspect ratio of the scrap ferrous unloading area.

[0059] For example, the scrap iron unloading area may be divided into a series of sub-areas (especially when the aspect ratio of the scrap iron unloading area is large) or into a mesh of sub-areas (e.g., when the length and width of the scrap iron unloading area are not significantly different). Of course, the resulting division may result in the scrap iron unloading area including only one sub-area, for example, when the amount of scrap iron to be transferred for each lot grasped by the iron gripping device occupies the entire area of ​​the scrap iron unloading area on average, or when the area of ​​the scrap iron unloading area itself is too small to be divided into two or more sub-areas. The present disclosure does not limit the division method of the scrap iron unloading area.

[0060] 4 is an exemplary flowchart of additional steps of a method for collecting images of ferrous scrap according to an embodiment of the present disclosure. As shown in FIG. 4, step S210 of method 200, i.e., identifying a sub-area in the ferrous scrap unloading area where a lot of ferrous scrap is unloaded by the ferrous gripping device and a photographing position corresponding to the sub-area, may include steps S211 to S214.

[0061] Step S211: Obtain a video stream recording the process of the iron gripping device transferring one lot of iron scrap to the iron scrap unloading area.

[0062] In an example, the video stream may adopt a mainstream video file format to facilitate subsequent analysis and processing of the video stream.

[0063] In another example, a video stream may refer to a sequence of images collected consecutively in time, and the images in the sequence may adopt mainstream image file formats to facilitate subsequent image processing on these images. This disclosure does not limit the format of the video stream.

[0064] Step S212: Determine the motion trajectory of the iron gripping device based on the video stream.

[0065] In an example, the motion trajectory of the iron gripping device may be determined by identifying the position coordinates of the iron gripping device from the video stream.

[0066] By way of example and not limitation, the image acquisition device may be a gun-ball integrated imaging device. Thus, for example, a deep learning algorithm may determine (e.g., calculate) the motion trajectory of the iron gripping device from video frames or images captured using the global field of view of the gun-type camera in the gun-ball integrated imaging device.

[0067] In an example, the motion trajectory of the iron gripping device may include a transfer trajectory in which the iron gripping device grabs a lot of iron scrap from the iron scrap transport side (e.g., the deck of a cargo ship loaded with iron scrap moored at a pier) and transfers and unloads the lot of iron scrap to the iron scrap unloading area of ​​the iron scrap recovery side, and a motion trajectory in which the iron gripping device leaves the iron scrap unloading area after completing unloading of the lot of iron scrap and returns to the iron scrap transport side.

[0068] Step S213: determine a sub-area in the scrap iron dropping area where the scrap iron of the batch is dropped by the scrap iron holding device according to the motion trajectory of the scrap iron holding device.

[0069] In an example, a sub-area in which a singular point (e.g., a turning point or a location representing a long dwell time of the ferrous gripping device) identified in the motion trajectory of the ferrous gripping device is located may be determined as the sub-area in which the ferrous scrap for that lot is unloaded.

[0070] Step S214: Using deep learning technology, determine the position where the iron gripping device drops off the iron scrap of the lot as the photographing position.

[0071] It should be noted that deep learning techniques themselves are known to those skilled in the art, and any type of deep learning technique can be used to determine the shooting position, and the present disclosure is not limited thereto and will not be further described.

[0072] As described above, the photographing position refers to, for example, the imaging focus point where the lens of the image collecting device is focused in the image to be photographed or the sharpest focus point in the image to be photographed. Thus, the photographing position determined using deep learning technology can ensure that the photographed image covers the sub-area where the iron scrap of the lot is located and is better focused on the distribution center point of the iron scrap of the lot, thereby avoiding the occurrence of unfavorable situations such as the iron scrap being out of focus in the photographed image or focusing on the edge part of the iron scrap of the lot (e.g., a part with a small mass or volume of the iron scrap).

[0073] 5 is an exemplary flowchart of additional steps of a method for collecting images of ferrous scrap according to an embodiment of the present disclosure. As shown in FIG. 5, step S220 of method 200, i.e., acquiring three-dimensional point cloud data for a sub-area in response to determining that the ferrous gripping device has left the ferrous scrap unloading area, may include steps S221-S222.

[0074] Step S221: Determine that the iron gripping device has left the iron scrap unloading area based on at least one of the video stream and the motion trajectory of the iron gripping device.

[0075] In an example, based on the video stream acquired in step S211, it may be determined that the iron gripping device has left the iron scrap unloading area.

[0076] In another example, it may be determined that the iron gripping device has left the iron scrap unloading area based on the motion trajectory of the iron gripping device determined in step S212.

[0077] In yet another example, it may be determined that the ferrous gripping device has left the ferrous scrap unloading area based on both the video stream and the motion trajectory of the ferrous gripping device.

[0078] Step S222: start scanning the sub-region using a three-dimensional scanning device to obtain three-dimensional point cloud data of the sub-region;

[0079] As previously mentioned, scanning a sub-region using a three-dimensional scanning device may include scanning only the corresponding sub-region or scanning the entire scrap ferrous stack layer including the corresponding sub-region, and the present disclosure is not limited thereto.

[0080] Thus, after the iron gripping device leaves the iron scrap unloading area, acquiring three-dimensional point cloud data for the sub-area where the unloaded iron scrap is located can prevent three-dimensional point cloud information about the iron gripping device from being included in the acquired three-dimensional point cloud data, and further effectively reduce or remove noise in the three-dimensional point cloud data of the sub-area, thereby improving the accuracy of the acquired three-dimensional point cloud data.

[0081] 6 is an exemplary flowchart of additional steps of a method for collecting images of steel scrap according to an embodiment of the present disclosure. As shown in FIG. 6, step S230 of method 200, i.e., determining a magnification ratio for photographing a sub-region based on the three-dimensional point cloud data, may include steps S231 to S234.

[0082] Step S231: Using the three-dimensional point cloud data, perform surface reconstruction on one lot of iron scrap in the sub-region to obtain a surface mesh of the iron scrap of the lot.

[0083] It should be noted that performing surface reconstruction on an object using 3D point cloud data of the object is itself known to those skilled in the art, and any type of 3D point cloud processing means can be used to determine the surface mesh of a lot of iron scrap, and the present disclosure is not limited thereto and will not be further described.

[0084] In an example, based on the surface mesh of the obtained lot of ferrous scrap, the average height of the top surface of the lot of ferrous scrap may be calculated.

[0085] Step S232: Determine the height from the top surface of the iron scrap of the lot to the ground surface of the sub-region based on the surface mesh of the iron scrap of the lot and the ground point cloud data included in the 3D point cloud data.

[0086] The term "ground" as used in this disclosure is not limited to the earth's horizon and may refer to any given suitable reference plane.

[0087] By way of example and not limitation, the ground of a sub-area may refer to the bottom surface of the scrap iron unloading area, and in the case of a truck hopper, the ground may refer to the bottom inner or outer surface of the hopper.

[0088] As can be appreciated, when the ground plane of a sub-region is given, the distance between the detection port (e.g., lens, radiation transmitting / receiving port, etc.) of an electronic device such as an image acquisition device and / or a three-dimensional scanning device and the ground plane may be measurable.

[0089] As can be further appreciated, the distance between the detection port of an electronic device, such as an image collection device and / or three-dimensional scanning device, and the ground is generally greater than the distance between the top surface of any lot of ferrous scrap and the ground (i.e., the height of the sub-area from the top surface of the ferrous scrap of the lot to the ground), because the detection port of the electronic device must be able to perform image collection and / or radiation scanning of the sub-area from a bird's-eye or near-bird's-eye view.

[0090] In an example, the average height of the ground surface of the sub-region may be calculated based on the ground surface point cloud data included in the three-dimensional point cloud data.

[0091] In step S233, the distance between the image acquisition device and the photographing position is determined based on the position parameters of the image acquisition device photographing the sub-area relative to the scrap iron unloading area and the height from the top surface of the scrap iron of the lot to the ground of the sub-area.

[0092] In an example, the position parameters of the image collection device relative to the scrap iron unloading area may include the distance H between the lens of the image collection device and the ground of the sub-area (e.g., the bottom of the scrap iron unloading area) (i.e., the height from the image collection device to the ground of the sub-area), and the distance d from the shooting position to a perpendicular line to the ground that passes through the image collection device.

[0093] By way of example, and not by way of limitation, if the height from the top surface of a given lot of iron scrap to the ground in the sub-area in which the iron scrap of the lot is located is defined as h, then the distance s between the image acquisition device and the photographing position can be expressed as:

number

[0094] Step S234: Determine the magnification ratio for photographing the sub-region based on at least the distance between the image acquisition device and the photographing position.

[0095] In an example, the magnification at which the sub-region is imaged may be determined by varying the magnification depending on the distance between the image acquisition device and the image capture location.

[0096] As described above, the greater the distance between the scrap metal unloading position and a fixed point such as the image collecting device, the greater the magnification required to capture the corresponding sub-region. Therefore, in order to select an appropriate magnification for capturing the corresponding sub-region, a function may be constructed such that, for example, the magnification for capturing increases as the distance between the image collecting device and the capturing position increases.

[0097] 7 is an exemplary flowchart of additional steps of a method for collecting images of steel scrap according to an embodiment of the present disclosure. As shown in FIG. 7, step S234, i.e., determining a magnification for capturing images of the sub-region based on at least a distance between the image capture device and the capturing position, may include steps S2341 to S2343.

[0098] Step S2341: determining a reference photographing position in the iron scrap unloading area and a reference distance between the image collecting device and the reference photographing position;

[0099] In an example, the reference photography position may be a point in the scrap ferrous unloading area (e.g., the hopper of a transfer truck) where no transfer lots of scrap ferrous are loaded. In other words, the reference photography position may be established before each lot of scrap ferrous is transferred.

[0100] For example, but not by way of limitation, when the bottom inner surface of the hopper is the ground, the reference photographing position may be a point on the bottom inner surface of the hopper. If the distance between the lens of the collecting device and the ground is H0 and the distance from the reference photographing position to the perpendicular line of the ground passing through the image collecting device is d0, the reference distance s0 between the image collecting device and the reference photographing position is s0=sqrt(H0 2 +d0 2 ) can be determined.

[0101] As can be appreciated, the reference imaging positions may be selected by those skilled in the art as needed, and the present disclosure is not limited thereto.

[0102] Step S2342: The image collection device is focused on the reference shooting position, and the magnification of the image collection device is adjusted so that the image collection device can capture just the entire area of ​​the scrap iron unloading area, and the reference magnification of the image collection device is obtained.

[0103] As can be appreciated, the ability of the image collection device to capture just the entire area of ​​the scrap ferrous unloading area may in some cases mean that the boundary of the image captured by the image collection device just approximately overlaps with the periphery of the scrap ferrous unloading area (e.g., the frame of the truck hopper), thereby obtaining a reference magnification factor k0.

[0104] In step S2343, the magnification ratio for photographing the sub-region is determined based on the distance between the image acquisition device and the photographing position, the reference distance and the reference magnification ratio.

[0105] By way of example, and not by way of limitation, the magnification k for photographing the sub-region is given by

number

[0106] As can be understood, although the preceding paragraph describes determining the magnification factor k for photographing a sub-region in the form of a linear function, a person skilled in the art may select any suitable function formula as needed to determine the magnification factor of the image acquisition device for photographing different sub-regions, and the present disclosure is not limited thereto.

[0107] In some embodiments, initiating collection of images of the lot of iron scrap based on the imaging position and magnification may include focusing an image collection device on the imaging position, adjusting the magnification of the image collection device to the magnification, and photographing a sub-area to obtain an image of the lot of iron scrap.

[0108] This allows the magnification of the photograph to be automatically adjusted for the corresponding photographed area, ensuring that the photographed images of each lot of iron scrap are relatively consistent in size, thereby improving the accuracy of subsequent iron scrap grading.

[0109] FIG. 8 is a schematic diagram of an exemplary image collection scenario for steel scrap according to an embodiment of the present disclosure.

[0110] It should be noted that although FIG. 8 illustrates an image collection scenario for scrap iron during unloading by a harbor crane at a pier, those skilled in the art will appreciate that the image collection method for scrap iron of the present disclosure may be applied to any other suitable scrap iron transshipment scenario, and the present disclosure is not limited thereto.

[0111] The image collection scenario for iron scrap shown in Figure 8 includes an iron scrap transshipment device 1, a pole bracket device 2, an image collection device 3, a terminal card reader 4, an LED display device 5, an image processing device 6, a cabin 7, a transshipment platform 8, a gunball-integrated camera 9, a supplementary light 10, a port crane base 11, and an iron grabber 12.

[0112] As shown in the figure, the scrap iron transfer device 1 may be deployed on the quayside and is responsible for grabbing (or sucking) scrap iron from a cabin 7 and transferring it to a transfer platform 8. The pole bracket device 2 may be deployed near a parking space so that the camera faces the scrap iron transfer vehicle as directly as possible. The image collection device 3 (including a gun-ball-integrated camera 9 and a fill light 10, etc.) and the three-dimensional scanning device 13 may be mounted on the top of the pole bracket device 2. The terminal card reader 4 may be deployed at the bottom of the pole bracket device 2, and the LED display device 5 may be installed in the middle of the pole bracket device 2. The image processing device 6 may be remotely deployed in a server room and connected to the image collection device 3, the terminal card reader 4, the LED display device 5, and / or the three-dimensional scanning device 13 via a communication circuit (e.g., a wired communication circuit, a wireless communication channel, or a combination thereof).

[0113] As shown in the figure, the pole bracket device 2 may be shaped like an inverted "L", with a mounting bracket for a gunball-integrated camera 9 for the image acquisition device 3, a mounting bracket for a fill light 10, etc., on the top beam. The pole bracket device 2 may have a hollow design, and all communication lines and power supply lines can be routed and connected inside the bracket, thereby avoiding the lines being exposed to complex environments and improving safety and durability.

[0114] Furthermore, as shown in the figure, the iron scrap transshipment device 1 may be composed of a port crane base 11, an iron grabber 12, etc. The operator controls the interval at which the iron grabber 12 transships each lot of iron scrap according to the instructions displayed on the LED display device 5, and sequentially grabs the iron scrap in the cabin 7 and lowers it into the transshipment platform 8.

[0115] As can be seen, the gun-type camera in the gun-ball integrated camera 9 can be responsible for controlling the global view of the unloading area by the port crane at the quay, and the ball-type camera can be responsible for taking and collecting pictures of the scrap iron in the transfer platform 8. The supplementary light 10 can meet the lighting needs during nighttime work, and can also ensure the quality of the imaging of the scrap iron pictures during the day by reducing the impact of shadows caused by oblique sunlight on the transfer platform 8 on the imaging of the gun-ball integrated camera 9.

[0116] As further shown in the figure, the terminal card reader 4 may be installed at the bottom of the pole bracket device 2 and may be communicatively connected to other assemblies via communication circuits inside the pole bracket, thereby allowing an operator (e.g., a driver of a transfer vehicle) to initiate the smart collection and automatic grading process of scrap iron images.

[0117] Optionally, the LED display device 5 may be installed in the center of the pole bracket device 2, which is communicatively connected to other assemblies via a communication circuit inside the pole bracket and can display text information in various colors (e.g., displaying hints on operations that the operator of the iron scrap transfer device 1 needs to perform), and different information may be displayed in different colors according to type.

[0118] The image processing device 6 may include an image processing server as the core processing unit of the smart collection and automatic grading process of scrap iron images, which performs information interaction and control with each assembly at the wharf site via communication lines.

[0119] In an example, the image processing device 6 can automatically start the relevant task flow by receiving the logistics code information, the smart collection of scrap iron images, and the start command of the automatic grading process transmitted by the terminal card reader 4. After all the task flows are completed, the overall grading result of the transshipped scrap iron on the vehicle may be displayed on the interface of the terminal card reader 4 for the operator to view.

[0120] Optionally, interaction information such as logistics code information transmitted by the terminal card reader 4, smart collection of scrap iron images and commands to start the automatic grading process may alternatively be realized using a handheld terminal and associated applications.

[0121] Optionally, if the interaction information, such as logistics code information that needs to be transmitted, smart collection of scrap iron images and start command for automatic grading process, can be obtained directly from the merchant's traditional MES or ERP system interface, the terminal card reader 4 does not need to be installed.

[0122] Optionally, before each lot of iron scrap is transferred to the transfer platform 8, an image identification algorithm can be used to determine whether the parking position of the platform 8 of the transfer vehicle is within the best photographing range of the image collection device 3; if not, the parking of the platform 8 can be guided. For example, if the parking position of the vehicle does not meet the requirements, a presentation information can be generated and automatically fed back to the on-site LED display device 5 to present to the operator (e.g., the driver) that the parking position of the vehicle does not meet the requirements and that the parking adjustment needs to be made, until it is determined that the vehicle is parked near the best position.

[0123] Optionally, a photograph of each collected lot of scrap iron may be transmitted to the image processing device 6 and stored.

[0124] Accordingly, the computer vision algorithm in the image processing device 6 can perform classification on the collected images of the scrap iron from multiple feature dimensions, such as thickness, material type, size, diameter, density fitting, non-steel impurities, and dangerous sealant, to comprehensively obtain the evaluation level of the scrap iron to be transshipped for the entire vehicle. Additionally, when the image processing device 6 detects the presence of dangerous sealant in the image of the scrap iron, alert text information can be transmitted to the LED display device 5 via the communication circuit to promptly alert the operator of the scrap iron transshipment device 1 to take corresponding measures.

[0125] FIG. 9 is a block diagram of an apparatus for collecting images of steel scrap, according to an embodiment of the present disclosure.

[0126] As shown in FIG. 9 , according to an embodiment of the present disclosure, an apparatus 900 for collecting images of iron scrap is provided, the apparatus 900 including: a first module 910 for identifying a sub-area in a scrap iron unloading area where a lot of iron scrap is unloaded by an iron gripping device and a photographing position corresponding to the sub-area, where the photographing position represents a position in the sub-area where the iron gripping device unloads the lot of iron scrap; a second module 920 for acquiring three-dimensional point cloud data for the sub-area in response to determining that the iron gripping device has left the scrap iron unloading area; a third module 930 for determining a magnification ratio for photographing the sub-area based on the three-dimensional point cloud data; and a fourth module 940 for starting collecting images of the lot of iron scrap based on the photographing position and the magnification ratio.

[0127] Here, the operations of the above-mentioned modules 910 to 940 of the apparatus 900 are similar to the operations of steps 210 to 240 of the above-mentioned method 200, respectively, and will not be described again here. In some embodiments, the apparatus 900 may further include one or more additional modules or units for performing functions corresponding to additional steps of the method 200 (e.g., steps 201 to 202, steps 211 to 214, steps 221 to 222, steps 231 to 234, steps 2341 to 2343, etc.), and for the sake of simplicity of illustration, these modules or units are not shown in FIG.

[0128] In the related art, the machine vision-based grading method for iron scrap has many shortcomings, such as large overlaps between photographed areas and images of each iron scrap being large or small, all of which affect the accuracy of grading the iron scrap for the entire ship or vehicle. The apparatus 900 not only accurately positions the iron gripping device to unload a lot of iron scrap, ensuring that each lot of iron scrap being unloaded is comprehensively photographed and avoiding overlaps in the photographed areas, but also automatically adjusts the magnification of the photograph for the corresponding photographed area to ensure that the sizes of the photographed images of each lot of iron scrap are relatively consistent, thereby improving the accuracy of subsequent iron scrap grading.

[0129] In some embodiments, the apparatus 900 may further include a fifth module for acquiring an initial image including the iron scrap unloading area before the iron gripping device transfers each lot of iron scrap to the iron scrap unloading area, and for segmenting the iron scrap unloading area based on the initial image to acquire multiple sub-areas of the iron scrap unloading area.

[0130] In some embodiments, the first module 910 of the apparatus 900 may include: a first unit for acquiring a video stream recording a process in which the iron gripping device transfers a lot of iron scrap to the iron scrap unloading area; a second unit for determining a movement trajectory of the iron gripping device based on the video stream; a third unit for determining a sub-area in the iron scrap unloading area where the lot of iron scrap will be unloaded by the iron gripping device based on the movement trajectory of the iron gripping device; and a fourth unit for using deep learning technology to determine the position where the iron gripping device unloads the lot of iron scrap as the shooting position.

[0131] In some embodiments, the second module 920 of the apparatus 900 may include a fifth unit for determining that the iron gripping device has left the iron scrap unloading area based on at least one of the video stream and the movement trajectory of the iron gripping device, and a sixth unit for starting to scan the sub-area using the three-dimensional scanning device to obtain three-dimensional point cloud data of the sub-area.

[0132] In some embodiments, the third module 930 of the apparatus 900 may include: a seventh unit for performing surface reconstruction on a lot of iron scrap in the sub-area using the three-dimensional point cloud data to obtain a surface mesh of the iron scrap of the lot; an eighth unit for determining a height from the top surface of the iron scrap of the lot to the ground of the sub-area based on the surface mesh of the iron scrap of the lot and the ground point cloud data included in the three-dimensional point cloud data; a ninth unit for determining a distance between the image collection device and an imaging position based on position parameters of an image collection device that images the sub-area relative to the iron scrap unloading area and the height from the top surface of the iron scrap of the lot to the ground of the sub-area; and a tenth unit for determining a magnification ratio for imaging the sub-area based on at least the distance between the image collection device and the imaging position.

[0133] In some embodiments, the tenth unit may be further configured to determine a reference shooting position in the iron scrap unloading area and a reference distance between the image collection device and the reference shooting position, focus the image collection device on the reference shooting position, adjust the magnification of the image collection device so that the image collection device can capture just the entire area of ​​the iron scrap unloading area, obtain a reference magnification of the image collection device, and determine a magnification for photographing the sub-area based on the distance between the image collection device and the shooting position, the reference distance, and the reference magnification.

[0134] In some embodiments, the fourth module 940 of the apparatus 900 may be further configured to focus the image acquisition device at the imaging position, adjust the magnification of the image acquisition device to the magnification, and image the sub-area to obtain an image of the scrap iron of the lot.

[0135] While particular functionality is discussed above with reference to particular modules, it should be noted that the functionality of each module discussed herein may be separated into multiple modules, and / or at least some of the functionality of multiple modules may be combined into a single module. As discussed herein, a particular module performing an operation may include the particular module itself performing the operation or, alternatively, the particular module calling or otherwise accessing another assembly or module that performs the operation (or performs the operation in conjunction with the particular module). Thus, a particular module performing an operation may include the particular module itself performing the operation and / or another module that the particular module calls or otherwise accesses to perform the operation.

[0136] It should be further understood that this specification may describe various technologies in the general context of software or hardware elements or program modules. Each module described above in FIG. 9 may be implemented in hardware or hardware combining software and / or firmware. For example, these modules may be implemented as computer program code / instructions configured to be executed on one or more processors and stored on a computer-readable storage medium. Alternatively, these modules may be implemented as hardware logic / circuitry. The hardware logic / circuitry may include one or more components of integrated circuit chips (e.g., a processor (e.g., a central processing unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.)), memory, one or more communication interfaces, and / or other circuitry) and may optionally execute received program code and / or include embedded firmware to perform functions.

[0137] 10 is a block diagram of a system for collecting images of iron scrap according to an embodiment of the present disclosure. As shown in FIG. 10, the system 1000 for collecting images of iron scrap may include an image collection device 1010 for collecting images including at least a portion of an iron scrap unloading area, a three-dimensional scanning device 1020 for scanning at least a portion of the iron scrap unloading area to obtain three-dimensional point cloud data of the corresponding area, and a processing device 1030 operably connected to the image collection device 1010 and the three-dimensional scanning device 1020 to perform additional steps of a method for collecting images of iron scrap according to the present disclosure, such as method 200 and each embodiment thereof.

[0138] As can be appreciated, image acquisition device 1010 may include any suitable type of camera, video camera, etc., and this disclosure is not limited thereto. Three-dimensional scanning device 1020 may include any suitable type of commercially available three-dimensional scanning equipment, such as radar or laser radar, and this disclosure is not limited thereto. Processing device 1030 may be any customized or commercially available processor, such as a central processing unit (CPU), a distributed processing unit, a semiconductor-based (e.g., in the form of a microchip or chipset) microprocessor, etc., and this disclosure is not limited thereto.

[0139] According to embodiments of the present disclosure, an electronic device, a readable storage medium, and a computer program product are further provided.

[0140] According to an embodiment of the present disclosure, there is provided an electronic device, which may include at least one processor and a memory communicatively coupled to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the method for collecting images of ferrous scrap of the present disclosure.

[0141] According to an embodiment of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions that may be used to cause a computer to perform the method for collecting images of ferrous scrap of the present disclosure.

[0142] According to an embodiment of the present disclosure, there is provided a computer program product, which may include a computer program that, when executed by a processor, can implement the method for collecting images of steel scrap of the present disclosure.

[0143] As can be seen, the scrap iron image collection method disclosed herein has the advantages of being versatile, automatic, and requiring no human intervention. The scrap iron image collection method disclosed herein can accurately locate unloading locations, ensure comprehensive photography of more scrap iron, and effectively reduce the probability of photographing the same location multiple times, thereby significantly improving photography coverage, reducing the overlap rate of photographed areas, and more accurately grading the scrap iron for the entire vehicle. Furthermore, the scrap iron image collection method disclosed herein can self-adaptively adjust the magnification of photography, ensuring the stability of the image size of each photographed scrap iron and improving the accuracy of identification.

[0144] Referring to FIG. 11 , a block diagram of an electronic device 1100 that can be used as a server or client of the present disclosure is described, which is an example of a hardware device applicable to various aspects of the present disclosure. The electronic device represents various types of digital electronic computing devices, such as laptop computers, desktop computers, stages, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital processing devices, cellular phones, smartphones, wearable devices, and other similar computing devices. The components, their connections, and their functions shown herein are merely exemplary and are not intended to limit the practice of the present disclosure as described and / or claimed herein.

[0145] 11, electronic device 1100 includes a computing unit 1101, which can perform various appropriate operations and processes based on a computer program stored in a read-only memory (ROM) 1102 or loaded from a storage unit 1108 into a random access memory (RAM) 1103. The RAM 1103 may further store various programs and data necessary for the operation of electronic device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0146] The components of the electronic device 1100 are connected to an I / O interface 1105 and include an input unit 1106, an output unit 1107, a storage unit 1108, and a communication unit 1109. The input unit 1106 may be any type of device capable of inputting information into the electronic device 1100. The input unit 1106 can receive input numeric or character information and generate key signal input for user settings and / or function control of the electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackboard, trackball, joystick, microphone, and / or remote control. The output unit 1107 may be any type of device capable of presenting information, and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. The storage unit 1108 may include, but is not limited to, a magnetic disk or an optical disk. The communication unit 1109 enables the electronic device 1100 to exchange information / data with other devices via a computer network, e.g., the Internet, and / or various telecommunications networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, e.g., a Bluetooth® device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0147] The computing unit 1101 may be any of a variety of general-purpose and / or special-purpose processing assemblies having processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning network algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the methods and processes described above, such as method 200 and its embodiments. For example, in some embodiments, method 200 and its embodiments may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, some or all of the computer program may be loaded and / or installed into the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, it may perform one or more steps of the method 200 described above and additional steps of its embodiments. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform the additional steps of the method 200 and its respective embodiments in any other suitable manner (eg, by firmware).

[0148] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: Implemented in one or more computer programs, which may be executed and / or interpreted by a programmable system including at least one programmable processor, which may be a special purpose or general purpose programmable processor, and which may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0149] Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that when the program code is executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program code may be entirely executed on a machine, partially executed on a machine, partially executed on a machine and partially executed on a remote machine as a separate software package, or entirely executed on a remote machine or server.

[0150] In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wire-based electrical connections, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0151] To provide for interaction with a user, the systems and techniques described herein may be implemented in a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, etc.) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse and trackball, etc.) by which a user may provide input to the computer. Other types of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback), and input from the user may be received in any form (including sound, speech, or tactile input).

[0152] The systems and techniques described herein may be implemented in a computing system including backstage components (e.g., a data server), middleware components (e.g., an application server), front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with the system or technique implementation), or any combination of backstage components, middleware components, or front-end components. The components of the system may be interconnected (e.g., via a communications network) by any form or medium of digital data communication. Examples of communications networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0153] The computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship is created by running computer programs on corresponding computers that have a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server incorporating a blockchain.

[0154] It should be understood that steps may be reordered, added, or deleted using the various types of flows shown above. For example, the steps described in this disclosure may be performed in parallel, sequentially, or in a different order, and this specification is not limited thereto, as long as the technical solutions disclosed in this disclosure can achieve the desired results.

[0155] Although the embodiments or examples of the present disclosure have been described with reference to the drawings, it should be understood that the above-described methods, systems, and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the appended claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalent elements. Furthermore, steps may be performed in a different order than described in this disclosure. Furthermore, various elements in the embodiments or examples may be combined in various ways. In essence, as technology evolves, many elements described herein may be replaced by equivalent elements that appear later in this disclosure.

Claims

1. 1. A method for collecting images of ferrous scrap, performed by an apparatus for collecting images of ferrous scrap, comprising: During the process of the iron gripping device transferring one batch of iron scrap to the iron scrap unloading area each time, the first module: identifying a sub-area in the scrap iron drop-off area where the lot of scrap iron will be dropped off by the iron gripping device and a photographing location corresponding to the sub-area, the photographing location representing a location in the sub-area where the iron gripping device will drop off the lot of scrap iron; acquiring, by a second module, three-dimensional point cloud data for the sub-area in response to determining that the iron gripping device has left the iron scrap unloading area; determining, by a third module, a magnification ratio for photographing the sub-region based on the three-dimensional point cloud data; and starting, by a fourth module, collecting images of the lot of iron scrap based on the photographing position and the magnification. method.

2. Before the iron gripping device transfers each lot of iron scrap to the iron scrap unloading area, the method includes: acquiring an initial image including the iron scrap unloading area; The method of claim 1 , further comprising: segmenting the ferrous scrap drop-off area based on the initial image to obtain a plurality of sub-areas of the ferrous scrap drop-off area.

3. The step of identifying a sub-area in the scrap iron dropping area where the scrap iron lot is dropped by the scrap iron gripping device and a photographing position corresponding to the sub-area includes: acquiring a video stream recording the process of the iron gripping device transferring the lot of iron scrap to the iron scrap unloading area; determining a motion trajectory of the iron gripping device based on the video stream; determining the sub-area within the iron scrap unloading area where the lot of iron scrap is unloaded by the iron gripping device based on the motion trajectory of the iron gripping device; and determining, as the photographing position, a position where the iron gripping device drops off the lot of iron scrap using deep learning techniques.

4. acquiring three-dimensional point cloud data for the sub-area in response to determining that the iron gripping device has left the iron scrap unloading area, determining that the iron gripping device has left the iron scrap unloading area based on at least one of the video stream and the movement trajectory of the iron gripping device; and commencing scanning the sub-region using a three-dimensional scanning device to obtain three-dimensional point cloud data of the sub-region.

5. The step of determining a magnification ratio for photographing the sub-region based on the three-dimensional point cloud data includes: performing surface reconstruction on the one lot of iron scrap in the sub-region using the three-dimensional point cloud data to obtain a surface mesh of the one lot of iron scrap; determining a height from the top surface of the one lot of iron scrap to the ground of the sub-region based on the surface mesh of the one lot of iron scrap and ground point cloud data included in the three-dimensional point cloud data; determining a distance between the image acquisition device and the image acquisition position based on a position parameter of an image acquisition device that images the sub-area relative to the scrap iron unloading area and the height from the top surface of the lot of scrap iron to the ground of the sub-area; determining the magnification at which to image the sub-region based on at least the distance between the image acquisition device and the image capture position.

6. determining the magnification for capturing the image of the sub-region based on at least the distance between the image acquisition device and the capturing position, determining a reference photographing position within the scrap iron unloading area and a reference distance between the image acquisition device and the reference photographing position; The method includes: focusing the image acquisition device on the reference photographing position and adjusting a magnification of the image acquisition device so that the image acquisition device can capture just the entire area of ​​the scrap iron unloading area, thereby obtaining a reference magnification of the image acquisition device; and determining the magnification for photographing the sub-area based on the distance between the image acquisition device and the photographing position, the reference distance, and the reference magnification. The method of claim 5.

7. The step of starting to collect images of the one lot of iron scrap based on the photographing position and the magnification includes:

2. The method of claim 1, further comprising the steps of focusing an image acquisition device at the imaging position, adjusting a magnification of the image acquisition device to the magnification, and photographing the sub-area to obtain an image of the lot of iron scrap.

8. 1. An apparatus for collecting images of ferrous scrap, comprising: a first module for identifying a sub-area in the iron scrap unloading area where the iron gripping device unloads one lot of iron scrap each time, and a photographing position corresponding to the sub-area, wherein the photographing position represents a position in the sub-area where the iron gripping device unloads the one lot of iron scrap; a second module for acquiring three-dimensional point cloud data for the sub-area in response to determining that the ferrous gripping device has left the ferrous scrap unloading area; a third module for determining a magnification ratio for photographing the sub-region based on the three-dimensional point cloud data; and a fourth module for starting to collect images of the one lot of iron scrap based on the photographing position and the magnification. Device.

9. 1. A system for collecting images of iron scrap, comprising: an image acquisition device for acquiring an image including at least a portion of the scrap ferrous unloading area; a three-dimensional scanning device for scanning at least a portion of a steel scrap unloading area to obtain three-dimensional point cloud data of the corresponding area; A system for collecting images of iron scrap, comprising: a processing device operably connected to the image collection device and the three-dimensional scanning device, for performing the method of any one of claims 1 to 7.

10. An electronic device, at least one processor; and at least one memory communicatively connected to said at least one processor, said at least one memory storing instructions that, when executed alone or jointly by said at least one processor, cause said at least one processor to perform the method of any one of claims 1 to 7. electronic equipment.

11. A non-transitory computer readable storage medium having stored thereon computer instructions which, when executed alone or jointly by one or more processors of a computer, cause the computer to perform the method of any one of claims 1 to 7.

12. A computer program comprising computer instructions which, when executed by one or more processors of a computer, alone or in combination, cause the computer to carry out a method according to any one of claims 1 to 7.

Images