Methods, devices, systems, electronic equipment, and media for acquiring images of scrap steel

Abstract

A method for acquiring scrap steel images is provided, comprising: during the transfer of each batch of scrap steel to an unloading area, identifying a sub-region within the unloading area to which the batch of scrap steel is unloaded by a steel grabbing device and a corresponding photographing point within the sub-region, wherein the photographing point indicates the location within the sub-region where the steel grabbing device unloads the batch of scrap steel; in response to determining that the steel grabbing device has left the unloading area, acquiring three-dimensional point cloud data about the sub-region; determining the magnification of the photographing of the sub-region based on the three-dimensional point cloud data; and initiating the acquisition of images of the batch of scrap steel based on the photographing point and the magnification. A scrap steel image acquisition system is also provided, comprising: an image acquisition device for acquiring images including at least a portion of the unloading area; a three-dimensional scanning device for scanning at least a portion of the unloading area to obtain three-dimensional point cloud data of the corresponding area; and a processing device operatively connected to the image acquisition device and the three-dimensional scanning device for executing the scrap steel image acquisition method.

Description

Technical Field

[0001] This disclosure relates to the field of machine vision, and in particular to a method, apparatus, scrap steel image acquisition system, electronic device, computer-readable storage medium, and computer program product for acquiring images of scrap steel. Background Technology

[0002] With the development requirements of a green economy and the goal of achieving carbon peaking and carbon neutrality as soon as possible, many steel mills, both domestically and internationally, are using scrap steel as an important raw material for steelmaking in order to reduce their dependence on iron ore. Steel mills need to purchase large quantities of scrap steel every year, and the types, shapes, and sizes of scrap steel are vast, complex, and often contain impurities such as soil and oil. Traditionally, steel mills have relied on manual visual inspection to grade the purchased scrap steel, and the grading results are manually entered into and uploaded to ERP or MES systems after the scrap steel is collected, stored, and transported. Summary of the Invention

[0003] This disclosure provides a method, apparatus, scrap steel image acquisition system, electronic device, computer-readable storage medium, and computer program product for acquiring images of scrap steel.

[0004] According to one aspect of this disclosure, a method for acquiring images of scrap steel is provided, comprising: during the process of a steel grabbing device transferring a batch of scrap steel to a scrap steel unloading area each time, identifying a sub-region within the scrap steel unloading area to which the batch of scrap steel is unloaded by the steel grabbing device and a corresponding photographing point for the sub-region, the photographing point indicating the location within the sub-region where the steel grabbing device unloads the batch of scrap steel; in response to determining that the steel grabbing device has left the scrap steel unloading area, acquiring three-dimensional point cloud data about the sub-region; determining a magnification factor for photographing the sub-region based on the three-dimensional point cloud data; and initiating the acquisition of images of the batch of scrap steel based on the photographing point and the magnification factor.

[0005] According to another aspect of this disclosure, an apparatus for acquiring images of scrap steel is provided, comprising: a first module for identifying a sub-region within the scrap steel unloading area to which the batch of scrap steel is unloaded by the steel grabbing device and a corresponding photographing point within the sub-region, wherein the photographing point indicates the location within the sub-region where the steel grabbing device unloads the batch of scrap steel; a second module for acquiring three-dimensional point cloud data about the sub-region in response to determining that the steel grabbing device has left the scrap steel unloading area; a third module for determining a magnification factor for photographing the sub-region based on the three-dimensional point cloud data; and a fourth module for initiating the acquisition of images of the batch of scrap steel based on the photographing point and the magnification factor.

[0006] According to another aspect of this disclosure, a scrap steel image acquisition system is provided, comprising: an image acquisition device for acquiring an image including at least a portion of a scrap steel unloading area; a three-dimensional scanning device for scanning at least a portion of the scrap steel unloading area to obtain three-dimensional point cloud data of the corresponding area; and a processing device operatively connected to the image acquisition device and the three-dimensional scanning device for performing the method of this disclosure for acquiring scrap steel images.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory storing instructions executable by the at least one processor to enable the at least one processor to perform the method of this disclosure for acquiring images of scrap steel.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to execute the method of this disclosure for acquiring images of scrap steel.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method of acquiring images of scrap steel disclosed herein.

[0010] According to one or more embodiments of this disclosure, not only can the unloading position of the steel grabbing device be accurately located, but also the overlapping of the photographing areas can be avoided while ensuring that each batch of unloaded scrap steel is photographed in a comprehensive manner. Furthermore, the magnification can be automatically adjusted for the corresponding photographing area to ensure that the size of each batch of scrap steel photographed is relatively uniform, thereby improving the accuracy of subsequent scrap steel rating.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0013] Figure 1 A schematic diagram of an exemplary system in which the various methods described herein may be implemented according to embodiments of the present disclosure is shown;

[0014] Figure 2An exemplary flowchart of a method for acquiring images of scrap steel according to an embodiment of the present disclosure is shown;

[0015] Figure 3 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown;

[0016] Figure 4 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown;

[0017] Figure 5 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown;

[0018] Figure 6 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown;

[0019] Figure 7 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown;

[0020] Figure 8 An illustration of an exemplary scrap steel image acquisition scenario according to an embodiment of the present disclosure is shown.

[0021] Figure 9 A structural block diagram of an apparatus for acquiring images of scrap steel according to an embodiment of the present disclosure is shown;

[0022] Figure 10 A structural block diagram of a scrap steel image acquisition system according to an embodiment of the present disclosure is shown;

[0023] Figure 11 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0026] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0027] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0028] Figure 1 This is a schematic diagram illustrating an example system 100 in which various methods described herein may be implemented according to exemplary embodiments.

[0029] refer to Figure 1 The system 100 includes a client device 110, a server 120, and a network 130 that communicatively couples the client device 110 and the server 120.

[0030] Client device 110 includes a display 114 and a client application (APP) 112 that can be displayed on the display 114. Client application 112 can be an application that needs to be downloaded and installed before running, or a lightweight application (lite app). If client application 112 is an application that needs to be downloaded and installed before running, it can be pre-installed and activated on client device 110. If client application 112 is a mini-app, user 102 can run client application 112 directly on client device 110 without installing it, by searching for client application 112 in the host application (e.g., by the name of client application 112) or scanning the graphic code of client application 112 (e.g., barcode, QR code, etc.). In some embodiments, client device 110 can be any type of mobile computing device, including mobile computers, mobile phones, wearable computing devices (e.g., smartwatches, smart glasses, etc.), or other types of mobile devices. In some embodiments, client device 110 can alternatively be a fixed computing device, such as a desktop computer, server computer, or other types of fixed computing devices.

[0031] Server 120 is typically a server deployed by an Internet Service Provider (ISP) or Internet Content Provider (ICP). Server 120 can represent a single server, a cluster of multiple servers, a distributed system, or a cloud server providing basic cloud services (such as cloud databases, cloud computing, cloud storage, and cloud communications). It will be understood that, although... Figure 1 The diagram shows that server 120 communicates with only one client device 110, but server 120 can provide background services to multiple client devices simultaneously.

[0032] Examples of network 130 include combinations of local area networks (LANs), wide area networks (WANs), personal area networks (PANs), and / or communication networks such as the Internet. Network 130 can be wired or wireless. In some embodiments, technologies and / or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to process data exchanged through network 130. Furthermore, encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some of the links. In some embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0033] For the purposes of this disclosure's embodiments, Figure 1 In the example, client application 112 can be a scrap steel image acquisition application that provides various functions related to the scrap steel image acquisition process, such as the division of the scrap steel unloading area, the selection of reference imaging points, and the parameter settings of the image acquisition device and / or 3D scanning device. Correspondingly, server 120 can be a server used in conjunction with the aforementioned scrap steel image acquisition application. Server 120 can provide online scrap steel image acquisition services to client application 112 running on client device 110. Alternatively, local scrap steel image acquisition services can also be provided by client application 112 running on client device 110.

[0034] Figure 1 The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0035] In related technologies, steel mills typically use manual visual inspection to grade the scrap steel they acquire. After the scrap steel is collected, stored, and transferred, the grading results are manually entered and uploaded to the ERP or MES system. However, manual visual inspection has the following drawbacks: 1) During the manual visual grading process, grading personnel need to climb onto the cargo ships or transfer vehicles loaded with scrap steel, resulting in blind spots and making it difficult to visually inspect the entire scrap steel unloading area, which also poses significant safety hazards; 2) The grading process relies entirely on the subjective experience of the grading personnel, which is prone to misjudgment, potentially leading to disputes with scrap steel suppliers, and lacks means for post-event verification; 3) Manual grading cannot be automated around the clock, the process is time-consuming, and it affects the efficiency of scrap steel transfer.

[0036] With the continuous advancement of technology, some machine vision-based scrap steel rating methods have begun to be applied in the field of scrap steel rating. However, the areas captured by these machine vision-based scrap steel rating methods overlap significantly, and the size of each scrap steel image varies, affecting the accuracy of scrap steel rating for the entire ship or vehicle.

[0037] To this end, this disclosure proposes a more efficient and versatile machine vision-based scrap steel image acquisition method and system, which can replace the traditional scrap steel rating method of manual visual inspection, realize uninterrupted intelligent acquisition of scrap steel images indoors and outdoors around the clock, and improve the efficiency and accuracy of scrap steel rating.

[0038] Figure 2 An exemplary flowchart of a method 200 for acquiring images of scrap steel according to an embodiment of the present disclosure is shown. It can be utilized... Figure 1 The client or server implementation shown in the figure Figure 2 Method 200.

[0039] like Figure 2 As shown, according to an embodiment of the present disclosure, a method 200 for acquiring images of scrap steel is provided, including the following steps 210 to 240.

[0040] In some embodiments, steps S210 to S240 may be performed during the process of the steel grabbing device transferring a batch of scrap steel to the scrap steel unloading area each time.

[0041] Step S210: Identify a sub-area and a corresponding photo-taking location within the scrap steel unloading area to which a batch of scrap steel is unloaded by the steel grabbing device. The photo-taking location indicates the position within the sub-area where the steel grabbing device unloads the batch of scrap steel.

[0042] In the example, the steel-grabbing device can be a steel-grabbing machine or an electromagnet chuck, etc., and this disclosure does not impose any limitations on it.

[0043] In the example, the scrap unloading area can be the deck of a cargo ship that can load scrap or the compartment of a transfer vehicle, etc., and this disclosure does not impose any restrictions on it.

[0044] As used in this disclosure, the term "a batch of scrap steel" may refer to a collection of scrap steel that is transferred in one go from a scrap steel transporter to a scrap steel recycler by a scrap steel grabbing device (e.g., from a cargo ship loaded with scrap steel docked at a pier to a scrap steel unloading area on the pier (such as a scrap steel transport truck) or vice versa).

[0045] As used in this disclosure, the term "photographing point" can refer to the focal point within the image to be captured, monitored by the viewfinder (e.g., hardware or software viewfinder) of the image acquisition device; that is, the focal point of the image captured by the lens of the image acquisition device or the point of sharpest convergence within the image to be captured. It is understood that the photographing point need not be located at the geometric center of the image to be captured, and the number of photographing points within the image to be captured can be greater than or equal to one; this disclosure does not impose any limitation in this regard.

[0046] Step S220: In response to determining that the steel grabbing device has left the scrap steel unloading area, acquire three-dimensional point cloud data about the sub-region.

[0047] Understandably, in order to improve transfer efficiency and maximize the weight of each batch of scrap steel grabbed (e.g., thereby reducing the workload of the scrap grabbing device in transferring scrap steel back and forth), the scrap grabbing device is usually very large (e.g., occupying multiple sub-areas within the scrap steel unloading area). Therefore, acquiring 3D point cloud data about the sub-area where a batch of scrap steel was unloaded after the scrap grabbing device leaves the scrap steel unloading area can avoid the acquired 3D point cloud data containing 3D point cloud information about the scrap grabbing device, thereby effectively reducing or even eliminating noise in the 3D point cloud data of the sub-area.

[0048] As used in this disclosure, the term "three-dimensional point cloud data" can refer to a set of points in the form of spatial coordinates of each sample point on the surface of an object, obtained by scanning an object with the aid of a three-dimensional scanning device (such as LiDAR). Three-dimensional point cloud data may include not only coordinate information but also intensity information and / or angle information, etc., and this disclosure does not impose any limitations in this regard.

[0049] In the example, 3D point cloud data about a sub-region can be obtained using a 3D scanning device.

[0050] In the example, two-dimensional image data (e.g., an image of the captured sub-region) can be acquired using an image acquisition device, and then three-dimensional point cloud data can be generated based on the two-dimensional image data using Neural Radiance Fields (NeRF) technology to save costs. This disclosure does not impose any limitations on the method of acquiring three-dimensional point cloud data.

[0051] Step S230: Determine the magnification factor for taking pictures of the sub-region based on the 3D point cloud data.

[0052] In practice, each batch of scrap steel is usually unloaded into the scrap steel unloading area in a certain arrangement.

[0053] As an example, and not a limitation, the initial scrap unloading area is empty, excluding any batches of transferred scrap. Assuming the scrap unloading area is divided into N non-overlapping sub-areas, where these N sub-areas completely cover the area occupied by the scrap unloading area, the first N batches of scrap are unloaded into their respective N sub-areas, forming the first layer of scrap stock in the scrap unloading area. Subsequently, the (N+1)th to (2N)th batches of scrap are also unloaded into their respective N sub-areas, forming the second layer of scrap stock in the scrap unloading area, and so on.

[0054] Therefore, the obtained three-dimensional point cloud data of the sub-region can be used as characteristic information of a batch of scrap steel in the sub-region. Information such as volume, weight, height, and distance (e.g., the distance between the unloading location of the batch of scrap steel and a fixed point such as an image acquisition device) of the corresponding batch of scrap steel in the sub-region can be directly or indirectly derived from such three-dimensional point cloud data.

[0055] In the example, the magnification required to photograph a corresponding sub-region of the scrap steel can be determined based on the distance between the unloading location of a batch of scrap steel contained in the 3D point cloud data and a fixed point such as an image acquisition device. It is understood that the greater the distance between the unloading location of the scrap steel and the fixed point, the greater the magnification required to photograph the corresponding sub-region. Of course, this disclosure does not impose any limitations on the relationship (e.g., functional relationship) between the distance between the unloading location of the scrap steel and the fixed point, and the magnification required to photograph the corresponding sub-region.

[0056] In the example, given that the height of a given layer of scrap steel pile does not vary much in different places, the acquired 3D point cloud data for a sub-region can be the 3D point cloud data for the entire scrap steel pile containing a corresponding batch of scrap steel in that sub-region. Thus, for each sub-region in a layer of scrap steel pile, the 3D point cloud data of that layer of scrap steel pile can be acquired as the 3D point cloud data for each sub-region, thereby reducing the workload of the 3D scanning device and improving efficiency.

[0057] For example, following the non-limiting example of the scrap steel pile mentioned above, assuming that the height of a given layer of scrap steel pile does not vary much in different places, then for at most N sub-regions in that layer of scrap steel pile, the three-dimensional point cloud data of that layer of scrap steel pile can be obtained as the three-dimensional point cloud data of each of these at most N sub-regions, and it can be concluded from such three-dimensional point cloud data that the height information of the corresponding imaging points of these at most N sub-regions can be consistent.

[0058] Step S240: Initiate the acquisition of images of this batch of scrap steel based on the shooting location and magnification.

[0059] In the example, initiating the acquisition of images of a batch of scrap steel based on the shooting position and magnification may include: sending an instruction to the image acquisition device to make the image acquisition device focus on the corresponding sub-region where the scrap steel is located based on the shooting position included in the instruction, and taking a picture of the sub-region according to the magnification included in the instruction, that is, the image of the batch of scrap steel corresponding to the sub-region.

[0060] In related technologies, machine vision-based scrap steel grading methods have many shortcomings, such as significant overlap in the captured areas and inconsistent image sizes, all of which affect the accuracy of scrap steel grading for the entire ship or truckload. Method 200 not only accurately locates the position where the steel-grabbing device unloads a batch of scrap steel, thus ensuring comprehensive photography of each batch while avoiding overlapping areas, but also automatically adjusts the magnification of the images for each captured area, ensuring relatively uniform image size across batches and thereby improving the accuracy of subsequent scrap steel grading.

[0061] Figure 3 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown. Figure 3 As shown, before the steel grabbing device transfers each batch of scrap steel to the scrap steel unloading area, method 200 also includes steps S201 to S202.

[0062] Step S201: Obtain an initial image including the scrap steel unloading area.

[0063] In the example, the initial image can be obtained from a data file that contains previously collected images of the scrap unloading area.

[0064] As an example and not a limitation, in cases where the scrap unloading area is the cargo bed of a transfer vehicle (e.g., a truck) capable of loading scrap, such a data file may include previously acquired overhead or near-overhead views of the cargo beds (e.g., truck beds) of various types of transfer vehicles.

[0065] In another example, an image acquisition device can acquire an initial image of the scrap steel unloading area before the steel grabbing device grabs the first batch of scrap steel to be transferred.

[0066] As used in this disclosure, when referring to an image that includes a scrap steel unloading area, it means that the image contains the scrap steel unloading area, so that the image of the scrap steel unloading area can be obtained through cropping operations, etc.

[0067] Step S202: Divide the scrap steel unloading area based on the initial image to obtain multiple sub-regions of the scrap steel unloading area.

[0068] In the example, the scrap steel unloading area can be divided based on the initial image, taking into account factors such as the average volume of each batch of scrap steel to be transferred grabbed by the steel grabbing device, the working mode of the steel grabbing device's boom, and the aspect ratio of the scrap steel unloading area.

[0069] As an example and not a limitation, the scrap steel unloading area can be divided into multiple sub-areas arranged in a single row (especially when the length-to-width ratio of the scrap steel unloading area is large) or multiple sub-areas arranged in a grid pattern (e.g., when the length and width dimensions of the scrap steel unloading area are not significantly different), and so on. Of course, the division can also result in the scrap steel unloading area containing only one sub-area, for example, because each batch of scrap steel to be transferred grabbed by the steel-grabbing device occupies an average area of ​​the entire scrap steel unloading area, or because the area of ​​the scrap steel unloading area itself is too small to be divided into two or more sub-areas, etc. This disclosure does not impose any limitations on the method of dividing the scrap steel unloading area.

[0070] Figure 4 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown. Figure 4 As shown, step S210 of method 200, namely, identifying a sub-region and the corresponding photographing location of the sub-region within the scrap steel unloading area to which a batch of scrap steel is unloaded by the scrap steel grabbing device, may include steps S211 to S214.

[0071] Step S211: Acquire a video stream recording the process of the steel grabbing device transferring a batch of scrap steel to the scrap steel unloading area.

[0072] In the example, the video stream can be in a mainstream video file format to facilitate subsequent parsing and processing.

[0073] In another example, a video stream can also refer to a sequence of images acquired sequentially over time, where the images can be in mainstream image file formats to facilitate subsequent image processing. This disclosure does not impose any restrictions on the form of the video stream.

[0074] Step S212: Determine the motion trajectory of the steel-grabbing device based on the video stream.

[0075] In the example, the movement trajectory of the steel-grabbing device can be determined by identifying its position coordinates from the video stream.

[0076] As an example and not a limitation, the image acquisition device can be a PTZ camera system. Thus, for example, the motion trajectory of the steel-grabbing device can be determined (e.g., calculated) from video frames or images captured using the global field of view of the PTZ camera in the PTZ camera system via deep learning algorithms.

[0077] In the example, the movement trajectory of the steel grabbing device may include the transfer trajectory of the steel grabbing device grabbing a batch of scrap steel from the scrap steel transport party (e.g., the deck of a cargo ship loaded with scrap steel docked at the pier) and transferring and unloading the batch of scrap steel to the scrap steel unloading area at the scrap steel recycling party, as well as the movement trajectory of the steel grabbing device leaving the scrap steel unloading area and returning to the scrap steel transport party after unloading a batch of scrap steel.

[0078] Step S213: Based on the movement trajectory of the steel grabbing device, determine a sub-region within the scrap steel unloading area to which the batch of scrap steel is unloaded by the steel grabbing device.

[0079] In the example, the sub-region where the scrap steel is unloaded can be identified by the singularity (e.g., a turning point or a point where the scrap steel stays for a longer period of time) marked by the movement trajectory of the steel grabbing device.

[0080] Step S214: Use deep learning technology to determine the location where the steel grabbing device unloads the batch of scrap steel, and use it as the photo taking point.

[0081] It should be noted that deep learning technology itself is known to those skilled in the art, and any suitable type of deep learning technology can be used to determine the location of the image. This disclosure does not impose any restrictions on this and will not elaborate further.

[0082] As mentioned earlier, the imaging point refers to, for example, the focal point of the lens of the image acquisition device within the image to be captured, or the clearest convergence point within the image. Therefore, by utilizing deep learning technology to determine the imaging point, the captured image can better focus on the distribution center of the batch of scrap steel while covering the sub-region where the batch is located, avoiding unfavorable situations such as the scrap steel being out of focus or focusing on the edge parts of the batch (e.g., scrap steel portions with smaller mass or volume).

[0083] Figure 5 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown. Figure 5 As shown, step S220 of method 200, namely, in response to determining that the steel grabbing device leaves the scrap steel unloading area, acquiring three-dimensional point cloud data about the sub-region may include steps S221 to S222.

[0084] Step S221: Based on at least one of the video stream and the motion trajectory of the steel grabbing device, determine that the steel grabbing device has left the scrap unloading area.

[0085] In the example, the departure of the steel grabbing device from the scrap unloading area can be determined based on the video stream obtained in step S211.

[0086] In another example, the departure of the steel grabbing device from the scrap unloading area can be determined based on the movement trajectory of the steel grabbing device as determined in step S212.

[0087] In yet another example, the departure of the steel grabbing device from the scrap unloading area can be determined based on both the video stream and the movement trajectory of the steel grabbing device.

[0088] Step S222: Initiate scanning of the sub-region using a 3D scanning device to obtain 3D point cloud data of the sub-region.

[0089] As previously stated, scanning a sub-region using a three-dimensional scanning device may include scanning only the corresponding sub-region or scanning the entire scrap metal stack including the corresponding sub-region, and this disclosure does not impose any limitations in this regard.

[0090] Therefore, acquiring three-dimensional point cloud data of the sub-region where the scrap steel is located after the steel grabbing device leaves the scrap steel unloading area can avoid the acquisition of three-dimensional point cloud data containing three-dimensional point cloud information about the steel grabbing device, thereby effectively reducing or even eliminating noise in the three-dimensional point cloud data of the sub-region and improving the accuracy of the acquired three-dimensional point cloud data.

[0091] Figure 6 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown. Figure 6As shown, step S230 of method 200, namely, determining the magnification of the sub-region to be photographed based on the three-dimensional point cloud data, may include steps S231 to S234.

[0092] Step S231: Use 3D point cloud data to reconstruct the surface of a batch of scrap steel in a sub-region to obtain the surface mesh of the batch of scrap steel.

[0093] It should be noted that the use of three-dimensional point cloud data to reconstruct the surface of an object is known to those skilled in the art, and any suitable type of three-dimensional point cloud processing method can be used to determine the surface mesh of a batch of scrap steel. This disclosure does not impose any restrictions on this and will not elaborate further here.

[0094] In the example, the average height of the top surface of the batch of scrap steel can be calculated based on the surface mesh obtained for that batch.

[0095] Step S232: Based on the surface mesh and ground point cloud data included in the three-dimensional point cloud data of the batch of scrap steel, determine the height of the top surface of the batch of scrap steel from the ground of the sub-region.

[0096] As used in this disclosure, the term "ground" may refer to any suitable given reference plane, and is not limited to the ground plane.

[0097] As an example rather than a limitation, the ground of a sub-area can refer to the bottom surface of the scrap unloading area. For example, in the truck bed, the ground can refer to the inner or outer bottom surface of the truck bed.

[0098] It is understandable that, given a sub-region of ground, the distance between the detection port (such as a lens, X-ray transceiver port, etc.) of electronic devices such as image acquisition devices and / or 3D scanning devices and the ground can be measurable.

[0099] It can also be understood that the distance between the detection port of electronic equipment such as image acquisition devices and / or 3D scanning devices and the ground is generally greater than the distance between the top surface of any batch of scrap steel and the ground (i.e., the height of the top surface of the batch of scrap steel from the ground of the sub-region). This is because the detection port of the electronic equipment needs to be able to acquire images and / or perform X-ray scanning of the sub-region from a top-down or near-top-down perspective.

[0100] In the example, the average height of the ground in a sub-region can be calculated based on the ground point cloud data included in the 3D point cloud data.

[0101] Step S233: Based on the position parameters of the image acquisition device for taking pictures of the sub-region relative to the scrap steel unloading area and the height of the top surface of the batch of scrap steel from the ground of the sub-region, determine the distance between the image acquisition device and the shooting point.

[0102] In the example, the positional parameters of the image acquisition device relative to the scrap unloading area may include the distance H between the lens of the image acquisition device and the ground of the sub-area (e.g., the bottom surface of the scrap unloading area) (i.e., the height of the image acquisition device from the ground of the sub-area), and the distance d from the shooting point to the perpendicular line drawn between the image acquisition device and the ground.

[0103] As an example, and not a limitation, let h be the height of the top surface of a given batch of scrap steel from the ground of the sub-region containing that batch of scrap steel. Then the distance s between the image acquisition device and the shooting location can be determined as follows:

[0104] Step S234: Determine the magnification factor for taking pictures of the sub-region, based at least on the distance between the image acquisition device and the shooting position.

[0105] In the example, the magnification for photographing a sub-region can be determined by varying the distance between the image acquisition device and the photographing point.

[0106] As mentioned earlier, the greater the distance between the scrap steel unloading location and a fixed point such as an image acquisition device, the greater the magnification required to photograph the corresponding sub-region. Therefore, a function can be constructed, for example, to show that the magnification increases with the distance between the image acquisition device and the photographing point, so as to select a suitable magnification for photographing the corresponding sub-region.

[0107] Figure 7 An exemplary flowchart illustrating additional steps of a method for acquiring images of scrap steel according to embodiments of the present disclosure is shown. Figure 7 As shown, step S234, that is, determining the magnification of the sub-region to be photographed based at least on the distance between the image acquisition device and the photographing point, may include steps S2341 to S2343.

[0108] Step S2341: Determine the reference photographing location within the scrap steel unloading area and the reference distance between the image acquisition device and the reference photographing location.

[0109] In the example, the reference photographing point could be a point within the scrap unloading area (e.g., the bed of a transfer truck) where no batches of transferred scrap are loaded. In other words, the reference photographing point can be determined before each batch of scrap is transferred.

[0110] As an example, and not a limitation, in the case where the inner surface of the truck bed's bottom is considered the ground, the reference imaging point can be a point on the inner surface of the truck bed's bottom. Let H0 be the distance between the lens of the image acquisition device and the ground, and d0 be the distance from the reference imaging point to the perpendicular line drawn from the image acquisition device to the ground. Then, the reference distance s0 between the image acquisition device and the reference imaging point can be determined as s0 = sqrt(H0) 2 +d0 2 ).

[0111] It is understood that the reference photographic sites may be selected by those skilled in the art as needed, and this disclosure does not impose any restrictions in this regard.

[0112] Step S2342: By focusing the image acquisition device on the reference shooting position and adjusting the magnification of the image acquisition device, the image acquisition device is able to capture only the entire area of ​​the scrap steel unloading area, thus obtaining the reference magnification of the image acquisition device.

[0113] It is understandable that the image acquisition device is able to capture only the entire area of ​​the scrap unloading area. In some cases, this can be equivalent to the boundary of the image captured by the image acquisition device being substantially coincident with the perimeter of the scrap unloading area (e.g., the edge of the truck bed), thus obtaining the reference magnification k0.

[0114] Step S2343: Based on the distance between the image acquisition device and the shooting location, the reference distance, and the reference magnification, determine the magnification of the sub-region to be photographed.

[0115] As an example, and not a limitation, the magnification factor k for photographing a sub-region can be determined as follows:

[0116]

[0117] It is understood that although the determination of the magnification factor k for photographing a sub-region is described above in the form of a linear function, those skilled in the art can choose any suitable function as needed to determine the magnification factor for photographing different sub-regions by the image acquisition device, and this disclosure does not impose any restrictions in this regard.

[0118] In some embodiments, initiating the acquisition of images of a batch of scrap steel based on the shooting location and magnification may include: focusing the image acquisition device on the shooting location and adjusting the magnification of the image acquisition device to that magnification, taking pictures of the sub-region to obtain images of the batch of scrap steel.

[0119] Therefore, the magnification of the shooting can be automatically adjusted for the corresponding shooting area to ensure that the size of the scrap steel images of each batch is relatively uniform, thereby improving the accuracy of subsequent scrap steel grading.

[0120] Figure 8 An illustration of an exemplary scrap steel image acquisition scenario according to an embodiment of the present disclosure is shown.

[0121] It should be noted that, although Figure 8 The illustration depicts a scrap steel image acquisition scenario using a port crane unloading operation. However, those skilled in the art will understand that the scrap steel image acquisition method of this disclosure can be applied to any other suitable scrap steel transfer scenario, and this disclosure does not impose any limitations on it.

[0122] In such Figure 8 The scrap steel image acquisition scene shown includes a scrap steel transfer device 1, a pole support device 2, an image acquisition device 3, a terminal card reader 4, an LED display device 5, an image processing device 6, a ship cabin 7, a transfer car 8, a PTZ camera 9, a supplementary light 10, a port crane base 11, and a steel grabber 12.

[0123] As shown in the figure, the scrap steel transfer device 1 can be deployed on the dockside to grab (or suck up) scrap steel from the ship's hold 7 and transfer it to the transfer truck 8. The pole support device 2 can be deployed next to the parking space so that the camera is positioned as directly as possible facing the scrap steel transfer vehicle. The image acquisition device 3 (including a PTZ camera 9, supplementary lighting 10, etc.) and the 3D scanning device 13 can be mounted on top of the pole support device 2. Furthermore, the terminal card reader 4 can be deployed at the bottom of the pole support device 2, and the LED display device 5 can be located in the middle of the pole support device 2. The image processing device 6 can be remotely deployed in a server room and connected to the image acquisition device 3, the terminal card reader 4, the LED display device 5, and / or the 3D scanning device 13 via communication lines (e.g., wired communication lines or wireless communication channels or combinations thereof).

[0124] As shown in the figure, the pole support device 2 can be inverted "L" shape, with mounting brackets for the integrated camera 9 and supplementary lighting 10 in the image acquisition device 3 on the top crossbeam. The pole support device 2 can adopt a hollow design so that all communication lines and power supply lines can be routed and connected from inside the support, thereby avoiding exposed lines in complex environments and improving safety and durability.

[0125] As further shown in the figure, the scrap steel transfer device 1 can be composed of a port crane base 11, a scrap steel grabber 12, etc. The operator controls the interval of each batch of scrap steel transfer by the scrap steel grabber 12 according to the instructions displayed on the LED display device 5, so as to grab the scrap steel in the ship hold 7 in sequence and unload it into the transfer car 8.

[0126] It is understandable that the bullet camera in the integrated camera system 9 can be responsible for the overall field of view of the unloading area of ​​the port crane, while the PTZ camera can be responsible for taking pictures of the scrap steel inside the transfer carriage 8. The supplementary light 10 can provide lighting for nighttime operations and can also reduce the impact of the shadows cast by the oblique sunlight in the transfer carriage 8 on the imaging of the integrated camera system 9 during the day, thereby ensuring the image quality of the scrap steel.

[0127] As further shown in the figure, the terminal card reader 4 can be installed at the bottom of the pole support device 2. It can be connected to other components through the communication lines inside the pole support, so that the operator (e.g., the driver of the transfer vehicle) can initiate the intelligent collection and automatic rating process of scrap steel images.

[0128] Optionally, the LED display device 5 can be located in the middle of the pole support device 2. It can be connected to other components through the communication lines inside the pole support to display character information in multiple colors (e.g., displaying prompts for the operations that the operator of the scrap steel transfer device 1 needs to perform). Different information can be classified and displayed in different colors.

[0129] Image processing device 6, as the core processing unit for intelligent acquisition and automatic rating of scrap steel images, may include an image processing server, which interacts with and controls various components at the dock site through communication lines.

[0130] For example, the image processing device 6 can automatically initiate relevant task processes by receiving logistics code information, intelligent scrap steel image acquisition, and automatic rating process start commands transmitted by the terminal card reader 4. After all task processes are completed, the overall rating result of the transferred scrap steel on the vehicle can be displayed on the interface of the terminal card reader 4 for the operator to view.

[0131] Optionally, the interactive information transmitted by the terminal card reader 4, such as logistics code information, intelligent collection of scrap steel images, and start instructions for the automatic rating process, can be replaced by a handheld terminal and supporting applications.

[0132] Optionally, if the factory's existing MES or ERP system interface can directly obtain the required logistics code information, intelligent scrap steel image acquisition, and automatic rating process start instructions, the terminal card reader 4 may not be required.

[0133] Optionally, before transferring each batch of scrap steel to the transfer compartment 8, an image recognition algorithm can be used to determine whether the parking position of the transfer vehicle's compartment 8 is within the optimal shooting range of the image acquisition device 3. If not, the parking of the compartment 8 can be guided. For example, if the vehicle's parking position does not meet the requirements, a prompt message can be generated and automatically fed back to the on-site LED display device 5 to prompt the operator (e.g., the driver) that the vehicle's parking position does not meet the requirements and needs to be readjusted and reversed into the parking space until the vehicle is determined to be parked near the optimal position.

[0134] Optionally, the photographs of each batch of scrap steel collected can be transmitted back to the image processing device 6 for storage.

[0135] Accordingly, the computer vision algorithm in the image processing device 6 can identify multiple feature dimensions of the acquired scrap steel image, such as thickness, material type, size, diameter, density fitting, non-steel impurities, and hazardous materials in the seals, and comprehensively determine the evaluation level of the entire vehicle of transferred scrap steel. Additionally, after the image processing device 6 detects the presence of hazardous materials in the seals in the scrap steel image, warning character information can be transmitted to the LED display device 5 via a communication line to promptly remind the operator of the scrap steel transfer device 1 to take appropriate action.

[0136] Figure 9 A structural block diagram of an apparatus for acquiring images of scrap steel according to an embodiment of the present disclosure is shown.

[0137] like Figure 9 As shown, according to an embodiment of this disclosure, an apparatus 900 for acquiring images of scrap steel is provided, comprising a first module 910 for identifying a sub-region within a scrap steel unloading area to which a batch of scrap steel is unloaded by a steel grabbing device and a corresponding photographing point within the sub-region, wherein the photographing point indicates the location within the sub-region where the steel grabbing device unloads the batch of scrap steel; a second module 920 for acquiring three-dimensional point cloud data about the sub-region in response to determining that the steel grabbing device has left the scrap steel unloading area; a third module 930 for determining the magnification of the sub-region to be photographed based on the three-dimensional point cloud data; and a fourth module 940 for initiating the acquisition of images of the batch of scrap steel based on the photographing point and the magnification.

[0138] Here, the operation of the modules 910-940 of the device 900 is similar to the operation of steps 210-240 of the method 200 described above, and will not be repeated here. In some embodiments, the device 900 may also include one or more additional modules or units to perform functions corresponding to the additional steps of the method 200 (e.g., steps 201-202, steps 211-214, steps 221-222, steps 231-234, steps 2341-2343, etc.). These modules or units are not shown for the purpose of illustrative simplicity. Figure 7 middle.

[0139] In related technologies, machine vision-based scrap steel grading methods have many shortcomings, such as significant overlap in the captured areas and inconsistent image sizes, all of which affect the accuracy of scrap steel grading for the entire ship or truckload. With the help of Device 900, not only can the location where the steel-grabbing device unloads a batch of scrap steel be accurately positioned, ensuring comprehensive photography of each batch while avoiding overlapping areas, but the device can also automatically adjust the magnification of the images for each captured area, ensuring relatively uniform image size across batches and thus improving the accuracy of subsequent scrap steel grading.

[0140] In some embodiments, the device 900 may further include a fifth module for acquiring an initial image of the scrap steel unloading area before the steel grabbing device transfers each batch of scrap steel to the scrap steel unloading area; and for dividing the scrap steel unloading area based on the initial image to obtain multiple sub-areas of the scrap steel unloading area.

[0141] In some embodiments, the first module 910 of the device 900 may include: a first unit for acquiring a video stream recording the process of a steel grabbing device transferring a batch of scrap steel to a scrap steel unloading area; a second unit for determining the motion trajectory of the steel grabbing device based on the video stream; a third unit for determining a sub-area within the scrap steel unloading area to which the batch of scrap steel is unloaded by the steel grabbing device based on the motion trajectory of the steel grabbing device; and a fourth unit for determining the location where the steel grabbing device unloads the batch of scrap steel using deep learning technology, as a photographing point.

[0142] In some embodiments, the second module 920 of the device 900 may include: a fifth unit for determining, based on at least one of a video stream and the motion trajectory of the steel grabbing device, that the steel grabbing device has left the scrap steel unloading area; and a sixth unit for initiating a scanning of a sub-region using a three-dimensional scanning device to obtain three-dimensional point cloud data of the sub-region.

[0143] In some embodiments, the third module 930 of the device 900 may include: a seventh unit for reconstructing the surface of a batch of scrap steel in a sub-region using three-dimensional point cloud data to obtain a surface mesh of the batch of scrap steel; an eighth unit for determining the height of the top surface of the batch of scrap steel from the ground of the sub-region based on the surface mesh of the batch of scrap steel and the ground point cloud data included in the three-dimensional point cloud data; a ninth unit for determining the distance between the image acquisition device and the photography point based on the position parameters of the image acquisition device to be photographed relative to the scrap steel unloading area and the height of the top surface of the batch of scrap steel from the ground of the sub-region; and a tenth unit for determining the magnification of the sub-region to be photographed based at least on the distance between the image acquisition device and the photography point.

[0144] In some embodiments, the tenth unit may be further configured to: determine a reference photographing location within the scrap steel unloading area and a reference distance between the image acquisition device and the reference photographing location; obtain a reference magnification of the image acquisition device by focusing the image acquisition device on the reference photographing location and adjusting the magnification of the image acquisition device so that the image acquisition device can capture only the entire area of ​​the scrap steel unloading area; and determine the magnification of the sub-area to be photographed based on the distance between the image acquisition device and the photographing location, the reference distance, and the reference magnification.

[0145] In some embodiments, the fourth module 940 of the device 900 may be further configured to: focus the image acquisition device on the image acquisition point and adjust the magnification of the image acquisition device to that magnification to take pictures of the sub-region in order to obtain an image of the batch of scrap steel.

[0146] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific actions performed by the modules discussed herein include the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action can include the specific module performing the action itself and / or another module that performs the action, called or otherwise accessed by the specific module.

[0147] It should also be understood that this article can describe various technologies in the general context of software and hardware components or program modules. The above regarding... Figure 9The described modules can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. Hardware logic / circuit may include integrated circuit chips (which include processors (e.g., Central Processing Unit (CPU), microcontrollers, microprocessors, digital signal processors (DSPs), etc.), memory, one or more communication interfaces, and / or one or more components of other circuitry), and may optionally execute received program code and / or include embedded firmware to perform functions.

[0148] Figure 10 A structural block diagram of a scrap steel image acquisition system according to an embodiment of the present disclosure is shown. Figure 10 As shown, the scrap steel image acquisition system 1000 may include an image acquisition device 1010 for acquiring an image including at least a portion of the scrap steel unloading area; a three-dimensional scanning device 1020 for scanning at least a portion of the scrap steel unloading area to obtain three-dimensional point cloud data of the corresponding area; and a processing device 1030 operatively connected to the image acquisition device 1010 and the three-dimensional scanning device 1020 to perform the method for acquiring scrap steel images disclosed herein, such as method 200 and additional steps of its various embodiments.

[0149] It is understood that the image acquisition device 1010 may include various suitable types of cameras, video cameras, etc., and this disclosure does not impose any limitations on it. The 3D scanning device 1020 may include suitable types of commercially available 3D scanning equipment such as radar and lidar, and this disclosure also does not impose any limitations on it. The processing device 1030 may be any custom or commercially available processor, such as a central processing unit (CPU), a distributed processing unit, a semiconductor-based microprocessor (e.g., in the form of a microchip or chipset), etc., and this disclosure does not impose any limitations on it.

[0150] According to embodiments of this disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0151] According to embodiments of the present disclosure, an electronic device is provided. The electronic device may include: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the present disclosure for acquiring images of scrap steel.

[0152] According to embodiments of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. These computer instructions can be used to cause a computer to execute the method of this disclosure for acquiring images of scrap steel.

[0153] According to embodiments of this disclosure, a computer program product is provided. This computer program product may include a computer program that, when executed by a processor, implements the method of this disclosure for acquiring images of scrap steel.

[0154] It is understood that the scrap steel image acquisition scheme proposed in this disclosure has advantages such as versatility, automation, and no human intervention. Using this scrap steel image acquisition scheme, the unloading location can be accurately located, ensuring comprehensive coverage while effectively reducing the probability of repeatedly photographing the same area. This significantly improves the image coverage rate and reduces the repetition rate of photographed areas, enabling more accurate rating of the entire truckload of scrap steel. Furthermore, the scrap steel image acquisition scheme of this disclosure can adaptively adjust the magnification of the photographs, ensuring the stability of the size of each captured scrap steel image and improving recognition accuracy.

[0155] refer to Figure 11 The present invention describes a structural block diagram of an electronic device 1100 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0156] like Figure 11 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. The RAM 1103 may also store various programs and data required for the operation of the electronic device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0157] Multiple components in electronic device 1100 are connected to I / O interface 1105, including: input unit 1106, output unit 1107, storage unit 1108, and communication unit 1109. Input unit 1106 can be any type of device capable of inputting information to electronic device 1100. Input unit 1106 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 1107 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1108 may include, but is not limited to, a hard disk and an optical disk. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and / or chipsets, such as Bluetooth devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices and / or the like.

[0158] The computing unit 1101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning network algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as method 200 and its embodiments. For example, in some embodiments, method 200 and its embodiments may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of method 200 described above and additional steps of its embodiments may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform additional steps of method 200 and its various embodiments by any other suitable means (e.g., by means of firmware).

[0159] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0160] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0161] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0162] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0163] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0164] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0165] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0166] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A method for acquiring images of scrap steel, comprising: During the process of the steel grabbing device transferring a batch of scrap steel to the scrap steel unloading area each time, A sub-area within the scrap unloading area to which the batch of scrap steel is unloaded by the steel grabbing device is identified, and a corresponding photographic location within the sub-area is defined as the position within the sub-area where the steel grabbing device unloads the batch of scrap steel. In response to determining that the steel grabbing device has left the scrap steel unloading area, three-dimensional point cloud data about the sub-region is acquired; The magnification factor for photographing the sub-region is determined based on the three-dimensional point cloud data. as well as Initiate the acquisition of images of the batch of scrap steel based on the stated photographic location and the stated magnification. The determination of the magnification factor for photographing the sub-region based on the 3D point cloud data includes: The surface of the batch of scrap steel in the sub-region is reconstructed using the three-dimensional point cloud data to obtain the surface mesh of the batch of scrap steel; Based on the surface mesh of the batch of scrap steel and the ground point cloud data included in the three-dimensional point cloud data, the height of the top surface of the batch of scrap steel from the ground of the sub-region is determined. Based on the position parameters of the image acquisition device that is to take pictures of the sub-region relative to the scrap steel unloading area and the height of the top surface of the batch of scrap steel from the ground of the sub-region, the distance between the image acquisition device and the picture location is determined; The magnification factor for photographing the sub-region is determined based at least on the distance between the image acquisition device and the photographing location. The positional parameters include the distance between the lens of the image acquisition device and the ground of the sub-region, and the distance from the shooting point to the perpendicular line drawn between the image acquisition device and the ground.

2. The method according to claim 1, wherein, Before the steel grabbing device transfers each batch of scrap steel to the scrap steel unloading area, the method further includes: Acquire an initial image including the scrap steel unloading area; The scrap steel unloading area is divided based on the initial image to obtain multiple sub-regions of the scrap steel unloading area.

3. The method according to claim 1 or 2, wherein, The identification of a sub-area within the scrap unloading area to which the batch of scrap steel is unloaded by the steel grabbing device, and the corresponding photographic location of the sub-area, includes: Acquire a video stream recording the process of the steel grabbing device transferring the batch of scrap steel to the scrap steel unloading area; The motion trajectory of the steel-grabbing device is determined based on the video stream; Based on the movement trajectory of the steel grabbing device, the sub-region within the scrap steel unloading area to which the batch of scrap steel is unloaded by the steel grabbing device is determined; The location where the steel-grabbing device unloads the batch of scrap steel is determined using deep learning technology, and this location is used as the photographing point.

4. The method according to claim 3, wherein, In response to determining that the steel grabbing device has left the scrap unloading area, acquiring three-dimensional point cloud data about the sub-region includes: Based on at least one of the video stream and the motion trajectory of the steel grabbing device, it is determined that the steel grabbing device has left the scrap steel unloading area; The sub-region is scanned using a 3D scanning device to obtain 3D point cloud data of the sub-region.

5. The method according to claim 1, wherein, Determining the magnification factor for photographing the sub-region, based at least on the distance between the image acquisition device and the photographing location, includes: Determine the reference photographing location within the scrap steel unloading area and the reference distance between the image acquisition device and the reference photographing location; By focusing the image acquisition device on the reference shooting position and adjusting the magnification of the image acquisition device, the image acquisition device is able to capture only the entire area of ​​the scrap steel unloading area, thus obtaining the reference magnification of the image acquisition device. Based on the distance between the image acquisition device and the shooting location, the reference distance, and the reference magnification, the magnification to be used to photograph the sub-region is determined.

6. The method according to claim 1 or 2, wherein, Initiating the acquisition of images of the batch of scrap steel based on the photographing location and the magnification includes: The image acquisition device is focused on the photographing point and its magnification is adjusted to the specified magnification to photograph the sub-region, thereby obtaining an image of the batch of scrap steel.

7. An apparatus for acquiring images of scrap steel, comprising: The first module is used to identify a sub-area of ​​the scrap steel unloading area to which the batch of scrap steel is unloaded by the steel grabbing device and the corresponding photographic location of the sub-area during the process of each time the steel grabbing device transfers a batch of scrap steel to the scrap steel unloading area. The photographic location indicates the position of the steel grabbing device unloading the batch of scrap steel in the sub-area. The second module is used to acquire three-dimensional point cloud data about the sub-region in response to determining that the steel grabbing device has left the scrap steel unloading area. The third module is used to determine the magnification factor for taking pictures of the sub-region based on the three-dimensional point cloud data; as well as The fourth module is used to initiate the acquisition of images of the batch of scrap steel based on the photographing location and the magnification. The determination of the magnification factor for photographing the sub-region based on the 3D point cloud data includes: The surface of the batch of scrap steel in the sub-region is reconstructed using the three-dimensional point cloud data to obtain the surface mesh of the batch of scrap steel; Based on the surface mesh of the batch of scrap steel and the ground point cloud data included in the three-dimensional point cloud data, the height of the top surface of the batch of scrap steel from the ground of the sub-region is determined. Based on the position parameters of the image acquisition device that is to take pictures of the sub-region relative to the scrap steel unloading area and the height of the top surface of the batch of scrap steel from the ground of the sub-region, the distance between the image acquisition device and the picture location is determined; The magnification factor for photographing the sub-region is determined based at least on the distance between the image acquisition device and the photographing location. The positional parameters include the distance between the lens of the image acquisition device and the ground of the sub-region, and the distance from the shooting point to the perpendicular line drawn between the image acquisition device and the ground.

8. A scrap steel image acquisition system, comprising: An image acquisition device for acquiring images of at least a portion of the scrap steel unloading area; A three-dimensional scanning device is used to scan at least a portion of the scrap steel unloading area to obtain three-dimensional point cloud data of the corresponding area; as well as A processing device, operatively connected to the image acquisition device and the three-dimensional scanning device, is used to perform the method according to any one of claims 1 to 6.

9. An electronic device, comprising: At least one processor; as well as At least one memory communicatively connected to the at least one processor, the at least one memory storing instructions that, when executed individually or jointly by the at least one processor, cause the at least one processor to perform the method of any one of claims 1-6.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, When the computer instructions are executed individually or jointly by one or more processors of the computer, the computer performs the method according to any one of claims 1-6.

11. A computer program product comprising computer instructions that, when executed individually or jointly by one or more processors of a computer, cause the computer to perform the method according to any one of claims 1-6.

Images