A method and device for inventorying scrap steel yard

Abstract

The present invention proposes a scrap steel yard inventory method and device, comprising: obtaining a three-dimensional point cloud image and a visible light image of the scrap steel yard; constructing a semantic segmentation network, inputting the three-dimensional point cloud image into the semantic segmentation network, obtaining a three-dimensional point cloud map of all material piles in the scrap steel yard, and obtaining the volume of each material pile; obtaining the waste category of each material pile in the scrap yard through the visible light image, obtaining the mass of each waste pile according to the waste category and volume of each waste pile, and then obtaining the total mass of the waste in the scrap steel yard; the present invention can effectively improve inventory accuracy, reduce manual participation, save labor costs, and improve processing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of steel production and manufacturing, and in particular to a method and device for storing scrap steel in a warehouse. Background Art

[0002] As a recyclable resource, scrap steel plays a vital role in the steel industry. With the large-scale recycling of scrap steel, the storage and turnover volume in scrap steel warehouses is increasing. To improve scrap steel turnover efficiency, a scrap steel inventory method is needed to accurately calculate the current scrap steel storage volume. Currently, scrap steel inventory methods mainly rely on manual estimation of the current scrap steel stockpile volume based on historical experience, resulting in low accuracy and inconsistent results among different workers. Summary of the Invention

[0003] In view of the above problems existing in the prior art, the present invention proposes a scrap steel yard inventory method and device, which mainly solves the problem that the existing inventory relies on manual labor, resulting in insufficient accuracy and consistency.

[0004] In order to achieve the above-mentioned and other purposes, the technical solutions adopted by the present invention are as follows.

[0005] A method for storing scrap steel in a warehouse, comprising:

[0006] Obtain 3D point cloud images and visible light images of the scrap steel yard;

[0007] Constructing a semantic segmentation network, inputting the three-dimensional point cloud image into the semantic segmentation network, obtaining a three-dimensional point cloud image of all stockpiles in the scrap steel yard, and obtaining the volume of each stockpile;

[0008] The waste type of each pile in the scrap yard is obtained through the visible light image, and the mass of each scrap pile is obtained according to the waste type and volume of each scrap pile, thereby obtaining the total mass of the scrap in the scrap steel yard.

[0009] Optionally, three-dimensional point cloud data and visible light images of the scrap steel yard are obtained, including:

[0010] Constructing a data acquisition platform based on a drone, the data acquisition platform comprising: a drone, a point cloud data acquisition device, a visible light image acquisition device, and a positioning device; wherein the point cloud data acquisition device, the visible light image acquisition device, and the positioning device are all arranged on the drone;

[0011] The drone travels along a designated route to acquire point cloud data and visible light images covering the entire scrap steel yard;

[0012] A three-dimensional point cloud image of the scrap steel yard is calculated based on the point cloud data.

[0013] Optionally, calculating a three-dimensional point cloud image of the scrap steel yard based on the point cloud data includes:

[0014] Obtaining the position and attitude of the drone through the positioning device;

[0015] According to the position and posture combined with the point cloud data, a three-dimensional point cloud image of the scrap steel yard is acquired using a three-dimensional reconstruction algorithm.

[0016] Optionally, the three-dimensional reconstruction algorithm includes a SLAM algorithm and a three-dimensional reconstruction algorithm.

[0017] Optionally, the semantic segmentation network includes: a PointNet+ network combined with a deep learning three-dimensional point cloud semantic segmentation network.

[0018] Optionally, the point cloud data acquisition device includes a laser radar.

[0019] Optionally, the positioning device includes an inertial navigation module consisting of two GPS antennas, which is used to measure the position and attitude of the UAV.

[0020] Optionally, obtaining the waste type of each waste pile in the waste yard through the visible light image, and obtaining the mass of each waste pile according to the waste type and volume of each waste pile, includes:

[0021] Building a recognition network to obtain the waste type of each pile in the visible light image;

[0022] Constructing mass-volume calculation models for each waste category;

[0023] The mass of each pile of waste is obtained by matching the corresponding mass-volume calculation model with the waste type corresponding to each pile of waste in the visible light image.

[0024] A scrap steel storage device, comprising:

[0025] An image acquisition module, used to acquire a three-dimensional point cloud image and a visible light image of the scrap steel yard;

[0026] A stockpile segmentation module is configured to construct a semantic segmentation network, input the three-dimensional point cloud image into the semantic segmentation network, obtain a three-dimensional point cloud image of all stockpiles in the scrap steel yard, and obtain the volume of each stockpile;

[0027] The disk inventory calculation module is used to obtain the scrap type of each scrap pile in the scrap yard through the visible light image, obtain the mass of each scrap pile according to the scrap type and volume of each scrap pile, and then obtain the total mass of the scrap in the scrap steel yard.

[0028] Optionally, the image acquisition module includes:

[0029] A collection platform construction unit, configured to construct a data collection platform based on a drone, the data collection platform comprising: a drone, a point cloud data collection device, a visible light image collection device, and a positioning device; wherein the point cloud data collection device, the visible light image collection device, and the positioning device are all provided on the drone;

[0030] Planning and collection unit, the UAV travels along a specified route to obtain point cloud data and visible light images covering the entire scrap steel yard;

[0031] An image reconstruction unit calculates a three-dimensional point cloud image of the scrap steel yard based on the point cloud data.

[0032] As described above, the scrap steel yard inventory method and device of the present invention have the following beneficial effects.

[0033] The system automatically segments the stockpile based on 3D point cloud images and calculates the stockpile mass based on the stockpile category combined with visible light images. This automates the inventory process of the entire scrap steel yard with high accuracy and consistency, does not rely on manual labor, and can greatly improve inventory efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The figure is a flow chart of a method for storing scrap steel in a scrap steel yard according to one embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0036] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0037] See also Figure 1 The present invention provides a method for storing scrap steel in a warehouse, which may include the following steps.

[0038] Step S1, obtaining a three-dimensional point cloud image and a visible light image of the scrap steel yard;

[0039] Step S2: constructing a semantic segmentation network, inputting the three-dimensional point cloud image into the semantic segmentation network, obtaining a three-dimensional point cloud image of all stockpiles in the scrap steel yard, and obtaining the volume of each stockpile;

[0040] Step S3: obtaining the waste type of each waste pile in the scrap yard through the visible light image, obtaining the mass of each waste pile according to the waste type and volume of each waste pile, and then obtaining the total mass of the waste in the scrap yard.

[0041] The specific steps of the scrap steel yard storage method of the present invention are described below with reference to specific embodiments.

[0042] In step S1, obtaining a three-dimensional point cloud image and a visible light image of the scrap steel yard may include the following steps:

[0043] Step S101: constructing a data acquisition platform based on a drone, the data acquisition platform comprising: a drone, a point cloud data acquisition device, a visible light image acquisition device, and a positioning device; wherein the point cloud data acquisition device, the visible light image acquisition device, and the positioning device are all provided on the drone;

[0044] Step S102: the drone travels along a designated route to acquire point cloud data and visible light images covering the entire scrap steel yard;

[0045] Step S103 : calculating a three-dimensional point cloud image of the scrap steel yard based on the point cloud data.

[0046] Specifically, the point cloud data collection device can be a laser radar. Installed on the bottom of the drone, the laser radar transmits a laser beam downward to scan the scrap yard during the mission. The laser radar compares the reflected signals from various objects in the scrap yard with the transmitted signals to determine parameters such as the height, orientation, and posture of each object in the scrap yard, generating point cloud data. For example, the point cloud data can be represented as a multidimensional coordinate matrix.

[0047] The visible light image acquisition device can be a camera, which is mounted on the bottom of a drone. When the drone flies above the scrap steel yard, the camera collects visible light images of the scrap steel yard at different locations at different times.

[0048] The positioning device may include two GPS antennas, which are arranged horizontally symmetrically about the central axis of the drone to form an inertial navigation module. The inertial navigation module can perform position calibration based on the position signals measured by the two GPS antennas and the positional relationship between the two GPS antennas to determine the drone's position and attitude. The relative position of the two GPS antennas can be adjusted according to actual application requirements and is not limited here.

[0049] In one embodiment, a drone's trajectory can be pre-set based on the scrap yard's layout to capture point cloud data covering the entire yard. For example, assuming the yard is rectangular, the drone's trajectory is calculated by starting from the upper left corner of the yard and moving in a straight line toward the lower left corner. After moving horizontally for a certain distance, the drone then moves upward in a straight line, forming a square wave-like trajectory covering the entire yard. The specific trajectory setting can be adjusted based on the actual site and application requirements and is not limited here.

[0050] In one embodiment, as the drone moves along a set trajectory, a 3D point cloud map of the entire scrap yard can be created and mapped based on the drone's pose and position, as well as point cloud data collected by a LiDAR. Specifically, a SLAM algorithm and a 3D reconstruction algorithm can be used to construct the 3D point cloud map. The specific process is conventional and will not be detailed here.

[0051] In step S2, a semantic segmentation network is constructed, and the three-dimensional point cloud image is input into the semantic segmentation network to obtain a three-dimensional point cloud image of all stockpiles in the scrap steel yard, and the volume of each stockpile is obtained.

[0052] In one embodiment, the semantic segmentation network can be a combination of a PointNet+ network and a deep learning three-dimensional point cloud semantic segmentation network. Specifically, the PointNet+ network has two local feature extraction subnetworks, each of which includes a sampling layer, a clustering layer, and a feature extraction layer, PointNet. Random sampling is performed through the sampling layer, specifically, the farthest point sampling method can be used for sampling. Furthermore, a circle with a radius r set with the sampling point as the origin is drawn through the clustering layer, and the point cloud within the circle is regarded as a cluster, thereby dividing multiple clusters. Finally, a convolution operation is performed through the feature extraction layer to extract features in the cluster.

[0053] The deep learning 3D point cloud semantic segmentation network includes a classification network and a segmentation network. The classification network includes multiple fully connected layers and a classifier. The output of the feature extraction layer can be input into the fully connected layer of the classification network. Local features are extracted through multiple fully connected layers, and finally global features are obtained. The classification result is obtained by the classifier. For example, the classifier can use a softmax classifier. The specific classification process is not repeated here. The segmentation network directly splices the features of each layer in the local feature extraction subnetwork with the features of the corresponding layer with the same number of points in the segmentation network, and upsamples the global features to obtain the segmentation result.

[0054] After semantic segmentation of the scrap yard's 3D point cloud image, a 3D point cloud image of each pile is generated. Based on this 3D point cloud image, the volume of the corresponding pile can be calculated. The detailed calculation process is not detailed here.

[0055] In step S3, the waste type of each waste pile in the scrap yard is obtained through the visible light image, and the mass of each waste pile is obtained according to the waste type and volume of each waste pile, thereby obtaining the total mass of the waste in the scrap steel yard.

[0056] In one embodiment, images of waste piles containing various types of waste can be pre-organized as sample images to construct a training sample set. The waste types in the sample images are labeled. This labeled sample set is then fed into a recognition network for model training, generating a recognition model. The recognition network can employ a convolutional neural network. The specific network training process is conventional and will not be detailed here.

[0057] The three-dimensional point cloud image of each material pile obtained in step S2 is input into the recognition model to obtain the waste material category corresponding to each material pile.

[0058] Furthermore, a mass-volume calculation model can be constructed for different waste categories. Specifically, a correspondence between waste category and waste density can be established based on experience, and a mass-volume calculation model can be constructed based on waste density. Optionally, fertilizer samples of the corresponding category can be collected to simulate waste piles for waste density correction to ensure the accuracy of the calculation model. The estimated mass of each pile is calculated based on density and volume. The mass of all piles is calculated to obtain the total mass of the scrap in the entire scrap yard. This completes the inventory of the scrap yard. In another example, the mass of multiple piles of the same category can be calculated, and inventory can be conducted based on the waste category corresponding to the pile.

[0059] In one embodiment, a scrap steel yard inventory system is provided for executing the scrap steel yard inventory method described in the aforementioned method embodiment. Because the technical principles of the system embodiment are similar to those of the aforementioned method embodiment, the same technical details will not be repeated here.

[0060] In one embodiment, a scrap steel yard inventory device includes: an image acquisition module for acquiring a three-dimensional point cloud image and a visible light image of the scrap steel yard; a stockpile segmentation module for constructing a semantic segmentation network, inputting the three-dimensional point cloud image into the semantic segmentation network, acquiring a three-dimensional point cloud image of all stockpiles in the scrap steel yard, and acquiring the volume of each stockpile; and an inventory calculation module for acquiring the waste category of each stockpile in the scrap steel yard through the visible light image, acquiring the mass of each waste pile according to the waste category and volume of each waste pile, and thereby acquiring the total mass of the waste in the scrap steel yard.

[0061] In one embodiment, the image acquisition module includes: an acquisition platform construction unit, which is used to construct a data acquisition platform based on a drone, and the data acquisition platform includes: a drone, a point cloud data acquisition device, a visible light image acquisition device and a positioning device; wherein the point cloud data acquisition device, the visible light image acquisition device and the positioning device are all arranged on the drone; a planning acquisition unit, in which the drone travels along a specified route to obtain point cloud data and visible light images covering the entire scrap steel yard; an image reconstruction unit, which calculates a three-dimensional point cloud image of the scrap steel yard based on the point cloud data.

[0062] In summary, the present invention provides a scrap steel yard inventory method and device, which utilizes a drone, 3D laser radar, and camera to collect three-dimensional point cloud images and visible light images of scrap steel piles from the air, while simultaneously acquiring the drone's position and posture in real time. A three-dimensional reconstruction map of the scrap steel yard is then established using a three-dimensional reconstruction algorithm. A pile identification algorithm is then used to segment the three-dimensional image of the pile from the three-dimensional reconstruction map of the scrap steel yard, thereby calculating the volume of each pile. The type of each scrap steel pile is then identified, and the mass of each scrap steel pile is calculated based on a historical empirical model of the volume-mass of each scrap steel pile type. Finally, the mass of all scrap steel in the entire scrap steel yard is calculated to complete the scrap steel inventory. Manual intervention is reduced, inventory is automatically completed to ensure data consistency and improve processing efficiency. The volume of the pile is calculated based on point cloud image identification, and the mass is then obtained, avoiding errors caused by relying on manual experience for estimation and improving inventory accuracy. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has high industrial utilization value.

[0063] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A method for storing scrap steel in a warehouse, characterized in that: include: Obtain 3D point cloud images and visible light images of the scrap steel yard; Acquiring three-dimensional point cloud data and visible light images of a scrap steel yard, including: building a data acquisition platform based on a drone, the data acquisition platform including: a drone, a point cloud data acquisition device, a visible light image acquisition device, and a positioning device; wherein the point cloud data acquisition device, the visible light image acquisition device, and the positioning device are all disposed on the drone; the drone travels along a designated route to acquire point cloud data and visible light images covering the entire scrap steel yard; and calculating a three-dimensional point cloud image of the scrap steel yard based on the point cloud data; Constructing a semantic segmentation network, inputting the three-dimensional point cloud image into the semantic segmentation network, obtaining a three-dimensional point cloud image of all stockpiles in the scrap steel yard, and obtaining the volume of each stockpile; The waste category of each stockpile in the scrap steel yard is obtained through the visible light image, and the mass of each scrap pile is obtained according to the waste category and volume of each scrap pile, thereby obtaining the total mass of the scrap in the scrap steel yard; the waste category of each stockpile in the scrap yard is obtained through the visible light image, and the mass of each scrap pile is obtained according to the waste category and volume of each scrap pile, including: constructing a recognition network to obtain the waste category of each stockpile in the visible light image; constructing a mass-volume calculation model for each waste category; matching the corresponding mass-volume calculation model according to the waste category corresponding to each stockpile in the visible light image to obtain the mass of the corresponding stockpile.

2. The scrap steel yard inventory method according to claim 1, characterized in that: Calculating a three-dimensional point cloud image of the scrap steel yard based on the point cloud data includes: Obtaining the position and attitude of the drone through the positioning device; According to the position and posture combined with the point cloud data, a three-dimensional point cloud image of the scrap steel yard is acquired using a three-dimensional reconstruction algorithm.

3. The scrap steel yard inventory method according to claim 2, characterized in that: The three-dimensional reconstruction algorithm includes a SLAM algorithm and a three-dimensional reconstruction algorithm.

4. The method for storing scrap steel in a warehouse according to claim 1, characterized in that: The semantic segmentation network includes: PointNet+ network combined with deep learning three-dimensional point cloud semantic segmentation network.

5. The scrap steel storage method according to claim 1, characterized in that: The point cloud data acquisition device includes a laser radar.

6. The scrap steel storage method according to claim 1, characterized in that: The positioning device includes an inertial navigation module composed of two GPS antennas, which is used to measure the position and attitude of the drone.

7. A scrap steel storage device, characterized in that: include: An image acquisition module, used to acquire a three-dimensional point cloud image and a visible light image of the scrap steel yard; The image acquisition module includes: an acquisition platform construction unit for constructing a data acquisition platform based on a UAV, wherein the data acquisition platform includes: a UAV, a point cloud data acquisition device, a visible light image acquisition device, and a positioning device; wherein the point cloud data acquisition device, the visible light image acquisition device, and the positioning device are all installed on the UAV; a planning acquisition unit, wherein the UAV travels along a specified route to acquire point cloud data and visible light images covering the entire scrap steel yard; and an image reconstruction unit, which calculates a three-dimensional point cloud image of the scrap steel yard based on the point cloud data. A stockpile segmentation module is configured to construct a semantic segmentation network, input the three-dimensional point cloud image into the semantic segmentation network, obtain a three-dimensional point cloud image of all stockpiles in the scrap steel yard, and obtain the volume of each stockpile; The inventory calculation module is used to obtain the scrap category of each stockpile in the scrap steel yard through the visible light image, obtain the mass of each scrap pile based on the scrap category and volume of each scrap pile, and then obtain the total mass of the scrap in the scrap steel yard; obtain the scrap category of each stockpile in the scrap yard through the visible light image, and obtain the mass of each scrap pile based on the scrap category and volume of each scrap pile, including: constructing a recognition network to obtain the scrap category of each stockpile in the visible light image; constructing a mass-volume calculation model for each scrap category; matching the corresponding mass-volume calculation model according to the scrap category corresponding to each stockpile in the visible light image to obtain the mass of the corresponding stockpile.

Images