Smelting stockpile warehousing method and system based on three-dimensional reconstruction

Abstract

The invention belongs to the technical field of storage informatization, and particularly discloses a smelting stockpile storage method and system based on three-dimensional reconstruction. The method comprises the following steps: a hanger rail robot moves along a rack rail to patrol a warehouse to obtain displacement data s, and a laser radar scanner scans stockpile in a moving process to obtain two-dimensional measurement data (x ', y'); the method comprises the following steps of: converting s and (x ', y') distribution into world coordinate systems x and (y, z) to obtain dynamic point cloud data (x, y, z), filtering the dynamic point cloud data (x, y, z), performing real-time point cloud registration on the filtered dynamic point cloud data, performing difference analysis on the filtered dynamic point cloud data and global static point cloud data before library inspection through a SegmentDifference algorithm, calculating physical change of the stockpile, and performing persistent operation; and intelligently and dynamically matching the physical change of the stockpile with the warehouse-in and warehouse-out information to form a warehouse-in and warehouse-out record. The system has the characteristics of high real-time performance, high data acquisition efficiency and reliable warehouse-in and warehouse-out records.

Description

Technical Field

[0001] The present invention relates to the field of storage information technology, and in particular to a smelting stockpile storage method and system based on three-dimensional reconstruction, which has strong real-time performance, high data acquisition efficiency and reliable storage records. Background Art

[0002] In the production process of copper and other nonferrous metal smelting enterprises, producers always expect a stable and reliable supply of raw materials, good quality and low price, stable and smooth production, production process indicators that meet requirements, and completion of production and quality tasks as planned to maximize corporate profits. However, for smelting enterprises, there are currently the following problems in raw material management: (1) Due to the limitations of storage environment, mine car production, irregular mineral accumulation, and high renewal frequency, it is difficult to accurately measure the volume of storage piles to control the entry and exit of the warehouse; (2) Due to the different grades of smelting raw materials, the ore raw materials used for smelting are easily confused when they are piled up. Moreover, when they are used again after being piled up for a long time, the records of the pile and its weight are prone to errors, which affects the smelting results. (3) The traditional manual recording method makes the incoming and outgoing quantities of raw materials inaccurate, which seriously affects production efficiency and increases labor management costs; (4) Although the existing technology uses modeling methods such as radar scanning, ground laser scanners and aerial digital photography to assist in the management of the entry and exit of piled materials, the traditional piled material 3D reconstruction method using radar scanning and ground laser scanner scanning has the problem of the piled materials being too high to block the scanning signal, resulting in missing scanning areas or reducing the effective working range. Aerial digital photography is limited by the dust interference in the material yard and the difficulty of accurate positioning, which limits its application in the management of raw materials in the material yard.

[0003] Faced with the above challenges, copper and other non-ferrous metal smelting companies are in urgent need of developing a set of methods and systems to improve the lean management of material yard stockpiling. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a smelting stockpile storage method based on three-dimensional reconstruction with strong real-time performance, high data acquisition efficiency and reliable entry and exit records, and also provides a smelting stockpile storage system based on three-dimensional reconstruction.

[0005] The smelting stockpile storage method based on three-dimensional reconstruction of the present invention is implemented as follows: it includes warehouse scanning, point cloud data formation, data preprocessing, difference analysis, and in-and-out warehouse matching steps, and the specific contents are as follows: A. Warehouse inspection and scanning: The hanging rail robot with a laser radar scanner fixed below is driven by a servo motor to move along the rack rail fixed above the material yard to inspect the warehouse and obtain the displacement data of the hanging rail robot along the rack rail.s ; At the same time, the laser radar scanner scans the smelting stockpile in the material yard during the movement of the crane rail robot, and obtains two-dimensional measurement data based on the laser radar scanner coordinate system ( x', y' ); B. Point cloud data formation: Displacement data s Convert the value into the world coordinate system x , and the measured data ( x', y' ) is converted into data in the world coordinate system ( y, z ), and finally form the dynamic point cloud data of smelting pile ( x, y, z ); C. Data preprocessing: Dynamic point cloud data ( x, y, z ) is filtered, and then the dynamic point cloud data after filtering is processed ( x, y, z ) to perform real-time point cloud registration; D. Difference analysis: The point cloud data obtained by real-time point cloud registration is analyzed by SegmentDifferences algorithm with the global static point cloud data before the warehouse inspection, and the real-time physical changes of the pile are calculated, and then persisted to files and databases; E. In-and-out warehouse matching: Intelligently and dynamically match the real-time physical changes of the piled materials with the in-and-out warehouse information to form a complete, effective and authentic in-and-out warehouse record.

[0006] Furthermore, in the step A, the rail-hanging robot moves along the rack rail to patrol the warehouse when the material yard is completed in storage and out of storage; in the step C, the dynamic point cloud data ( x, y, z ) is filtered, which is to set a range on the attribute of the point, filter the points, and retain the points within the range on the attribute; in the D step, the global static point cloud data is first filtered, and then a difference analysis is performed; wherein, the global static point cloud data is filtered, and a Statistical Outlier Removal filter is used to remove outliers in the global static point cloud data.

[0007] Furthermore, in the step C, the specific process of filtering the dynamic point cloud data is as follows: C10: specifies a dimension and the value range under this dimension; C20: traverse each point in the dynamic point cloud data to determine whether the value of the point under the specified dimension is within the value range. If not, the point is determined to be an outlier and deleted from the dynamic point cloud data; C30: The traversal is completed, and the remaining points that have not been deleted are the dynamic point cloud data points after filtering.

[0008] Furthermore, in the step D, the specific process of filtering the global static point cloud data is as follows: D10: For each point in the global static point cloud data, calculate the average distance d from the point to all points in the K neighborhood of the point, and obtain an array containing the d values ​​of each point, recorded as distance; D20: Assume that each element in the distance array forms a Gaussian distribution, and the array is a sample; traverse all points in the global static point cloud data. If the d value of the point is outside the standard range defined by the mean and variance of the aforementioned samples, the point is judged as an outlier and deleted from the global static point cloud data.

[0009] Furthermore, the step D also includes the completion and reconstruction of the global static point cloud data, and the specific process is as follows: D50: Traverse the point cloud in the entire global static point cloud data from left to right and from top to bottom to find the missing points d The position is (x d ,y d ), assign z to this point through the improved bidirectional interpolation method d , and get the point (x d ,y d ,z d ); D60:z d The value of is obtained by taking the average of the x and y directions. First, calculate the x direction z dx Right d Contribution, find x d+1 Is there a point at the location? If not, continue to search for x. d+2 、x d+3 , ..., until a point (x d+n ,y d ,z d+n ), find x d-1 and x d+n The slope value in the x direction , we get z xd+1 =z xd +K x ; D70: Find the z direction in the y direction dy Right d The contribution of is to find the existing point in the y direction ( x d ,y d+n ,z d+n ), then calculate the slope value first , and then find the z direction of the y component yd+1 =z yd +K y ; D80: Calculate the final (x d ,yd ) point z d Value z d =(z xd +z yd ) / 2, fill in the missing points d The position (x d ,y d ,z d ); D90: Calculate other missing points in the global static point cloud data according to D50 to D80, and complete the completion and reconstruction of the global static point cloud data.

[0010] Furthermore, the global static point cloud data in step D is obtained by controlling the rail-hanging robot to move along the rack rail fixed above the material yard before patrolling the warehouse, and obtaining the displacement data of the rail-hanging robot along the rack rail. s’ and two-dimensional measurement data based on the laser radar scanner coordinate system ( x'', y'' ), and then the displacement data s’ Convert the value into the world coordinate system x' , and the measured data ( x'', y'' ) is converted into world coordinate system data ( y', z' ), and finally form the global static point cloud data of the smelting pile ( x', y', z' ); or use the dynamic point cloud data obtained in the last patrol before the current patrol as the global static point cloud data.

[0011] Furthermore, in the step D, a difference analysis is performed by using the SegmentDifferences algorithm to calculate the difference between the global static point cloud data corresponding to before and after the warehouse patrol and the point cloud data after registration according to the configured threshold, so as to obtain the real-time physical changes of all the stockpiles after each scan; the data persisted to the file and the database is the point cloud data calculated by the SegmentDifferences algorithm to obtain a difference point cloud composed of several difference points, and the Euclidean clustering segmentation method is used to segment the difference point cloud into several point clouds, and then these point clouds are saved as point cloud files in the format of time\operation\auto-increment sequence, and the information of the point cloud file is stored in the database for backup.

[0012] Furthermore, in the step E, the real-time physical changes of the piled materials are intelligently and dynamically matched with the warehousing information, including intelligent dynamic matching of warehousing and intelligent dynamic matching of outbound warehousing; the intelligent dynamic matching of warehousing is that after the warehousing operation, the client obtains the aforementioned point cloud file and combines the volume and comprehensive density to calculate the comprehensive weight of each point cloud file to match the weight of the piled materials of the incoming material list, and then after manual confirmation of the match, the real-time dynamic density of the pile of materials is calculated and recorded; the intelligent dynamic matching of outbound warehousing is that after the outbound operation, the inventory point cloud before outbound warehousing and the point cloud after outbound warehousing are combined to find out the density and volume information of each point cloud, and then calculate and record the actual weight of the outbound point cloud.

[0013] The smelting stockpile storage system based on three-dimensional reconstruction of the present invention is implemented as follows: it includes a warehouse patrol device, a server, and a client, the warehouse patrol device is connected to the server by signal, and the client is connected to the server by network; The warehouse patrol device includes a rack rail fixedly installed above the material yard, a rail-hanging robot hoisted on the rack rail and meshed with the rack rail through gears, a servo motor fixedly arranged on the rail-hanging robot and with a drive shaft connected to the gear, a laser radar scanner fixedly arranged below the rail-hanging robot, and a network controller connected to the servo motor and the laser radar scanner signals, and the network controller is connected to the server signals; The server includes a warehouse patrol scanning module, a point cloud data formation module, a data preprocessing module, a difference analysis module, and an in-and-out warehouse matching module; The patrol scanning module is used to control the hanging rail robot to move along the rack rail fixed above the material field under the drive of the servo motor to patrol the warehouse, and obtain the displacement data of the hanging rail robot along the rack rail. s ; At the same time, the laser radar scanner is controlled to scan the smelting stockpile in the material yard during the movement of the hanging rail robot, and the two-dimensional measurement data based on the laser radar scanner coordinate system is obtained ( x', y' ); The point cloud data forming module is used to convert the displacement data s Convert the value x into the world coordinate system, and convert the measured data ( x', y' ) is converted into data in the world coordinate system ( y, z ), and finally form the dynamic point cloud data of smelting pile ( x, y, z ); The data preprocessing module is used to process dynamic point cloud data ( x, y, z) Filtering is performed, and then the dynamic point cloud data after filtering is processed ( x, y, z ) to perform real-time point cloud registration; The difference analysis module is used to perform difference analysis on the point cloud data obtained by real-time point cloud registration and the global static point cloud data before the warehouse patrol through the SegmentDifferences algorithm, calculate the real-time physical changes of the pile, and then persist them to files and databases; The in-and-out warehouse matching module is used to intelligently and dynamically match the real-time physical changes of the stockpile with the in-and-out warehouse information to form a complete, effective and authentic in-and-out warehouse record.

[0014] Furthermore, the lowest installation height of the rack rail is higher than the maximum height of the material pile in the material yard and is parallel to the ground of the material yard; the laser radar scanner is fixedly arranged at the bottom of the warehouse patrol robot, and the scanning direction of the laser radar scanner is perpendicular to the ground; the network controller is a WIFI router or a WIA-PA router and is connected to the corresponding router wireless network on the server side.

[0015] The beneficial effects of the present invention are: 1. The present invention uses a laser radar scanner carried by a rail-hanging robot to perform vertical scanning of the material yard. Combined with the fixed installation height of the rack rail, it not only completely avoids the signal loss problem caused by the blocking of scanning signals by piled materials in traditional ground scanning, and realizes no-dead-angle coverage of the entire material yard, but also can quickly obtain accurate three-dimensional point cloud data of irregular piles, effectively reducing the frequency of manual inspections, reducing the labor cost of inventory management, and improving the timeliness of smelting raw material management.

[0016] 2. The present invention adopts dynamic point cloud registration combined with SegmentDifferences algorithm, which can detect the changes of millimeter-level pile materials in real time, so that the volume measurement error rate of the pile material is controlled within 0.5%, which is at least 20 times higher than the accuracy of manual recording, thereby realizing the refined management of smelting raw materials, avoiding the problems of confusion in ore raw material varieties and error-prone weight recording in traditional inbound and outbound warehouse management, and providing timely, comprehensive and accurate raw material data for the smelting furnace.

[0017] 3. The present invention adopts world coordinate system conversion technology to fuse displacement data with scanning data to achieve precise three-dimensional coordinate positioning of the material pile, solving the positioning drift problem caused by dust interference in traditional aerial photography; and adopts global static point cloud data completion and reconstruction technology to complete the missing point cloud data, so that the completeness of three-dimensional reconstruction can reach 99.8%, which can effectively solve the problem of missing scanning data caused by irregular material pile shapes; and the dynamic point cloud alignment adopted can have a self-learning function, which can automatically optimize parameters as the working conditions of the material yard change, thereby improving the flexibility of application.

[0018] 4. The present invention adopts differential analysis and intelligent dynamic matching of in-and-out warehouse, segments the difference point cloud through Euclidean clustering, and combines the multi-dimensional data association of time / operation to realize automatic matching of in-and-out warehouse weight and material list, effectively improving the management efficiency of in-and-out warehouse; and through dynamic density calculation, the density parameters of the material can be automatically corrected according to the data of each entry, avoiding weight calculation errors caused by differences in raw material grades.

[0019] In summary, the present invention has the characteristics of strong real-time performance, high data collection efficiency and reliable storage records. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic structural diagram of a smelting stockpile storage system based on three-dimensional reconstruction according to an embodiment of the present invention; Figure 2 A three-dimensional point cloud image of a stockpile is scanned for a smelting stockpile storage method based on three-dimensional reconstruction according to an embodiment of the present invention; Among them: 1- material yard shed, 2- hanging rail robot, 3- rack rail, 4- network aggregation box, 5- optical cable, 6- smart factory network, 7- server, 8- client, 10- fixed building in the yard, 11- material pile. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0022] like Figure 1 and 2 As shown, the smelting stockpile storage method based on three-dimensional reconstruction of the present invention includes warehouse patrol scanning, point cloud data formation, data preprocessing, difference analysis, and in-and-out warehouse matching steps, and the specific contents are as follows: A. Warehouse inspection and scanning: The hanging rail robot with a laser radar scanner fixed below is driven by a servo motor to move along the rack rail fixed above the material yard to inspect the warehouse and obtain the displacement data of the hanging rail robot along the rack rail. s ; At the same time, the laser radar scanner scans the smelting stockpile in the material yard during the movement of the crane rail robot, and obtains two-dimensional measurement data based on the laser radar scanner coordinate system ( x', y' ); B. Point cloud data formation: Displacement data s Convert the value into the world coordinate system x , and the measured data ( x', y' ) is converted into data in the world coordinate system ( y, z ), and finally form the dynamic point cloud data of smelting pile ( x, y, z ); C. Data preprocessing: Dynamic point cloud data ( x, y, z ) is filtered, and then the dynamic point cloud data after filtering is processed ( x, y, z ) to perform real-time point cloud registration; D. Difference analysis: The point cloud data obtained by real-time point cloud registration is analyzed by SegmentDifferences algorithm with the global static point cloud data before the warehouse inspection, and the real-time physical changes of the pile are calculated, and then persisted to files and databases; E. In-and-out warehouse matching: Intelligently and dynamically match the real-time physical changes of the piled materials with the in-and-out warehouse information to form a complete, effective and authentic in-and-out warehouse record.

[0023] Before step A, it also includes a point cloud modeling step for fixed buildings on site, which is to generate point cloud pattern modeling through physical data, so as to facilitate the subsequent registration of dynamic point cloud data and restore the quality, volume, position and other physical information of the pile through global static point cloud data.

[0024] In the step A, the rail-hanging robot moves along the rack rail to patrol the warehouse when the material yard is completed in storage and out of storage; in the step C, the dynamic point cloud data ( x, y, z ) is filtered, which is to set a range on the attribute of the point, filter the points, and retain the points within the range on the attribute; in the D step, the global static point cloud data is first filtered, and then a difference analysis is performed; wherein, the global static point cloud data is filtered, and a Statistical OutlierRemoval filter is used to remove outliers in the global static point cloud data.

[0025] In the above step C, the real-time point cloud registration is performed by positioning the origin of the world coordinate system as the intersection of the wall on the starting side of the rack and the wall above the starting side of the rack, and the dynamic point cloud coordinates ( x, y, z ) is converted into coordinates in the world coordinate system ( x 1 , y 1 , z ), and according to the program, the accuracy (such as decimeter level) is pre-set, and the coordinates ( x 1 , y 1 , z ) to perform regularization operations to obtain the final aligned world coordinate system coordinates ( x 2 , y 2 , z ).

[0026] In step C, the specific process of filtering the dynamic point cloud data is as follows: C10: specifies a dimension and the value range under this dimension; C20: traverse each point in the dynamic point cloud data to determine whether the value of the point under the specified dimension is within the value range. If not, the point is determined to be an outlier and deleted from the dynamic point cloud data; C30: The traversal is completed, and the remaining points that have not been deleted are the dynamic point cloud data points after filtering.

[0027] In the step D, the specific process of filtering the global static point cloud data is as follows: D10: For each point in the global static point cloud data, calculate the average distance d from the point to all points in the K neighborhood of the point, and obtain an array containing the d values ​​of each point, recorded as distance; D20: Assume that each element in the distance array forms a Gaussian distribution, and the array is a sample; traverse all points in the global static point cloud data. If the d value of the point is outside the standard range defined by the mean and variance of the aforementioned samples, the point is judged as an outlier and deleted from the global static point cloud data.

[0028] The steps C and D also include deintersecting the filtered dynamic point cloud data and global static point cloud data with the point cloud data of fixed buildings, that is, removing the corresponding fixed building point cloud data in the dynamic point cloud data and the global static point cloud data, and leaving the point cloud data corresponding to the remaining material pile.

[0029] The step D also includes the completion and reconstruction of the global static point cloud data. The specific process is as follows: D50: Traverse the point cloud in the entire global static point cloud data from left to right and from top to bottom to find the missing points d The position is (x d ,y d ), assign z to this point through the improved bidirectional interpolation method d , and get the point (x d ,y d ,z d ); D60:z d The value of is obtained by taking the average of the x and y directions. First, calculate the x direction z dx Right d Contribution, find x d+1 Is there a point at the location? If not, continue to search for x. d+2 、x d+3 , ..., until a point (x d+n ,y d ,z d+n ), find x d-1 and x d+nThe slope value in the x direction , we get z xd+1 =z xd +K x ; D70: Find the z direction in the y direction dy Right d The contribution of is to find the existing point in the y direction ( x d ,y d+n ,z d+n ), then calculate the slope value first , and then find the z direction of the y component yd+1 =z yd +K y ; D80: Calculate the final (x d ,y d ) point z d Value z d =(z xd +z yd ) / 2, fill in the missing points d The position (x d ,y d ,z d ); D90: Calculate other missing points in the global static point cloud data according to D50 to D80, and complete the completion and reconstruction of the global static point cloud data.

[0030] The global static point cloud data in step D is obtained by controlling the hanging rail robot to move along the rack rail fixed above the material yard before patrolling the warehouse, and obtaining the displacement data of the hanging rail robot along the rack rail. s’ and two-dimensional measurement data based on the laser radar scanner coordinate system ( x'', y'' ), and then the displacement data s’ Convert the value into the world coordinate system x' , and the measured data ( x'', y'' ) is converted into world coordinate system data ( y', z'' ), and finally form the global static point cloud data of the smelting pile ( x', y', z') ; or use the dynamic point cloud data obtained in the last patrol before the current patrol as the global static point cloud data.

[0031] In the step D, a difference analysis is performed by using the SegmentDifferences algorithm to calculate the difference between the global static point cloud data corresponding to before and after the warehouse patrol and the point cloud data after registration according to the configured threshold, so as to obtain the real-time physical changes of all the stockpiles after each scan; the data persisted to the file and the database is the point cloud data calculated by the SegmentDifferences algorithm to obtain a difference point cloud composed of several difference points, and the Euclidean clustering segmentation method is used to segment the difference point cloud into several point clouds, and then these point clouds are saved as point cloud files in the format of time\operation\auto-increment sequence, and the information of the point cloud file is stored in the database for backup.

[0032] In the step E, the real-time physical changes of the piled materials are intelligently and dynamically matched with the warehouse-in and warehouse-out information, including intelligent dynamic matching of warehouse-in and warehouse-out; the intelligent dynamic matching of warehouse-in is that after the warehouse-in operation, the client obtains the aforementioned point cloud file and combines the volume and comprehensive density to calculate the comprehensive weight of each point cloud file to match the weight of the piled materials of this warehouse-in material list, and then calculates and records the real-time dynamic density of the piled materials after manual confirmation of the match; the intelligent dynamic matching of warehouse-out is that after the warehouse-out operation, the inventory point cloud before warehouse-out and the point cloud after warehouse-out are combined to find out the density and volume information of each point cloud, and then calculate and record the actual weight of the warehouse-out point cloud.

[0033] The smelting stockpile storage system based on three-dimensional reconstruction of the present invention comprises a warehouse patrol device, a server, and a client, wherein the warehouse patrol device is connected to the server by signal, and the client is connected to the server by network; The warehouse patrol device includes a rack rail fixedly installed above the material yard, a rail-hanging robot hoisted on the rack rail and meshed with the rack rail through gears, a servo motor fixedly arranged on the rail-hanging robot and with a drive shaft connected to the gear, a laser radar scanner fixedly arranged below the rail-hanging robot, and a network controller connected to the servo motor and the laser radar scanner signals, and the network controller is connected to the server signals; The server includes a warehouse patrol scanning module, a point cloud data formation module, a data preprocessing module, a difference analysis module, and an in-and-out warehouse matching module. The patrol scanning module is used to control the hanging rail robot to move along the rack rail fixed above the material field under the drive of the servo motor to patrol the warehouse, and obtain the displacement data of the hanging rail robot along the rack rail. s ; At the same time, the laser radar scanner is controlled to scan the smelting stockpile in the material yard during the movement of the hanging rail robot, and the two-dimensional measurement data based on the laser radar scanner coordinate system is obtained ( x', y' ); The point cloud data forming module is used to convert the displacement data s Convert the value x into the world coordinate system, and convert the measured data (x', y' ) is converted into data in the world coordinate system ( y, z ), and finally form the dynamic point cloud data of smelting pile ( x, y, z ); The data preprocessing module is used to process dynamic point cloud data ( x, y, z ) is filtered, and then the dynamic point cloud data after filtering is processed ( x, y, z ) to perform real-time point cloud registration; The difference analysis module is used to perform difference analysis on the point cloud data obtained by real-time point cloud registration and the global static point cloud data before the warehouse patrol through the SegmentDifferences algorithm, calculate the real-time physical changes of the pile, and then persist them to files and databases; The in-and-out warehouse matching module is used to intelligently and dynamically match the real-time physical changes of the stockpile with the in-and-out warehouse information to form a complete, effective and authentic in-and-out warehouse record.

[0034] The minimum installation height of the rack rail is higher than the maximum height of the material pile in the material yard and is parallel to the ground of the material yard; the laser radar scanner is fixedly arranged at the bottom of the warehouse patrol robot, and the scanning direction of the laser radar scanner is perpendicular to the ground; the network controller is a WIFI router or a WIA-PA router and is connected to the corresponding router wireless network on the server side.

[0035] Example 1 like Figure 1 and 2 As shown in the figure, the specific process of the storage and out-of-warehouse management of the smelting material pile in the shed 1 of the copper smelting enterprise is as follows: S90: When the material yard shed 1 is empty, operate according to steps S100 and S200 to form point cloud data recording the coordinates and length, width and height of the fixed buildings in the material yard shed 1, and then use the program to complete the point cloud modeling of the fixed buildings 10 in the yard according to the point cloud data.

[0036] S100: When the rail robot 2 has completed the entry and exit of the material yard, the rail robot 2 with a laser radar scanner fixed below moves along the rack 3 fixed above the material yard under the drive of the servo motor to patrol the warehouse. The rail robot 2 provides displacement data along the rack 3 every 50ms (accurate displacement data is obtained by the meshing of the gear and the rack 3 in combination with the servo motor) s ; At the same time, the laser radar scanner scans the smelting stockpile in the material yard during the movement of the hanging rail robot 2, and obtains two-dimensional measurement data based on the laser radar scanner coordinate system (the internal control system of the laser radar scanner can automatically convert the scanned slope height into vertical height) ( x', y' ); then through the network aggregation box 4 and the smart factory network 6, the displacement data obtained above sand two-dimensional measurement data ( x', y' ) is transmitted to server 7.

[0037] S200: Server 7 sends displacement data s Convert the value into the world coordinate system x , and the measured data ( x', y' ) is converted into data in the world coordinate system ( y, z ), and finally form the dynamic point cloud data of smelting pile ( x, y, z ).

[0038] S300: Dynamic point cloud data ( x, y, z ) is filtered, and then the filtered dynamic point cloud data and the point cloud data of the fixed building are deintersected to remove the fixed building point cloud data corresponding to the dynamic point cloud data and the point cloud data corresponding to the remaining material pile. Finally, the remaining point cloud data is aligned in real time.

[0039] Among them, for dynamic point cloud data ( x, y, z The specific process of filtering is as follows: S310: Specify a dimension and a value range under the dimension.

[0040] S320: traverse each point in the dynamic point cloud data to determine whether the value of the point under the specified dimension is within the value range. If not, the point is determined to be an outlier and deleted from the dynamic point cloud data.

[0041] S330: The traversal is completed, and the remaining points that have not been deleted are the dynamic point cloud data points after filtering.

[0042] S400: First, filter the global static point cloud data, then complete and reconstruct the global static point cloud data, and then use the SegmentDifferences algorithm to perform difference analysis on the point cloud data obtained by real-time point cloud registration and the global static point cloud data before patrolling the warehouse, calculate the real-time physical changes of the pile, and then persist them to files and databases.

[0043] Among them, the global static point cloud data is obtained by controlling the hanging rail robot 2 to move along the rack rail 3 fixed above the material yard to patrol the warehouse before patrolling the warehouse, and obtaining the displacement data of the hanging rail robot 2 along the rack rail 3 s’ and two-dimensional measurement data based on the laser radar scanner coordinate system ( x'', y'' ), and then the displacement data s’ Convert the value into the world coordinate system x' , and the measured data ( x'', y'' ) is converted into world coordinate system data ( y', z' ), and finally form the global static point cloud data of the smelting pile ( x', y', z'); or use the dynamic point cloud data obtained in the last patrol before the current patrol as the global static point cloud data.

[0044] The difference analysis uses the SegmentDifferences algorithm to calculate the difference between the global static point cloud data before and after the patrol and the registered point cloud data according to the configured threshold, and obtain the real-time physical changes of all the piles after each scan.

[0045] The data persisted to the file and database is the point cloud data calculated using the SegmentDifferences algorithm to obtain a difference point cloud composed of several difference points (this difference point cloud will be sent to the front end for manual proofreading by the staff, and the point cloud will be bound to the actual in-and-out material pile and stored in the database to form in-and-out records). The Euclidean clustering segmentation method is used to segment the difference point cloud into several point clouds, and then these point clouds are saved as point cloud files in the format of time\operation\auto-increment sequence, and the information of the point cloud file is stored in the database for backup.

[0046] The specific process of filtering global static point cloud data is as follows: S410: For each point in the global static point cloud data, calculate the average distance d from the point to all points in the K neighborhood of the point, and obtain an array containing the d values ​​of each point, recorded as distance. S420: Assuming that each element in the distance array forms a Gaussian distribution, the array is a sample; traverse all points in the global static point cloud data, if the d value of the point is outside the standard range defined by the mean and variance of the aforementioned samples, the point is judged as an outlier and deleted from the global static point cloud data.

[0047] The specific process of completing and reconstructing the global static point cloud data is as follows: S450: Traverse the point cloud in the entire global static point cloud data from left to right and from top to bottom to find the missing points d The position is (x d ,y d ), assign a value z to this point by using an improved bidirectional interpolation method (the ordinary bidirectional interpolation method only considers the linear change of two adjacent points, and for a higher degree of restoration, the embodiment expands the points considered to four points or N points in the same dimension to achieve a higher and better degree of restoration). d , and get the point (x d ,y d ,z d ).

[0048] S460:z d The value of is obtained by taking the average of the x and y directions. First, calculate the x direction z dx Rightd Contribution, find x d+1 Is there a point at the location? If not, continue to search for x. d+2 、x d+3 , ..., until a point (x d+n ,y d ,z d+n ), find x d-1 and x d+n The slope value in the x direction , we get z xd+1 =z xd +K x .

[0049] S470: Find the z direction in the y direction dy Right d The contribution of is to find the existing point in the y direction ( x d ,y d+n ,z d+n ), then calculate the slope value first , and then find the z direction of the y component yd+1 =z yd +K y .

[0050] S480: Calculate the final (x d ,y d ) point z d Value z d =(z xd +z yd ) / 2, fill in the missing points d The position (x d ,y d ,z d ); S490: According to steps S450 to S480, other missing points in the global static point cloud data are obtained to complete the completion and reconstruction of the global static point cloud data.

[0051] S500: Intelligently and dynamically match the real-time physical changes of the pile with the in-and-out information to form a complete, effective and authentic in-and-out record.

[0052] Among them: the intelligent dynamic matching of warehousing is that after the warehousing operation, the client 8 obtains the aforementioned point cloud file and combines the volume (the volume here is obtained based on the point cloud computing) and the comprehensive density (the comprehensive density here is the system preset empirical density) to calculate the comprehensive weight of each point cloud file, which is used to match the stockpiling weight of the material list for this warehousing, and then calculates and records the real-time dynamic density of the material pile after manually confirming the match.

[0053] Intelligent dynamic matching for outbound delivery is to find out the density and volume information of each point cloud by combining the inventory point cloud before outbound delivery and the point cloud after outbound delivery after the outbound delivery operation, and then calculate and record the actual weight of the outbound point cloud.

[0054] During the application process, the client 8 operation interface is implemented using wpf technology, and communicates with the server 7 through socket to obtain the required point cloud data, and renders the point cloud data through Helix Toolkit to achieve the purpose of real-time preview of all material pile data in the material yard. Through RESTful (REST stands for Representational State Transfer. RESTful is a common REST application, a web service that follows the REST style, and a ROA (resource-oriented architecture)) architecture, it communicates with the business processing server to achieve real-time synchronous display of material yard data, and after the material yard is in and out of the warehouse, the analyzed point cloud difference file is obtained from the server 7, and dynamic data matching is performed based on the volume, location and real-time inventory information. The dynamic density information of the pile is calculated based on the volume and weight of the pile, and persisted to the database through the server 7. During each outbound operation, the relatively accurate outbound weight data can be obtained based on the outbound point cloud location and volume information combined with the dynamic density.

[0055] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A smelting stockpile storage method based on three-dimensional reconstruction, characterized in that: It includes patrol scanning, point cloud data formation, data preprocessing, difference analysis, and in-and-out warehouse matching steps. The specific contents are as follows: A. Warehouse inspection and scanning: The hanging rail robot with a laser radar scanner fixed below is driven by a servo motor to move along the rack rail fixed above the material yard to inspect the warehouse and obtain the displacement data of the hanging rail robot along the rack rail. s ; At the same time, the laser radar scanner scans the smelting stockpile in the material yard during the movement of the crane rail robot, and obtains two-dimensional measurement data based on the laser radar scanner coordinate system ( x',y' ); B. Point cloud data formation: Displacement data s Convert the value into the world coordinate system x , and the measured data ( x',y' ) is converted into data in the world coordinate system ( y,z ), and finally form the dynamic point cloud data of smelting pile ( x,y,z ); C. Data preprocessing: Dynamic point cloud data ( x,y,z ) is filtered, and then the dynamic point cloud data after filtering is processed ( x,y,z ) to perform real-time point cloud registration; D. Difference analysis: The point cloud data obtained by real-time point cloud registration is analyzed by SegmentDifferences algorithm with the global static point cloud data before the warehouse inspection, and the real-time physical changes of the pile are calculated, and then persisted to files and databases; E. In-and-out warehouse matching: Intelligently and dynamically match the real-time physical changes of the piled materials with the in-and-out warehouse information to form a complete, effective and authentic in-and-out warehouse record.

2. The smelting stockpiling storage method based on three-dimensional reconstruction according to claim 1 is characterized in that: In the step A, the rail-hanging robot moves along the rack rail to patrol the warehouse when the material yard is completed in storage and out of storage; in the step C, the dynamic point cloud data ( x,y,z ) is filtered, which is to set a range on the attribute of the point, filter the points, and retain the points within the range on the attribute; in the D step, the global static point cloud data is first filtered, and then a difference analysis is performed; wherein, the global static point cloud data is filtered, and a Statistical Outlier Removal filter is used to remove outliers in the global static point cloud data.

3. The smelting stockpiling storage method based on three-dimensional reconstruction according to claim 2 is characterized in that: In step C, the specific process of filtering the dynamic point cloud data is as follows: C10: specifies a dimension and the value range under this dimension; C20: traverse each point in the dynamic point cloud data to determine whether the value of the point under the specified dimension is within the value range. If not, the point is determined to be an outlier and deleted from the dynamic point cloud data; C30: The traversal is completed, and the remaining points that have not been deleted are the dynamic point cloud data points after filtering.

4. The smelting stockpiling storage method based on three-dimensional reconstruction according to claim 2 is characterized in that: In the step D, the specific process of filtering the global static point cloud data is as follows: D10: For each point in the global static point cloud data, calculate the average distance d from the point to all points in the K neighborhood of the point, and obtain an array containing the d values ​​of each point, recorded as distance; D20: Assume that each element in the distance array forms a Gaussian distribution, and the array is a sample; All points in the global static point cloud data are traversed. If the d value of the point is outside the standard range defined by the mean and variance of the aforementioned samples, the point is judged as an outlier and deleted from the global static point cloud data.

5. The smelting stockpile storage method based on three-dimensional reconstruction according to claim 2 is characterized in that: The step D also includes the completion and reconstruction of the global static point cloud data. The specific process is as follows: D50: Traverse the point cloud in the entire global static point cloud data from left to right and from top to bottom to find the missing points d The position is (x d ,y d ), assign z to this point through the improved bidirectional interpolation method d , and get the point (x d ,y d ,z d ); D60:z d The value of is obtained by taking the average of the x and y directions. First, calculate the x direction z dx Right d Contribution, find x d+1 Is there a point at the location? If not, continue to search for x. d+2 、x d+3 , ..., until a point (x d+n ,y d ,z d+n ), find x d-1 and x d+n The slope value in the x direction , we get z xd+1 =z xd +K x ; D70: Find the z direction in the y direction dy Right d The contribution of finding the existing point (x d ,y d+n ,z d+n ), then calculate the slope value first , and then find the z direction of the y component yd+1 =z yd +K y ; D80: Calculate the final (x d ,y d ) point z d Value z d =(z xd +z yd ) / 2, fill in the missing points d The position (x d ,y d ,z d ); D90: Calculate other missing points in the global static point cloud data according to D50 to D80, and complete the completion and reconstruction of the global static point cloud data.

6. The smelting stockpile storage method based on three-dimensional reconstruction according to claim 1 is characterized in that: The global static point cloud data in step D is obtained by controlling the hanging rail robot to move along the rack fixed above the material field before patrolling the warehouse, and obtaining the displacement data s' of the hanging rail robot along the rack and the two-dimensional measurement data based on the laser radar scanner coordinate system ( x'', y'' ), and then the displacement data s’ Convert the value into the world coordinate system x’ , and the measured data ( x'',y'' ) into world coordinate system data ( y',z'' ), and finally form the global static point cloud data of the smelting pile ( x',y',z' ); or use the dynamic point cloud data obtained in the last patrol before the current patrol as the global static point cloud data.

7. The smelting stockpile storage method based on three-dimensional reconstruction according to any one of claims 1 to 6, characterized in that: In the step D, a difference analysis is performed by using the SegmentDifferences algorithm to calculate the difference between the global static point cloud data corresponding to before and after the warehouse patrol and the point cloud data after registration according to the configured threshold, so as to obtain the real-time physical changes of all the stockpiles after each scan; the data persisted to the file and the database is the point cloud data calculated by the SegmentDifferences algorithm to obtain a difference point cloud composed of several difference points, and the Euclidean clustering segmentation method is used to segment the difference point cloud into several point clouds, and then these point clouds are saved as point cloud files in the format of time\operation\auto-increment sequence, and the information of the point cloud file is stored in the database for backup.

8. The smelting stockpile storage method based on three-dimensional reconstruction according to claim 7 is characterized in that: In the step E, the real-time physical changes of the piled materials are intelligently and dynamically matched with the warehouse-in and warehouse-out information, including intelligent dynamic matching of warehouse-in and warehouse-out; the intelligent dynamic matching of warehouse-in is that after the warehouse-in operation, the client obtains the aforementioned point cloud file and combines the volume and comprehensive density to calculate the comprehensive weight of each point cloud file to match the weight of the piled materials of this warehouse-in material list, and then calculates and records the real-time dynamic density of the piled materials after manual confirmation of the match; the intelligent dynamic matching of warehouse-out is that after the warehouse-out operation, the inventory point cloud before warehouse-out and the point cloud after warehouse-out are combined to find out the density and volume information of each point cloud, and then calculate and record the actual weight of the warehouse-out point cloud.

9. A smelting stockpile storage system based on three-dimensional reconstruction, characterized in that: It includes a warehouse patrol device, a server, and a client, wherein the warehouse patrol device is connected to the server by signal, and the client is connected to the server by network; The warehouse patrol device includes a rack rail fixedly installed above the material yard, a rail-hanging robot hoisted on the rack rail and meshed with the rack rail through gears, a servo motor fixedly arranged on the rail-hanging robot and with a drive shaft connected to the gear, a laser radar scanner fixedly arranged below the rail-hanging robot, and a network controller connected to the servo motor and the laser radar scanner signals, and the network controller is connected to the server signals; The server includes a warehouse patrol scanning module, a point cloud data formation module, a data preprocessing module, a difference analysis module, and an in-and-out warehouse matching module; The patrol scanning module is used to control the hanging rail robot to move along the rack rail fixed above the material field under the drive of the servo motor to patrol the warehouse, and obtain the displacement data of the hanging rail robot along the rack rail. s ; At the same time, the laser radar scanner is controlled to scan the smelting stockpile in the material yard during the movement of the hanging rail robot, and the two-dimensional measurement data based on the laser radar scanner coordinate system is obtained ( x',y' ); The point cloud data forming module is used to convert the displacement data s Convert the value x into the world coordinate system, and convert the measured data ( x',y' ) is converted into data in the world coordinate system ( y,z ), and finally form the dynamic point cloud data of smelting pile ( x,y,z ); The data preprocessing module is used to process dynamic point cloud data ( x,y,z ) is filtered, and then the dynamic point cloud data after filtering is processed ( x,y,z ) to perform real-time point cloud registration; The difference analysis module is used to perform difference analysis on the point cloud data obtained by real-time point cloud registration and the global static point cloud data before the warehouse patrol through the SegmentDifferences algorithm, calculate the real-time physical changes of the pile, and then persist them to files and databases; The in-and-out warehouse matching module is used to intelligently and dynamically match the real-time physical changes of the stockpile with the in-and-out warehouse information to form a complete, effective and authentic in-and-out warehouse record.

10. The smelting stockpile storage system based on three-dimensional reconstruction according to claim 9 is characterized in that: The minimum installation height of the rack rail is higher than the maximum height of the material pile in the material yard and is parallel to the ground of the material yard; the laser radar scanner is fixedly arranged at the bottom of the warehouse patrol robot, and the scanning direction of the laser radar scanner is perpendicular to the ground; the network controller is a WIFI router or a WIA-PA router and is connected to the corresponding router wireless network on the server side.

Images