Fusion point cloud-based stock ground data management method and system

Abstract

The invention provides a stock yard data management method and system based on fusion point clouds, and relates to the technical field of fusion point clouds, and the method comprises the steps: arranging high-precision three-dimensional scanning equipment in a target monitoring area, constructing a complete three-dimensional point cloud model, carrying out the recognition and segmentation of the preprocessed point cloud data, constructing a three-dimensional model for a segmented material pile, and carrying out the recognition and segmentation of the three-dimensional model. The method comprises the following steps: identifying the shape and position change of a material pile, updating stock data information of the material pile in a stock yard in real time, acquiring stock data and stock yard geographic information in real time based on the Internet of Things technology and a geographic information system GIS, integrating the stock data and the stock yard geographic information into a genetic algorithm, and planning an optimal material taking and stacking path of stock yard equipment in real time. The technical problems that in the prior art, due to manual management, the data precision is not high, time and labor are consumed in data recording and updating, real-time data updating is difficult to achieve, and even potential safety hazards exist can be solved, the purpose of improving the data precision is achieved, and the technical effects of rapidly updating stock ground data and reducing safety risks are achieved.

Description

Technical Field

[0001] The present disclosure relates to the fields of big data processing and three-dimensional space information technology, and in particular to a material field data management method and system based on fused point cloud. Background Art

[0002] Material yard management is crucial in industries such as industry, construction, and mining. Traditional material yard management relies on manual labor. Since manual measurement is time-consuming and labor-intensive, and is limited by weather, terrain, and other conditions, it is difficult to achieve high-frequency, high-precision data collection. In the prior art, two-dimensional images are used to analyze material yard data, but two-dimensional data is difficult to fully reflect the three-dimensional spatial structure of the material yard, which limits the depth and breadth of data analysis. There are also problems such as inaccurate data and untimely updates. Therefore, based on limited and static data, it is difficult to achieve accurate inventory management, material flow analysis, and optimization decisions, and even wrong judgments may occur, resulting in safety hazards. Therefore, a method is needed to solve the above problems.

[0003] In summary, traditional material yard management relies on manual measurement and two-dimensional image analysis. This method is not only inefficient, but also difficult to ensure the accuracy and timeliness of data. Manual management leads to low data accuracy, and recording and updating data is time-consuming and labor-intensive. It is difficult to achieve real-time data updates, and there are even technical problems that pose safety risks. Summary of the invention

[0004] The present disclosure provides a material yard data management method based on fused point cloud to solve the technical problems in the prior art that manual management leads to low data accuracy, recording and updating data is time-consuming and labor-intensive, it is difficult to achieve real-time data update, and there are even safety hazards.

[0005] According to a first aspect of the present disclosure, a material field data management method based on fused point cloud is provided, comprising:

[0006] High-precision 3D scanning equipment is deployed in the target monitoring area. The high-precision 3D scanning equipment is used to collect point cloud data of the material yard; the collected point cloud data is pre-processed, and the multi-scale features of the point cloud data are extracted using the PointNet++ deep learning network. The pre-processing includes filtering, noise removal, data compression and classification; the point cloud data obtained by multiple scanning devices and at different times are spatially registered, and the point cloud data spliced ​​together are used to build a complete 3D point cloud model. Modeling software and point cloud plug-ins are used for entity modeling; the multi-scale features of the pre-processed point cloud data are identified and segmented based on a multi-category geometric feature fusion model; the segmented objects are A three-dimensional model of a triangular mesh is constructed for the material pile, and the volume of the material pile is calculated using the mesh volume method based on the constructed three-dimensional model. Within a predetermined time interval, a three-dimensional laser scan is performed on the material yard to obtain multiple sets of point cloud data, and the newly acquired point cloud is aligned with the historical data to eliminate differences in scanning positions and angles. By comparing continuous point cloud data, changes in the shape and position of the material pile are identified, and the inventory data information of the material pile in the material yard is updated in real time according to changes in the volume and position of the material pile. Based on the Internet of Things technology and the Geographic Information System (GIS), inventory data and material yard geographic information are acquired in real time and integrated into the genetic algorithm to plan the optimal path for the material yard equipment to retrieve and stack materials in real time.

[0007] According to a second aspect of the present disclosure, a material field data management system based on fused point cloud is provided, comprising:

[0008] Scanning equipment deployment module, the scanning equipment deployment module is used to deploy high-precision 3D scanning equipment in the target monitoring area, and the high-precision 3D scanning equipment is used to collect point cloud data of the material yard; point cloud data preprocessing module, the point cloud data preprocessing module is used to preprocess the collected point cloud data, and use the PointNet++ deep learning network to extract the multi-scale features of the point cloud data. The preprocessing includes filtering, noise removal, data compression and classification; solid modeling module, the solid modeling module is used to spatially align the point cloud data acquired by multiple scanning devices and at different times, and stitch the point cloud data together to build a complete 3D point cloud model, and use modeling software and point cloud plug-ins for solid modeling; multi-scale feature recognition and segmentation module, the multi-scale feature recognition and segmentation module is used to identify and segment the multi-scale features of the preprocessed point cloud data based on a multi-category geometric feature fusion model. The multi-category geometric feature fusion model is a combination of multi-scale feature fusion and residual mixed attention point cloud registration algorithm for feature extraction, and uses a residual structure The hybrid attention module enhances effective features, predicts the transformed point cloud data through Transformer and soft pointer, and solves the rigid transformation with differentiable singular value decomposition; the material pile volume calculation module, the material pile volume calculation module is used to construct a three-dimensional model of a triangular mesh for the segmented material pile, and calculate the volume of the material pile by the mesh volume method based on the constructed three-dimensional model; the material pile inventory data information update module, the material pile inventory data information update module is used to perform three-dimensional laser scanning on the material yard within a predetermined time interval to obtain multiple sets of point cloud data, align the newly collected point cloud with the historical data to eliminate the difference in scanning position and angle, and identify the changes in the shape and position of the material pile by comparing continuous point cloud data. According to the changes in the volume and position of the material pile, the material pile inventory data information of the material yard is updated in real time; the path planning module, the path planning module is used to obtain inventory data and material yard geographic information in real time based on the Internet of Things technology and geographic information system (GIS) and integrate them into the genetic algorithm to plan the optimal path for the material yard equipment to take and stack materials in real time.

[0009] One or more technical solutions provided in the present disclosure have at least the following technical effects or advantages: by deploying high-precision 3D scanning equipment in the target monitoring area, the high-precision 3D scanning equipment is used to collect point cloud data of the material field; the collected point cloud data is preprocessed, and the multi-scale features of the point cloud data are extracted using the PointNet++ deep learning network, and the preprocessing includes filtering, noise removal, data compression and classification; the point cloud data obtained by multiple scanning devices and at different times are spatially registered, and the point cloud data spliced ​​together are used to build a complete 3D point cloud model, and modeling software and point cloud plug-ins are used for entity modeling; the multi-scale features of the preprocessed point cloud data are fused based on a multi-type geometric feature fusion model. The method can identify and segment the material pile based on the degree of features; construct a three-dimensional model of triangular mesh for the segmented material pile, and calculate the volume of the material pile by the mesh volume method based on the constructed three-dimensional model; perform three-dimensional laser scanning on the material yard within a predetermined time interval to obtain multiple sets of point cloud data, align the newly collected point cloud with the historical data to eliminate the differences in scanning position and angle, identify the changes in the shape and position of the material pile by comparing continuous point cloud data, and update the material pile inventory data information in real time according to the changes in the volume and position of the material pile; based on the Internet of Things technology and the Geographic Information System GIS, the inventory data and material yard geographic information are acquired in real time and integrated into the genetic algorithm to plan the optimal path for the material yard equipment to take and stack materials in real time. The method solves the technical problems in the prior art that the data accuracy is low due to manual management, the recording and updating of data is time-consuming and labor-intensive, it is difficult to achieve real-time data update, and there are even safety hazards, and achieves the technical effect of improving data accuracy, achieving rapid update of material yard data, and reducing safety risks.

[0010] It should be understood that the contents described in this section are not intended to indicate the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0012] Figure 1 A schematic diagram of a flow chart of a material field data management method based on fused point cloud provided in an embodiment of the present disclosure;

[0013] Figure 2 A schematic diagram of the structure of a material yard data management system based on fused point cloud provided in an embodiment of the present disclosure.

[0014] Explanation of the reference numerals: scanning equipment layout module 11, point cloud data preprocessing module 12, entity modeling module 13, multi-scale feature recognition and segmentation module 14, material pile volume calculation module 15, material pile inventory data information update module 16, path planning module 17. DETAILED DESCRIPTION

[0015] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0016] Embodiment 1

[0017] The material field data management method based on fused point cloud provided by the embodiment of the present disclosure is referred to Figure 1 For illustration, the methods include:

[0018] High-precision 3D scanning equipment is deployed in the target monitoring area to collect point cloud data of the material yard;

[0019] Specifically, the target monitoring area is the material field to be point cloud data. High-precision three-dimensional scanning equipment is a device that uses high-precision three-dimensional technology to scan point cloud data. Specific equipment includes: ground laser radar, which is suitable for high-precision scanning of fixed points and can provide high-density point cloud data; laser radar carried by drones, which is suitable for scanning large-scale and difficult-to-reach areas and can quickly obtain three-dimensional data of the entire material field; handheld three-dimensional scanners, which are suitable for detailed scanning of local areas and are easy to move and operate; scanning systems carried by vehicles or robots, which are suitable for long-distance linear scanning, such as continuous measurement of the edge of the material pile. A coordinate system is established in the target monitoring area with a random point as the origin, and a high-precision three-dimensional scanning device is arranged in the target monitoring area, the arrangement coordinates are recorded, the power circuit of the high-precision three-dimensional scanning device is controlled to be turned on, and the high-precision three-dimensional scanning device is started. The point cloud data collected and determined by the high-precision three-dimensional scanning device is a three-dimensional data representation method, which consists of a set of discrete points, each point has a three-dimensional coordinate (X, Y, Z) to represent the position in space, and these points together depict the three-dimensional shape of the material pile. At the same time, the point cloud data can be used to construct a three-dimensional model of the material pile.

[0020] Preprocess the collected point cloud data and use the PointNet++ deep learning network to extract multi-scale features of the point cloud data. The preprocessing includes filtering, noise removal, data compression and classification.

[0021] Specifically, during the point cloud data acquisition process, noise points such as outliers, duplicate points or points of non-target objects may be introduced due to equipment limitations, environmental conditions or operational errors. The first step of preprocessing is to remove these noise points through filtering techniques, including statistical filtering, which removes outliers based on the local statistical characteristics of the point cloud, such as the density or distance of the points; voxel filtering, which divides the space into voxels, averages or selects point cloud data within each voxel, reduces the amount of data and removes noise. The amount of point cloud data is usually very large, and direct processing consumes a lot of computing resources and time. Data compression aims to reduce the amount of data while trying to maintain the structural information of the point cloud. Common methods include: downsampling, which reduces the number of points, such as random downsampling, uniform downsampling or importance-based sampling; feature point extraction, which identifies key points in the point cloud, such as edge points, corner points or feature points of the surface, for subsequent processing. The last step of preprocessing is preliminary classification, which divides the point cloud data into different categories. This step can be achieved through simple rules based on height, density or geometric features, or preliminary classification using machine learning models. The PointNet++ deep learning network is used to extract multi-scale features of point cloud data. PointNet++ extracts multi-scale features of point cloud data including: sampling and grouping, hierarchical sampling in the point cloud, building a point set hierarchy from coarse to fine, each layer represents a point cloud view of different scales, and the points around each sampling point are gathered through grouping; local feature extraction, using the PointNet module to learn local features on each layer of grouped point cloud; the extracted multi-scale features can be used for tasks such as point cloud classification, segmentation, registration or reconstruction, improving the accuracy and robustness of the model. Through the above preprocessing and feature extraction process, the processing efficiency and analysis accuracy of point cloud data can be effectively improved.

[0022] Perform spatial registration on point cloud data acquired from multiple scanning devices at different times, stitch together the point cloud data, build a complete 3D point cloud model, and use modeling software and point cloud plug-ins for solid modeling;

[0023] Specifically, spatial registration is the process of aligning point cloud data from different sources to the same spatial coordinate system. The main methods include: feature-based registration, which calculates the registration transformation by identifying and matching feature points between point clouds; iterative closest point algorithm ICP, which automatically finds the closest point pair between point clouds, and iteratively optimizes the registration parameters until the best match; global registration, such as the RANSAC algorithm, processes large point cloud data sets to improve registration efficiency. The registered point cloud data needs to be fused, and the construction of a complete three-dimensional model includes point cloud fusion algorithms, which use voxel rasterization or density-based point cloud fusion methods to remove duplicate points and optimize the model; data consistency verification to ensure the geometric and physical consistency of the fused point cloud and avoid abnormal structures. After building a complete three-dimensional point cloud model, the use of modeling software and plug-ins for entity modeling includes: point cloud to mesh conversion to convert point cloud data into triangular meshes to form a continuous surface; model optimization, reducing the number of faces, smoothing, and improving the computational efficiency and aesthetics of the model; feature extraction, extracting entity features from the mesh model, such as boundaries, holes, etc.; texture mapping, mapping real texture information to the model surface to increase realism. Among them, point cloud data is usually composed of a large number of discrete points, while the solid model requires a continuous surface representation. Therefore, the first step is to convert the point cloud into a triangular mesh. Common conversion methods include: Delaunay triangulation, which automatically constructs a triangular mesh of the point cloud based on the Delaunay triangulation algorithm; Poisson Surface Reconstruction, which uses the Poisson equation to reconstruct a continuous surface from point cloud data; Alpha Shapes, which generates surfaces of different densities and details by adjusting the Alpha value. Further, the model is optimized. The generated initial mesh model may contain too many faces and irregular shapes, and optimization processing is required to improve the computational efficiency and visual effect of the model, including mesh simplification, which reduces the complexity of the model by reducing the number of faces while maintaining key shape features; smoothing, which applies a smoothing algorithm to eliminate noise and irregular shapes in the mesh and make the model surface smoother; topological optimization, which corrects topological errors in the model, such as open boundaries or overlapping faces. Then, texture mapping is performed, first mapping the texture information in the point cloud or original image to the mesh model to increase the realism of the model; UV mapping projects the surface of the mesh model onto a two-dimensional plane to create UV coordinates for texture mapping; texture mapping uses UV coordinates to apply the color information of the point cloud or image mapping to the model surface.Finally, solid modeling usually requires the help of professional 3D modeling software and point cloud processing plug-ins. Commonly used ones include MeshLab, a free point cloud and mesh processing tool that supports point cloud to mesh conversion, model optimization and texture mapping; Blender, a powerful 3D modeling and animation software that supports point cloud import and solid modeling through plug-ins; 3D Studio Max, a professional 3D modeling software that supports point cloud data import, processing and solid model creation through plug-ins; CloudCompare, a multifunctional point cloud processing software that supports point cloud registration, meshing and solid modeling. Finally, point cloud data from multiple scanning devices and at different times are spatially registered and fused to build a high-precision 3D point cloud model, which is further converted into a solid model.

[0024] Identify and segment the multi-scale features of pre-processed point cloud data based on a multi-category geometric feature fusion model;

[0025] Specifically, first, the input point cloud data is preprocessed to remove noise and outliers, and statistical filtering or conditional filtering techniques are used to remove abnormal points in the point cloud and remove noise in the point cloud to improve data quality and processing accuracy. Voxel grid downsampling is used to reduce the number of points in the point cloud, thereby reducing computational complexity and improving processing speed. The ICP (Iterative Closest Point) algorithm is used for point cloud alignment and registration to align point cloud data from different sources or angles so that they can be represented in a unified coordinate system. Then, geometric feature extraction is performed, and the local surface direction of each point is extracted based on the normal calculation of the neighborhood of each point to describe the local geometric structure. The curvature is calculated by analyzing the position distribution of the neighborhood points to describe the degree of surface curvature near each point and reflect the local geometric morphology. The change in the normal direction or the gradient method is used to detect edges and identify the edges and contours of objects in the point cloud, which is helpful for segmenting different areas or objects. Shape descriptors (such as Spin Image or Shape) are used. Context) to extract shape features and obtain the shape information of the object, which can be a local or global shape description; finally, in the neighborhood of each point, by analyzing the local geometric morphology to extract features, capture the small-scale geometric information with rich details in the point cloud data, use the global feature extraction method (such as global average pooling or PointNet) to generate a global feature representation, obtain the global geometric structure information of the entire point cloud data, and fuse the local and global features through multi-scale convolution or pyramid pooling technology, fuse the features of different scales, and combine the local and global information to form a richer feature representation. Finally, feature fusion and enhancement are performed, and multiple geometric features (such as normals, curvatures, edges, shapes, etc.) and multi-scale features are fused to form a unified feature representation through weighted summation, feature concatenation, or fusion based on the attention mechanism. The original information of the feature is retained through residual connection, and the weights of different features are dynamically adjusted by the hybrid attention module to enhance important features. The residual structure and hybrid attention module are used to further enhance the representation ability of key features. Use deep learning models (such as MLP, CNN, etc.) to classify features, classify and identify each point in the point cloud based on the fused multi-class geometric features, and mark different objects or regions. Use clustering algorithms, conditional random fields (CRF), graph-based segmentation methods, etc. to segment point cloud data by category. According to the recognition results, perform semantic segmentation or instance segmentation on different parts of the point cloud to separate various objects or regions. This can effectively improve the accuracy and robustness of point cloud registration and is suitable for a variety of application scenarios such as 3D reconstruction, object recognition, and robot navigation. Multi-scale feature fusion aims to extract features of point cloud data at different scales to capture multi-level information from local details to global structures.This is usually achieved by building a hierarchical point cloud representation, for example, using PointNet++ or similar network architectures, which can perform point sampling and feature aggregation at different scales to obtain richer feature representations. Secondly, the residual structure allows the network to retain the input of the previous layer during learning, which helps alleviate the gradient vanishing problem and enables the network to learn deeper features. The hybrid attention module combines spatial attention and channel attention, which can automatically focus on key areas and features in the point cloud and enhance the network's ability to extract effective features. This module is usually designed as a pluggable component and can be embedded in the feature extraction network to improve the performance of the model. Furthermore, in the point cloud registration task, Transformer can be used to capture long-range dependencies in the point cloud and learn the relationship between points through the self-attention mechanism. Soft pointers are used as a flexible pointer network to predict the transformation parameters of the point cloud. By learning a continuous weight distribution to determine how to map the input point cloud to the output point cloud, rigid body registration of the point cloud is achieved. Then, in point cloud registration, singular value decomposition SVD is often used to solve rigid transformations, namely rotation and translation. However, the traditional SVD algorithm is non-differentiable, which limits its direct application in deep learning frameworks. Therefore, a differentiable SVD algorithm is proposed, which can transfer gradients in back-propagation, so that the network can learn the optimal rigid transformation parameters, thereby achieving more accurate point cloud registration.

[0026] A three-dimensional model of a triangular mesh is constructed for the segmented material pile, and the volume of the material pile is calculated by a mesh volume method based on the constructed three-dimensional model;

[0027] Specifically, the collected point cloud data is transformed and spliced ​​into coordinate systems, the material pile is identified based on its geometric features, and then the point cloud is segmented to distinguish different material piles; the point cloud data of each material pile is reconstructed to obtain its triangular mesh model; the volume of the material pile is calculated through the triangular mesh model; the top position of the material pile is determined by analyzing the point cloud data, and the results of segmentation and volume calculation are output. First, a three-dimensional model of the triangular mesh is constructed. Based on the preprocessed point cloud data, a three-dimensional modeling software or plug-in, such as MeshLab, CloudCompare, AutodeskReCap, etc., is used to construct the triangular mesh; point cloud to mesh conversion: Delaunay triangulation, Poisson SurfaceReconstruction and other algorithms are used to convert the point cloud data into a triangular mesh model; mesh optimization, by simplifying, smoothing and other means, optimize the accuracy and computational efficiency of the mesh model; boundary processing, ensure that the boundary of the mesh model coincides with the boundary of the material pile to avoid errors in volume calculation. Secondly, the volume of the material pile is calculated using the grid volume method. The grid volume method is a common method for calculating the volume of irregularly shaped objects. It is based on a triangular mesh model and estimates the total volume by decomposing the model into multiple tiny units and then accumulating the volumes of these units. The specific steps are as follows: unit division, decomposing the triangular mesh into multiple tiny triangular or tetrahedral units; volume calculation, for each unit, calculate its volume. For a triangular unit, it can be regarded as a plane, and its projected area is calculated and multiplied by the average height; for a tetrahedral unit, its volume is calculated directly; cumulative volume, the volumes of all units are accumulated to obtain an estimated value of the total volume of the material pile.

[0028] At predetermined time intervals, the material yard is scanned by 3D laser to obtain multiple sets of point cloud data. The newly acquired point cloud is registered with the historical data to eliminate the differences in scanning positions and angles. By comparing continuous point cloud data, changes in the shape and position of the material pile are identified. According to the changes in the volume and position of the material pile, the inventory data information of the material pile in the material yard is updated in real time.

[0029] Specifically, synchronize the time-series point cloud data set to the historical data network, that is, copy and send it to the historical data; perform comparative analysis to compare the changes in the shape and position of the material pile in the time-series point cloud data set and the historical data; and use it to update the inventory data information of the material pile in the material yard in real time according to the changes in the volume and position of the material pile. First, perform 3D laser scanning and data collection, including regular scanning, using a 3D laser scanner to scan the material yard at predetermined time intervals, such as daily or weekly, to obtain high-precision point cloud data; data recording, ensuring that the data of each scan is accurately recorded and stored, including scanning time, scanning position and angle information, for later data alignment and analysis. Then, point cloud data registration is performed, including feature point extraction, extracting stable feature points from the newly collected point cloud data, such as the edge or ground features of the material pile, which will be used in the subsequent registration process; preliminary registration, using the iterative closest point (ICP) algorithm or other point cloud registration technology, the newly collected point cloud data is preliminarily registered with the historical data to eliminate the influence caused by the difference in scanning position and angle; fine adjustment, by optimizing the registration parameters, such as rotation and translation, the registration results are further adjusted to improve the accuracy of the registration. Then, the identification and analysis of material pile changes include: difference point cloud generation, by comparing continuous point cloud data, generating difference point clouds, intuitively showing the shape and position changes of the material pile; change area detection, identifying the increase, decrease or movement areas of the material pile in the difference point cloud, these areas reflect the actual changes of the material pile; volume change calculation, using the grid volume method or other volume calculation methods, based on the point cloud data of the change area, accurately calculate the change in the volume of the material pile. Finally, the material pile inventory data is updated in real time, including inventory information synchronization, and the calculated material pile volume change and position change information is updated in real time to the material management system to ensure the real-time and accuracy of the inventory data; material flow analysis: combined with the position change and volume change of the material pile, the inflow and outflow of materials are analyzed to provide decision support for material management; reporting and early warning, based on the real-time updated inventory data, a material pile change report is generated, and an early warning is issued when necessary to remind management personnel to pay attention to abnormal changes in materials. Therefore, within a predetermined time interval, a three-dimensional laser scan of the material yard can be performed to obtain and process multiple sets of point cloud data, and the volume and position changes of the material pile can be monitored and updated in real time, thereby realizing accurate management of the material yard material pile inventory data.

[0030] Based on the Internet of Things technology and Geographic Information System (GIS), inventory data and material yard geographic information are acquired in real time and integrated into the genetic algorithm to plan the optimal path for material picking and stacking of material yard equipment in real time.

[0031] Specifically, based on the Internet of Things (IoT) technology and Geographic Information System (GIS), inventory data and material yard geographic information are obtained in real time, and then integrated into the genetic algorithm for optimal path planning. First, real-time data collection and integration, various IoT sensors such as RFID, barcodes, weight sensors, etc. are deployed in the material yard to collect material inventory information, equipment location and status, etc. in real time. Using GIS technology, the geographic information of the material yard is integrated, including spatial data such as material yard layout, road network, material pile location and shape, and the real-time inventory data collected by IoT and the geographic information of GIS are fused and processed to form a unified data platform to provide data support for subsequent path planning. Secondly, the genetic algorithm optimizes the path planning, and the material picking and stacking path planning problems of the material yard equipment are modeled as optimization problems of the genetic algorithm. Each "chromosome" represents a possible path plan, which consists of a series of operation instructions, such as material picking points, material stacking points, driving paths, etc., and the fitness function is designed to evaluate the pros and cons of the path plan. The fitness function may be based on multiple factors such as path length, time consumption, equipment utilization, material handling efficiency, etc. Through genetic operations such as selection, crossover and mutation, the path planning scheme is iteratively optimized to gradually approach the optimal solution. Finally, real-time path planning and adjustment, based on real-time inventory data and material yard geographic information, dynamically adjust the path planning to ensure that the equipment can respond to the latest material demand and material yard conditions, and send the optimized path plan to the material yard equipment to guide the equipment to perform material picking and stacking operations. At the same time, feedback data on equipment execution is collected for the optimization of the next round of path planning. During the path execution process, if abnormal situations such as equipment failure and material shortage are encountered, the path planning can be quickly adjusted to ensure the continuity and stability of material handling. Inventory data and material yard geographic information are obtained in real time through Internet of Things technology and geographic information system, and then integrated into the genetic algorithm for real-time path optimization to achieve efficient material picking and stacking of material yard equipment.

[0032] Among them, this embodiment can solve the technical problems in the prior art that manual management leads to low data accuracy, recording and updating data is time-consuming and labor-intensive, it is difficult to achieve real-time data updates, and even there are safety hazards. The goal of improving data accuracy can be achieved, and the technical effect of quickly updating material yard data and reducing safety risks can be achieved.

[0033] The method provided in the embodiment of the present disclosure also includes:

[0034] Extract feature points from point cloud data, including corner points, edges, textures, etc., for subsequent matching process;

[0035] The local geometric shape around the point is expressed by the FPFH feature descriptor, so that the point cloud registration feature matching can be preliminarily aligned. The local correlation of feature information can be established by combining the FPFH feature descriptor and Delaunay triangulation, which can then be used for matching feature point pairs.

[0036] Calculate the initial transformation matrix between the two point clouds based on the matched point pairs;

[0037] Iterative optimization is performed using the iterative closest point (ICP) algorithm.

[0038] Furthermore, an iterative closest point ICP algorithm is used for iterative optimization, and the iterative optimization includes:

[0039] S1: Find the nearest corresponding point for each point in the transformed point cloud;

[0040] S2: Use the closest point pair to calculate the rotation and translation vectors, usually solved by the singular value decomposition SVD linear algebra method;

[0041] S3: Apply the calculated transformation matrix to the source point cloud to update the position of the source point cloud;

[0042] S4: Check whether the change of the transformation matrix is ​​lower than the preset threshold. If the termination condition is met, the algorithm stops iterating; if not, return to S2 to continue the iteration process.

[0043] Specifically, the local geometry around the point is expressed by the FPFH feature descriptor, so that the point cloud registration feature matching can be preliminarily aligned. The combination of the FPFH feature descriptor and Delaunay triangulation can establish the local correlation of feature information, which can then be used to match feature point pairs. FPFH includes the following steps: Establish a kd tree: First, the point cloud data is constructed into a kd tree to quickly search for the nearest neighbor points; calculate the normal, for each point, by using the nearest neighbor point to calculate its normal direction. This can be achieved by calculating the eigenvalues ​​and eigenvectors of the covariance matrix of the eigenvector; create an FPFH descriptor, for each point, select its nearest neighbor points, and calculate the relative position and normal difference between them. Then, use this information to calculate the FPFH descriptor, which is represented by a histogram; normalization: Finally, for each point, its FPFH descriptor is normalized to eliminate the effects of scale and rotation; the result of FPFH feature extraction is a descriptor with a fixed length, which is used to represent the local geometric features in the point cloud data. These descriptors can be used in applications such as point cloud registration, target recognition, and three-dimensional reconstruction.

[0044] Furthermore, recognizing and segmenting the multi-scale features of the pre-processed point cloud data based on the multi-class geometric feature fusion model also includes:

[0045] The collected point cloud data is transformed and spliced ​​into coordinate system, the material pile is identified based on its geometric features, and then the point cloud is segmented to distinguish different material piles;

[0046] Perform surface reconstruction on the point cloud data of each stockpile to obtain its triangular mesh model;

[0047] Calculate the volume of the material pile through the triangular mesh model;

[0048] The top position of the stockpile is determined by analyzing the point cloud data, and the results of the segmentation and volume calculation are output.

[0049] Furthermore, the multi-category geometric feature fusion model also includes:

[0050] The multiple types of geometric features include: normal features, curvature features, edge features, and shape features;

[0051] PCA is used to calculate the normal direction of each point and extract the normal features. The curvature of the point is calculated by analyzing the distribution of the neighborhood points and the curvature features are extracted. Based on the change of the normal direction, the contour information of the object is captured and the edge features are extracted. The shape descriptor is used to describe the overall or local shape of the object and extract the shape features.

[0052] Extract local features through local neighborhood analysis, use global pooling layer to extract global features, capture overall shape information, and use multi-scale convolution technology to fuse multi-scale features;

[0053] Use the fused features to classify each point and use a deep learning classifier for identification;

[0054] According to the recognition results, semantic segmentation is used to divide the point cloud data into different categories;

[0055] Use the trained model to segment the point cloud and assign each point in the point cloud dataset to different categories to achieve the recognition and segmentation of the material pile;

[0056] Use the labeled point cloud dataset to train the segmentation model, use the test dataset to evaluate the performance of the model, and optimize the segmentation effect.

[0057] Furthermore, the top position of the pile is determined by analyzing the point cloud data, and the segmentation and volume calculation results are output, which also includes:

[0058] Determine the boundary of the stockpile by performing convex hull analysis on the point cloud data, wherein the convex hull analysis is to calculate the convex hull using the ConvexHull function;

[0059] After obtaining the boundary of the stockpile, calculate the height of each point relative to the ground plane;

[0060] Determine the highest point of the pile by comparing the heights of all points;

[0061] Use height threshold to segment the point cloud and retain points in the stockpile area;

[0062] The stockpile is segmented using a clustering algorithm to separate the stockpile from the background, and the clustering algorithm is Clusters, thereby obtaining multiple stockpile areas;

[0063] The point cloud volume is calculated by voxelization, and the formula is:

[0064]

[0065] in:

[0066] V is the volume of the stockpile;

[0067] vk is the volume of the kth voxel;

[0068] ck is the point cloud density of the kth voxel;

[0069] Determine the top of the pile, segment the pile area, and output the volume calculation results.

[0070] Determine the top position of the pile by analyzing the point cloud data, output the results of segmentation and volume calculation, and load the point cloud data: First, load the point cloud data into the computing environment. Point cloud data can be a file containing three-dimensional point coordinates, such as XYZ, PLY or LAS format; Denoising and filtering, for the original point cloud data, denoising and filtering algorithms can be applied to eliminate noise and unnecessary points. Common methods include statistical filtering, radius filtering and voxel filtering; point cloud segmentation, use segmentation algorithms to separate the pile part from the background part in the point cloud data. Common methods include clustering-based segmentation such as DBSCAN, MeanShift and plane model-based segmentation, such as RANSAC, Region Growing; Extract the top of the pile, for the segmented pile part, determine the top position of the pile by analyzing the features of the point cloud data. You can consider using indicators such as normal information, point cloud density or curvature to find the set of points on the top of the pile; calculate the volume of the pile: according to the height information of the top of the pile and the pile area obtained by segmentation, use the volume calculation method described in method 1 to calculate the volume of the pile; output the results of segmentation and volume calculation, which can be saved as a file or displayed in a visual form.

[0071] Furthermore, solid modeling also includes:

[0072] Slice the point cloud data in a specific direction to obtain the cross-sectional information of the object, and identify the geometric shape of the object through the cross-sectional information, which includes the internal structure and shape characteristics;

[0073] Use edge detection algorithms to identify mutation points in point clouds to determine the boundaries of objects, and extract object edges and contours from point cloud data;

[0074] Match features in point cloud data with known geometric features through pattern recognition algorithms;

[0075] Refine, align and merge features to ensure model continuity and integrity, creating a 3D entity with clear boundaries, accurate dimensions and geometric features.

[0076] Furthermore, based on the Internet of Things technology and the Geographic Information System (GIS), the inventory data and the geographic information of the material field are acquired in real time and integrated into the genetic algorithm. The optimal path method for real-time planning of the material picking and stacking paths of the material field equipment also includes:

[0077] The integrated GIS system integrates the geographic information of the material yard, including topography, road network, material pile distribution and equipment location;

[0078] Integrate IoT data with GIS information, pre-process the data, and provide accurate input for genetic algorithms;

[0079] Based on real-time updated inventory data and material yard geographic information, the optimal path of the equipment is dynamically planned using genetic algorithms;

[0080] The optimized path plan is sent to the material yard equipment, the execution status and environmental changes of the equipment are monitored in real time, and the path is dynamically adjusted when necessary.

[0081] Specifically, meaningful features are extracted from the point cloud data, such as the coordinates, normals, curvatures, colors, etc. of the points. Methods such as FPS can be used to select key points or to extract information from local areas; the extracted features are converted into descriptors that can represent geometric features. Common methods include SHOT, FPFH, etc.; the features of the point cloud data to be identified are matched with known geometric features. Some matching algorithms are used, such as nearest neighbor search, RANSAC, etc.; for each matching result, its similarity measure is calculated, such as distance, similarity score, etc. By setting a threshold, it is determined whether the match is valid; based on the matching results and evaluation, the matching of the features in the point cloud data with the known geometric features is determined. This can be a binary classification problem (match / mismatch) or a multi-category problem (multiple geometric features); based on the matching results and optimization measures, the process of feature extraction, description, matching, and pattern recognition is repeated to gradually optimize the matching accuracy of the model.

[0082] According to the goal, such as minimizing time cost or maximizing efficiency, design a fitness function to evaluate the quality of the path; define crossover and mutation operations to generate new paths; crossover can be done by exchanging some path segments, and mutation can introduce changes by randomly exchanging points or adding / deleting points in the path; select individuals in the population according to the fitness function, and select individuals with higher fitness as parents. Perform crossover and mutation operations on the selected parent individuals to generate new offspring individuals; update the population: update the original population through replacement operations, so that the new offspring individuals enter the next generation population; termination conditions: set termination conditions, such as reaching the maximum number of iterations or fitness.

[0083] Embodiment 2

[0084] Based on the same inventive concept as the material field data management method based on fusion point cloud in the above-mentioned embodiment, Figure 2 As shown, the present disclosure also provides a material field data management system based on fused point cloud, the system comprising:

[0085] The scanning equipment module 11 is used to deploy high-precision three-dimensional scanning equipment in the target monitoring area, and the high-precision three-dimensional scanning equipment is used to collect point cloud data of the material field;

[0086] Point cloud data preprocessing module 12, point cloud data preprocessing module 12 is used to preprocess the collected point cloud data, using PointNet++ deep learning network to extract multi-scale features of point cloud data, preprocessing includes filtering, noise removal, data compression and classification;

[0087] The entity modeling module 13 is used to spatially register point cloud data acquired by multiple scanning devices at different times, stitch together the point cloud data, build a complete three-dimensional point cloud model, and use modeling software and point cloud plug-ins to perform entity modeling;

[0088] A multi-scale feature recognition and segmentation module 14, which is used to recognize and segment the multi-scale features of the pre-processed point cloud data based on a multi-type geometric feature fusion model;

[0089] The material pile volume calculation module 15 is used to construct a three-dimensional model of a triangular mesh for the segmented material pile, and calculate the volume of the material pile by a mesh volume method based on the constructed three-dimensional model;

[0090] The material pile inventory data information updating module 16 is used to perform three-dimensional laser scanning on the material yard within a predetermined time interval to obtain multiple sets of point cloud data, align the newly acquired point cloud with the historical data to eliminate the difference in scanning position and angle, and identify the changes in the shape and position of the material pile by comparing the continuous point cloud data. According to the changes in the volume and position of the material pile, the material pile inventory data information of the material yard is updated in real time;

[0091] The path planning module 17 is used to obtain inventory data and material field geographic information in real time based on the Internet of Things technology and geographic information system (GIS), integrate them into the genetic algorithm, and plan the optimal path for material collection and stacking of material field equipment in real time.

[0092] The specific example of the material field data management method based on fused point cloud in the aforementioned embodiment 1 is also applicable to the material field data management system based on fused point cloud in this embodiment. Through the detailed description of the aforementioned material field data management method based on fused point cloud, those skilled in the art can clearly know the material field data management system based on fused point cloud in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0093] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of this disclosure can be achieved, and this document is not limited here.

[0094] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. Material field data management method based on fusion point cloud, It is characterized in that Methods include: High-precision 3D scanning equipment is deployed in the target monitoring area to collect point cloud data of the material yard; Preprocess the collected point cloud data and use the PointNet++ deep learning network to extract multi-scale features of the point cloud data. The preprocessing includes filtering, noise removal, data compression and classification. Perform spatial registration on point cloud data acquired from multiple scanning devices at different times, stitch together the point cloud data, build a complete 3D point cloud model, and use modeling software and point cloud plug-ins for solid modeling; Identify and segment the multi-scale features of pre-processed point cloud data based on a multi-category geometric feature fusion model; A three-dimensional model of a triangular mesh is constructed for the segmented material pile, and the volume of the material pile is calculated by a mesh volume method based on the constructed three-dimensional model; At predetermined time intervals, the material yard is scanned by 3D laser to obtain multiple sets of point cloud data. The newly acquired point cloud is registered with the historical data to eliminate the differences in scanning positions and angles. By comparing continuous point cloud data, changes in the shape and position of the material pile are identified. According to the changes in the volume and position of the material pile, the inventory data information of the material pile in the material yard is updated in real time. Based on the Internet of Things technology and Geographic Information System (GIS), inventory data and material yard geographic information are acquired in real time and integrated into the genetic algorithm to plan the optimal path for material picking and stacking of material yard equipment in real time.

2. The method according to claim 1, It is characterized in that Point cloud registration algorithms also include: Extract feature points from point cloud data, including corner points, edges, textures, etc., for subsequent matching process; The local geometric shape around the point is expressed by the FPFH feature descriptor, so that the point cloud registration feature matching can be preliminarily aligned. The local correlation of feature information can be established by combining the FPFH feature descriptor and Delaunay triangulation, which can then be used for matching feature point pairs. Calculate the initial transformation matrix between the two point clouds based on the matched point pairs; Iterative optimization is performed using the iterative closest point (ICP) algorithm.

3. The method according to claim 2, It is characterized in that The iterative closest point ICP algorithm also includes: S1: Find the nearest corresponding point for each point in the transformed point cloud; S2: Use the closest point pair to calculate the rotation and translation vectors, usually solved by the singular value decomposition SVD linear algebra method; S3: Apply the calculated transformation matrix to the source point cloud to update the position of the source point cloud; S4: Check whether the change of the transformation matrix is ​​lower than the preset threshold. If the termination condition is met, the algorithm stops iterating; if not, return to S2 to continue the iteration process.

4. The method according to claim 1, It is characterized in that The identifying and segmenting of multi-scale features of the pre-processed point cloud data based on the multi-class geometric feature fusion model also includes: The collected point cloud data is transformed and spliced ​​into coordinate system, the material pile is identified based on its geometric features, and then the point cloud is segmented to distinguish different material piles; Perform surface reconstruction on the point cloud data of each stockpile to obtain its triangular mesh model; Calculate the volume of the material pile through the triangular mesh model; The top position of the stockpile is determined by analyzing the point cloud data, and the results of the segmentation and volume calculation are output.

5. The method as claimed in claim 4, It is characterized in that The multi-category geometric feature fusion model also includes: The multiple types of geometric features include: normal features, curvature features, edge features, and shape features; PCA is used to calculate the normal direction of each point and extract the normal features. The curvature of the point is calculated by analyzing the distribution of the neighborhood points and the curvature features are extracted. Based on the change of the normal direction, the contour information of the object is captured and the edge features are extracted. The shape descriptor is used to describe the overall or local shape of the object and extract the shape features. Extract local features through local neighborhood analysis, use global pooling layer to extract global features, capture overall shape information, and use multi-scale convolution technology to fuse multi-scale features; Use the fused features to classify each point and use a deep learning classifier for identification; According to the recognition results, semantic segmentation is used to divide the point cloud data into different categories; Use the trained model to segment the point cloud and assign each point in the point cloud dataset to different categories to achieve the recognition and segmentation of the material pile; Use the labeled point cloud dataset to train the segmentation model, use the test dataset to evaluate the performance of the model, and optimize the segmentation effect.

6. The method of claim 4, It is characterized in that The top position of the pile is determined by analyzing the point cloud data, and the segmentation and volume calculation results are also output including: Determine the boundary of the stockpile by performing convex hull analysis on the point cloud data, wherein the convex hull analysis is to calculate the convex hull using the ConvexHull function; After obtaining the boundary of the stockpile, calculate the height of each point relative to the ground plane; Determine the highest point of the pile by comparing the heights of all points; Use height threshold to segment the point cloud and retain points in the stockpile area; The stockpile is segmented using a clustering algorithm to separate the stockpile from the background, and the clustering algorithm is Clusters, thereby obtaining multiple stockpile areas; The point cloud volume is calculated by voxelization, and the formula is: in: V is the volume of the stockpile; vk is the volume of the kth voxel; ck is the point cloud density of the kth voxel; Determine the top of the pile, segment the pile area, and output the volume calculation results.

7. The method of claim 1, It is characterized in that Solid modeling also includes: Slice the point cloud data in a specific direction to obtain the cross-sectional information of the object, and identify the geometric shape of the object through the cross-sectional information, which includes the internal structure and shape characteristics; Use edge detection algorithms to identify mutation points in point clouds to determine the boundaries of objects, and extract object edges and contours from point cloud data; Match features in point cloud data with known geometric features through pattern recognition algorithms; Refine, align and merge features to ensure model continuity and integrity, creating a 3D entity with clear boundaries, accurate dimensions and geometric features.

8. The method of claim 1, It is characterized in that Based on the Internet of Things technology and geographic information system GIS, real-time inventory data and material field geographic information are integrated into the genetic algorithm. The optimal path method for real-time planning of material picking and stacking paths for material field equipment also includes: The integrated GIS system integrates the geographic information of the material yard, including topography, road network, material pile distribution and equipment location; Integrate IoT data with GIS information, pre-process the data, and provide accurate input for genetic algorithms; Based on real-time updated inventory data and material yard geographic information, the optimal path of the equipment is dynamically planned using genetic algorithms; The optimized path plan is sent to the material yard equipment, the execution status and environmental changes of the equipment are monitored in real time, and the path is dynamically adjusted when necessary.

9. Material field data management system based on fusion point cloud, It is characterized in that A material field data management method based on fused point cloud for implementing any one of claims 1 to 8, the system comprising: The scanning equipment module is used to deploy high-precision three-dimensional scanning equipment in the target monitoring area, and the high-precision three-dimensional scanning equipment is used to collect point cloud data of the material field; Point cloud data preprocessing module: The point cloud data preprocessing module is used to preprocess the collected point cloud data. The PointNet++ deep learning network is used to extract multi-scale features of the point cloud data. The preprocessing includes filtering, noise removal, data compression and classification. Solid modeling module: The solid modeling module is used to spatially register point cloud data acquired by multiple scanning devices at different times, stitch together the point cloud data, build a complete three-dimensional point cloud model, and use modeling software and point cloud plug-ins for solid modeling; Multi-scale feature recognition and segmentation module: The multi-scale feature recognition and segmentation module is used to recognize and segment the multi-scale features of the pre-processed point cloud data based on a multi-type geometric feature fusion model; A material pile volume calculation module is used to construct a three-dimensional model of a triangular mesh for the segmented material pile, and calculate the volume of the material pile by a mesh volume method based on the constructed three-dimensional model; The material pile inventory data information update module is used to perform three-dimensional laser scanning on the material yard within a predetermined time interval to obtain multiple sets of point cloud data, align the newly collected point cloud with the historical data to eliminate the difference in scanning position and angle, and identify the changes in the shape and position of the material pile by comparing the continuous point cloud data. According to the changes in the shape and position of the material pile, the material pile inventory data information of the material yard is updated in real time; Path planning module: The path planning module is used to obtain inventory data and material yard geographic information in real time based on the Internet of Things technology and geographic information system (GIS), integrate it into the genetic algorithm, and plan the optimal path for the material picking and stacking of the material yard equipment in real time.

Images