A control method and system based on three-dimensional real-time model of stockpile stratification
Through the lidar, the three-dimensional point cloud data of the material stack is collected, combined with adaptive grid division and BiLSTM network, the material stack feature category is determined and the stack pickup control strategy is matched, which solves the problem of insufficient reflection of the material stack morphological characteristics in the existing technology. Improves the accuracy and reliability of the stacker.
Patent Information
- Application Number
- CN202111301531.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-04
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-11-04
AI Technical Summary
The prior art cannot effectively reflect the morphological characteristics of the material pile, resulting in insufficient matching and fine control methods of the stacking and collecting equipment.
LiDAR is used to collect three-dimensional point cloud data from each layer of the material stack in real time, and extract layer feature vectors through adaptive mesh division and BiLSTM network to determine the feature category of the current layer of the material stack, and match the control strategy of the stacker according to the feature category. .
The subdivision and classification of the working faces of each layer of the material stack is realized, the reliability and accuracy of the stacking and material collection operation is improved, and different characteristics are adapted to ensure the smooth and safe operation of the stack and material collection.
Smart Images

Figure CN114022709B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of intelligent equipment for coal material yards, and in particular relates to a control method and system based on a three-dimensional real-time model of material pile stratification. Background Art
[0002] A coal yard is a large space for stacking coal and is an essential part of thermal power plants and other units. A coal yard generally requires large facilities such as bucket wheel stackers and reclaimers to stack and reclaim coal.
[0003] In recent years, with the in-depth application of digital and intelligent technologies in traditional industries, an important research and development direction in this field is to use bucket wheel stacker and reclaimer as the implementation object, adopt positioning technology, control technology, laser scanning technology, three-dimensional graphics processing and information communication and other technical means to develop unmanned drive control system, realize the intelligent transformation of material yard stacker and reclaimer equipment, and realize unmanned operation and other functions.
[0004] A basic link in the intelligent transformation of stacker-reclaimers is to realize accurate detection and analysis of the material pile, including detecting the material level status of the working surface during stacking and reclaiming, and feeding the data back to the control system of the stacker-reclaimer in real time. The control system will perform data analysis to guide the next action of the bucket wheel excavator.
[0005] In the prior art, the detection of material level status can be achieved by using mechanical or remote sensing detection mechanisms. For example, the patent "A high material level detection device for automatic operation of a circular stacker and reclaimer" with publication number CN211971064U uses an induction switch to detect high material level. The patent "A material level height detection system for a stacker and reclaimer" with publication number CN210763220U uses a radar wave sensor in a protective cover to detect the material level height. However, the above prior art has a single function and can only achieve basic material level height detection. When applied to stacker and reclaimer equipment, it can only perform simple start-stop control.
[0006] In the prior art, there are also methods of collecting point cloud data of a stockpile using laser radar, or obtaining stockpile images by means of video camera, and then processing and integrating to form a stockpile model to reflect the height, volume, distribution and other states of the stockpile. For example, the patent application with publication number CN103292715A discloses a stockpile height detection device for a quayside-type ship loader throwing position, which includes a laser radar array, a radar motion mechanism and an industrial computer connected by a circuit. The patent application with publication number CN113320995A discloses an unmanned control system for a stacker-reclaimer in a raw material field, in which a three-dimensional laser scanner is used to obtain a three-dimensional model of the raw material field, and then the stacker-reclaimer positioning data, including the travel position, rotation angle and pitch angle, are obtained according to the three-dimensional model of the stockpile in the raw material field and the stacker-reclaimer strategy. However, the stockpile model of the above prior art is only to directly characterize and describe the height, volume, position distribution and the like of the stockpile, and cannot reflect the morphological characteristics of the entire stockpile, nor can it realize a more matching and more sophisticated stacker-reclaimer equipment control method based on the morphological characteristics of the stockpile. Summary of the invention
[0007] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is: how to provide a control method and system based on a three-dimensional real-time model of material pile stratification, so as to establish a three-dimensional real-time model of material pile stratification, reflect the morphological feature types of the working surfaces of each layer of the material pile based on the model, and then realize a more matched and more refined stacker and reclaimer equipment control method based on the morphological feature types of the material pile.
[0008] The present invention adopts the following technical solution:
[0009] The present invention discloses a control method based on a three-dimensional real-time model of a stockpile layer, which is characterized by comprising the following steps:
[0010] During the material stacking or material removal process, the laser radar is used to collect the 3D point cloud data of each layer of the material stack in real time;
[0011] Based on the point cloud spatial coordinate data of the working surface of each layer of the stockpile, adaptive meshing is performed to obtain the three-dimensional mesh of the working surface of each layer of the stockpile;
[0012] The three-dimensional grids contained in each layer of the working surface of the stockpile are represented as a layer entity set;
[0013] The BiLSTM network is used to extract the features of the layer entity set of each layer working surface of the pile to obtain the pile layer feature vector;
[0014] Based on the prototype network model trained with the support set, the feature category of the current layer of the pile is obtained according to the layer feature vector of the current layer of the pile;
[0015] According to the characteristic category of the working surface of the current layer of the stockpile, the control strategy for the stacker-reclaimer to execute the action is matched to realize the control of the stacker-reclaimer to execute the action.
[0016] Preferably, the specific steps of performing adaptive meshing for the three-dimensional point cloud spatial coordinate data of the working surface of each layer of the stockpile include:
[0017] The spatial coordinate information of the three-dimensional point cloud of each working surface is divided into initial grids. The initial grid is a polygon formed by connecting the feature points of the point cloud of each working surface as grid vertices;
[0018] Regularize the initial grid;
[0019] For the initial grid after regularization, adaptive grid subdivision is performed according to the grid size and grid quality index.
[0020] Preferably, mesh vertices are determined based on a Delaunay triangulated mesh algorithm to form an initial triangular mesh.
[0021] Preferably, the regularization of the initial mesh includes: supplementing the missing edges of the missing edge polygons, inserting new mesh vertices at the intersection points of the edges of the polygons, and merging the edges of the overlapping polygons;
[0022] Preferably, the BiLSTM network is used to extract features from the layer entity set of each layer of the working surface of the stockpile to obtain the stockpile layer feature vector, which specifically includes: by inputting the layer entity set of each layer of the working surface of the stockpile as an input into the BiLSTM network, the layer feature vector of the current layer of the stockpile can be obtained, which is expressed as X m ,Right now
[0023] X m =f BiLSTM (..., l′2, l′1, m′1, m′2, h′1, h′2,...)
[0024] where f BiLSTM (*) indicates that the layer entity set is input into BiLSTM for feature extraction, thereby mapping the layer entity combination into a pile layer feature vector.
[0025] Preferably, the prototype network calculates the layer feature vector X of the current layer of the input stockpile m The distribution of each category k relative to the K categories:
[0026]
[0027] Where k′ represents other categories among the K categories that do not belong to category k. Thus, the prototype network can be based on the layer feature vector X of the current layer of the pile mWith respect to the distribution of each category k among the K categories, the characteristic category of the current layer of the pile is determined.
[0028] The present invention provides a control system based on a three-dimensional real-time model of a stockpile layer, which is characterized by comprising:
[0029] The 3D point cloud real-time acquisition module is used to collect the 3D point cloud data of each layer of the material pile in real time using the laser radar during the material pile or material removal process;
[0030] The material pile working surface meshing module performs adaptive meshing based on the point cloud spatial coordinate data of each layer of the material pile working surface to obtain the three-dimensional mesh of each layer of the material pile working surface;
[0031] A three-dimensional grid representation processing module is used to represent the three-dimensional grids contained in each layer of the working surface of the stockpile as a layer entity set;
[0032] The stockpile layer feature extraction module is used to extract features from the layer entity set of each layer working surface of the stockpile using the BiLSTM network to obtain the stockpile layer feature vector;
[0033] The stockpile layer feature category classification module obtains the feature category of the current layer of the stockpile based on the prototype network model trained with the support set and the layer feature vector of the current layer of the stockpile;
[0034] The action control strategy module is used to match the control strategy of the stacker-reclaimer's execution of actions according to the feature category of the current layer working surface of the material pile, so as to realize the control of the stacker-reclaimer's execution of actions.
[0035] Preferably, the material pile working surface meshing module performs the specific steps of performing adaptive meshing on the three-dimensional point cloud spatial coordinate data of each layer of the material pile working surface, including:
[0036] The spatial coordinate information of the three-dimensional point cloud of each working surface is divided into initial grids. The initial grid is a polygon formed by connecting the feature points of the point cloud of each working surface as grid vertices;
[0037] Regularize the initial grid;
[0038] For the initial grid after regularization, adaptive grid subdivision is performed according to the grid size and grid quality index.
[0039] Preferably, the stockpile working surface meshing module determines mesh vertices based on a Delaunay triangulated mesh algorithm to form a triangular initial mesh.
[0040] Preferably, the regularization processing of the initial grid by the stockpile working surface grid division module includes: supplementing the missing edges of the missing edge polygons, inserting new grid vertices at the intersection points of the polygon edges, and merging the edges of the overlapping polygons;
[0041] Preferably, the stockpile layer feature extraction module uses a BiLSTM network to extract features from a layer entity set of each layer of the stockpile working surface to obtain a stockpile layer feature vector, specifically comprising: by inputting the layer entity set of each layer of the stockpile working surface into the BiLSTM network as an input, a layer feature vector of the current layer of the stockpile can be obtained, expressed as X m ,Right now
[0042] X m =f BiLSTM (..., l′2, l′1, m′1, m′2, h′1, h′2,...)
[0043] where f BiLSTM (*) indicates that the layer entity set is input into BiLSTM for feature extraction, thereby mapping the layer entity combination into a pile layer feature vector.
[0044] Preferably, the prototype network of the stockpile layer feature category classification module calculates the layer feature vector X of the current layer of the input stockpile m The distribution of each category k relative to the K categories:
[0045]
[0046] Where k′ represents other categories among the K categories that do not belong to category k. Thus, the prototype network can be based on the layer feature vector X of the current layer of the pile m With respect to the distribution of each category k among the K categories, the characteristic category of the current layer of the pile is determined.
[0047] Beneficial effects of the present invention:
[0048] The present invention can establish a three-dimensional real-time model of the material pile layering based on the point cloud spatial coordinate data of the working surfaces of each layer of the material pile. The model can reflect the subdivision and classification of the spatial characteristics of the working surfaces of each layer of the material pile during the material stacking and retrieving process, and then a more matched and more refined stacker-reclaimer equipment control method can be executed according to the subdivision and classification of the spatial characteristics of the working surfaces of each layer of the material pile, thereby improving the reliability and accuracy of the stacker-reclaimer operation, being able to adapt to the different characteristics of the working surfaces of the material pile, and ensuring the smooth and safe execution of the stacking and retrieving actions. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 This is a general flow chart of the three-dimensional real-time model control method based on the stockpile stratification of the present invention.
[0050] Figure 2 This is a flow chart of the steps of adaptively dividing each layer of a pile into three-dimensional grids according to the present invention.
[0051] Figure 3 This is a framework diagram of the control system based on the three-dimensional real-time model of the stockpile layering of the present invention. DETAILED DESCRIPTION
[0052] The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0053] The present invention provides a control method based on a three-dimensional real-time model of a stockpile layer, and the specific process is as follows: Figure 1 As shown, including:
[0054] S100: During the material stacking or material removal process, the laser radar is used to collect the three-dimensional point cloud data of each layer of the material stack in real time.
[0055] Install a laser radar at an appropriate position on the bucket wheel stacker and reclaimer. When the stacker and reclaimer is stacking or reclaiming materials, the working surface rises or falls layer by layer. Therefore, the laser radar can be used to perform a three-dimensional scan on each layer of the working surface of the material pile, realizing the real-time collection of point cloud spatial coordinate data of each layer of the working surface of the material pile.
[0056] S200: Based on the point cloud spatial coordinate data of the working surface of each layer of the material pile, adaptive meshing is performed to obtain a three-dimensional mesh of the working surface of each layer of the material pile.
[0057] Specifically, the specific process of performing adaptive meshing for the three-dimensional point cloud spatial coordinate data of each layer of the material pile is as follows: Figure 2 As shown, including:
[0058] S200A, performing initial mesh division on the spatial coordinate information of the three-dimensional point cloud of each layer of the working surface; the initial mesh is a polygon formed by connecting the feature points of the point cloud of each layer of the working surface as mesh vertices, for example, the mesh vertices can be determined based on the Delaunay triangulation algorithm to form a triangular initial mesh;
[0059] S200B, performing regularization processing on the initial mesh, including supplementing the missing edges of the missing edge polygons, inserting new mesh vertices at the intersection points of the edges of the polygons, and merging the edges of the overlapping polygons;
[0060] S200C, for the initial mesh after the regularization processing, adaptive mesh subdivision is performed according to the mesh size and mesh quality index; specifically, firstly, the meshes to be subdivided in the initial mesh after the regularization processing that need to be further adaptively subdivided are locked, and the meshes to be subdivided are meshes in the initial mesh whose any side length is greater than the set side length threshold, and any internal angle of the polygon is less than the angle threshold; then the center of the circumscribed circle of the mesh to be subdivided is inserted into the mesh to be subdivided as an inserted mesh vertex, and then the inserted mesh vertex and other mesh vertices of the mesh to be subdivided are connected to realize adaptive mesh subdivision; repeat the above steps until all the meshes to be subdivided are adaptively subdivided.
[0061] S300: The three-dimensional grids included in each layer of the working surface of the stockpile are characterized as a layer entity set.
[0062] Specifically, for each layer of the working surface of the material pile, the three-dimensional grid finally divided in step S200 is used as the basic unit, and it is represented as a layer entity x={..., l'2, l'1, m'1, m'2, h'1, h'2, ...}, where m={m'1, m'2, ...} represents the layer entity set of the current layer of the working surface of the material pile, and each three-dimensional grid of the current layer of the working surface of the material pile is mapped to the layer entities m'1, m'2, ... in the set according to its spatial position. l'={..., l'2, l'1} and h'={h'1, h'2, ...} respectively represent the layer entity sets of the lower adjacent layer and the upper adjacent layer of the current layer of the material pile.
[0063] S400: Using the BiLSTM network, feature extraction is performed on the layer entity set of each layer working surface of the stockpile to obtain a stockpile layer feature vector.
[0064] Specifically, by inputting the layer entity set of each layer of the working surface of the stockpile into the BiLSTM network as the input, the layer feature vector of the current layer of the stockpile can be obtained, which is expressed as X m ,Right now
[0065] X m =f BiLSTM (..., l′2, l′1, m′1, m′2, h′1, h′2,...)
[0066] where f BiLSTM (*) indicates that the layer entity set is input into BiLSTM for feature extraction, thereby mapping the layer entity combination into a pile layer feature vector.
[0067] S500: Based on the prototype network model trained with the support set, the feature category of the current layer of the pile is obtained according to the layer feature vector of the current layer of the pile.
[0068] Specifically, the prototype network is a model that can achieve feature classification after training with a small number of support set samples. After training, the prototype network model can make objects of the same category more similar in features, while objects of different categories have greater differences in feature representation. The support set of the prototype network can be expressed as S = {(x1, y1), (x2, y2), ... (x N ,y N )}, where x1, x2...x N Represents the sample features in the support set, y1, y2...y N Represents the category attributes corresponding to each support set sample. Assuming there are K categories, each category is represented by k, then k∈{1, 2..., K}, then for each category k, the support set S can be set k , support set S k The sample category attributes of the samples in are all k, then the prototype of each category k is calculated through the support set
[0069]
[0070] where f θ (*) represents the feature vector obtained by extracting the sample features, which is the feature extraction performed by the BiLSTM network in the present invention, c k The prototype represents the average representation of the sample feature vectors in the kth category.
[0071] Then, the prototype network can calculate the layer feature vector X of the current layer of the input pile m The distribution of each category k relative to the K categories:
[0072]
[0073] Where k′ represents other categories among the K categories that do not belong to category k. Thus, the prototype network can be based on the layer feature vector X of the current layer of the pile m With respect to the distribution of each category k among the K categories, the characteristic category of the current layer of the pile is determined.
[0074] S600: According to the feature category of the working surface of the current layer of the stockpile, the control strategy for the stacker-reclaimer to execute the action is matched to realize the control of the stacker-reclaimer to execute the action.
[0075] Specifically, the stacker-reclaimer can have a preset control strategy for executing actions, and each control strategy corresponds to the feature type of the working surface of the current layer of the pile. Therefore, according to the feature category of the current layer of the pile determined by the prototype network, the control strategy for the stacker-reclaimer to execute actions can be matched to realize the control of the stacker-reclaimer to execute actions.
[0076] To implement the above method, see Figure 3The present invention provides a control system based on a three-dimensional real-time model of a stockpile layer, which specifically includes:
[0077] The 3D point cloud real-time acquisition module is used to collect 3D point cloud data of each layer of the material pile in real time using LiDAR during the material stacking or material removal process.
[0078] Laser radars are installed at appropriate locations of bucket wheel stackers, and 3D point cloud real-time acquisition modules are connected to each group of laser radars. When the stacker-reclaimer is stacking or retrieving materials, the working surface rises or falls layer by layer, so the laser radar can be used to perform 3D scanning on each layer of the working surface of the material pile, and the 3D point cloud real-time acquisition module can realize the real-time acquisition of point cloud spatial coordinate data of each layer of the working surface of the material pile.
[0079] The material pile working surface mesh division module performs adaptive mesh division based on the point cloud spatial coordinate data of each layer of the material pile working surface to obtain the three-dimensional mesh of each layer of the material pile working surface.
[0080] Specifically, for the three-dimensional point cloud spatial coordinate data of each layer of the material pile working surface, the specific process of the material pile working surface meshing module performing adaptive meshing includes:
[0081] The spatial coordinate information of the three-dimensional point cloud of each layer of the working surface is divided into initial grids; the initial grid is a polygon formed by connecting the characteristic points of the point cloud of each layer of the working surface as grid vertices, for example, the grid vertices can be determined based on the Delaunay triangulation algorithm to form a triangular initial grid;
[0082] Regularize the initial mesh, including supplementing the missing edges of the missing polygons, inserting new mesh vertices at the intersection of the polygon edges, and merging the edges of the overlapping polygons;
[0083] For the initial mesh after regularization, adaptive mesh subdivision is performed according to the mesh size and mesh quality index; specifically, firstly, the meshes to be subdivided in the initial mesh after regularization that need to be further adaptively subdivided are locked, and the meshes to be subdivided are meshes in the initial mesh whose any side length is greater than the set side length threshold, and any internal angle of the polygon is less than the angle threshold; then the center of the circumscribed circle of the mesh to be subdivided is inserted into the mesh to be subdivided as the inserted mesh vertex, and then the inserted mesh vertex and other mesh vertices of the mesh to be subdivided are connected to realize adaptive mesh subdivision; the above steps are repeated until all the meshes to be subdivided are adaptively subdivided.
[0084] The three-dimensional grid representation processing module is used to represent the three-dimensional grids contained in each layer of the working surface of the stockpile as a layer entity set.
[0085] Specifically, for each layer of the working surface of the stockpile, the three-dimensional grid finally divided by the stockpile working surface grid division module is used as the basic unit, and the three-dimensional grid characterization processing module represents it as a layer entity x = {..., l'2, l'1, m'1, m'2, h'1, h'2, ...}, where m = {m'1, m'2, ...} represents the layer entity set of the current layer of the working surface of the stockpile, and each three-dimensional grid of the current layer of the working surface of the stockpile is mapped to the layer entities m'1, m'2, ... in the set according to its spatial position. l' = {..., l'2, l'1} and h' = {h'1, h'2, ...} respectively represent the layer entity sets of the lower adjacent layer and the upper adjacent layer of the current layer of the stockpile.
[0086] The material pile layer feature extraction module is used to use the BiLSTM network to extract the features of the layer entity set of each layer working surface of the material pile to obtain the material pile layer feature vector.
[0087] Specifically, the stockpile layer feature extraction module inputs the layer entity set of each layer of the stockpile working surface as the input into the BiLSTM network, and can obtain the layer feature vector of the current layer of the stockpile, which is expressed as X m ,Right now
[0088] X m =f BiLSTM (..., l′2, l′1, m′1, m′2, h′1, h′2,...)
[0089] where f BiLSTM (*) indicates that the layer entity set is input into BiLSTM for feature extraction, thereby mapping the layer entity combination into a pile layer feature vector.
[0090] The material pile layer feature category classification module obtains the feature category of the current layer of the material pile according to the layer feature vector of the current layer of the material pile based on the prototype network model trained with the support set.
[0091] Specifically, the prototype network is a model that can achieve feature classification after training with a small number of support set samples. After training, the prototype network model can make objects of the same category more similar in features, while objects of different categories have greater differences in feature representation. The support set of the prototype network can be expressed as S = {(x1, y1), (x2, y2), ... (x N ,y N )}, where x1, x2...x N Represents the sample features in the support set, y1, y2...y N Represents the category attributes corresponding to each support set sample. Assuming there are K categories, each category is represented by k, then k∈{1, 2..., K}, then for each category k, the support set S can be set k, support set S k The sample category attributes of the samples in are all k, then the prototype of each category k is calculated through the support set
[0092]
[0093] where f θ (*) represents the feature vector obtained by extracting the sample features, which is the feature extraction performed by the BiLSTM network in the present invention, c k The prototype represents the average representation of the sample feature vectors in the kth category.
[0094] Furthermore, the stockpile layer feature category classification module can calculate the layer feature vector X of the current layer of the input stockpile based on the prototype network. m The distribution of each category k relative to the K categories:
[0095]
[0096] Where k′ represents other categories among the K categories that do not belong to category k. Thus, the prototype network can be based on the layer feature vector X of the current layer of the pile m With respect to the distribution of each category k among the K categories, the characteristic category of the current layer of the pile is determined.
[0097] The action control strategy module is used to match the control strategy of the stacker-reclaimer's execution of actions according to the feature category of the current layer working surface of the material pile, so as to realize the control of the stacker-reclaimer's execution of actions.
[0098] Specifically, the stacker-reclaimer can have a preset control strategy for executing actions, and each control strategy corresponds to the feature type of the working surface of the current layer of the pile. Therefore, according to the feature category of the current layer of the pile determined by the prototype network, the control strategy for the stacker-reclaimer to execute actions can be matched to realize the control of the stacker-reclaimer to execute actions.
[0099] The present invention can establish a three-dimensional real-time model of the material pile layering based on the point cloud spatial coordinate data of the working surfaces of each layer of the material pile. The model can reflect the subdivision and classification of the spatial characteristics of the working surfaces of each layer of the material pile during the material stacking and retrieving process, and then a more matched and more refined stacker-reclaimer equipment control method can be executed according to the subdivision and classification of the spatial characteristics of the working surfaces of each layer of the material pile, thereby improving the reliability and accuracy of the stacker-reclaimer operation, being able to adapt to the different characteristics of the working surfaces of the material pile, and ensuring the smooth and safe execution of the stacking and retrieving actions.
[0100] The above are only preferred implementations of the present invention. It should be pointed out that a number of modifications and improved technical solutions made by those skilled in the art without departing from the technical solution should also be deemed to fall within the scope of protection required by the claims.
Claims
1. A control method based on a three-dimensional real-time model of a stockpile layer, characterized in that: The following steps are involved: During the material stacking or material removal process, the laser radar is used to collect the 3D point cloud data of each layer of the material stack in real time; Based on the point cloud spatial coordinate data of each layer of the working surface of the stockpile, adaptive meshing is performed to obtain a three-dimensional mesh of each layer of the working surface of the stockpile. The specific steps include: The spatial coordinate information of the three-dimensional point cloud of each layer of the working surface is initially meshed; the initial mesh is a polygon formed by connecting the feature points of the point cloud of each layer of the working surface as mesh vertices, and the initial mesh is regularized, including supplementing the missing edges of the missing polygons, inserting new mesh vertices at the intersection of the edges of the polygons, and merging the edges of the overlapping polygons. For the initial mesh after regularization, adaptive mesh subdivision is performed according to the mesh size and mesh quality index. First, the mesh to be subdivided that needs to be further adaptively subdivided in the initial mesh after regularization is locked. The mesh to be subdivided is a mesh in which any side length of the initial mesh is greater than the set side length threshold, and any internal angle of the polygon is less than the angle threshold; then the center of the circumscribed circle of the mesh to be subdivided is inserted into the mesh to be subdivided as an inserted mesh vertex, and then the inserted mesh vertex and other mesh vertices of the mesh to be subdivided are connected to achieve adaptive mesh subdivision, and the above steps are repeated until all the meshes to be subdivided are adaptively subdivided; The three-dimensional grids contained in each layer of the stockpile working surface are characterized as a layer entity set, including: The three-dimensional grid after adaptive subdivision is used as the basic unit and represented as a layer entity. ,in, Represents the layer entity set of the current layer working surface of the pile, and maps each 3D grid of the current layer working surface of the pile to a layer entity in the set according to its spatial position and The layer entity sets representing the lower adjacent layer and the upper adjacent layer of the current layer of the stockpile respectively; The BiLSTM network is used to extract the features of the layer entity set of each layer working surface of the pile to obtain the pile layer feature vector; Based on the prototype network model trained with the support set, the feature category of the current layer of the pile is obtained according to the layer feature vector of the current layer of the pile; According to the characteristic category of the working surface of the current layer of the stockpile, the control strategy for the stacker-reclaimer to execute the action is matched to realize the control of the stacker-reclaimer to execute the action.
2. The control method based on the three-dimensional real-time model of stockpile stratification according to claim 1 is characterized in that: The mesh vertices are determined based on the Delaunay triangulated mesh algorithm to form an initial triangular mesh.
3. The control method based on the three-dimensional real-time model of stockpile stratification according to claim 1 is characterized in that: The regularization of the initial mesh includes: supplementing the missing edges of the missing polygons, inserting new mesh vertices at the intersection of the polygon edges, and merging the edges of the overlapping polygons.
4. The control method based on the three-dimensional real-time model of stockpile stratification according to claim 1 is characterized in that: The BiLSTM network is used to extract the features of the layer entity set of each layer working surface of the stockpile to obtain the layer feature vector of the stockpile. Specifically, the layer entity set of each layer working surface of the stockpile is input into the BiLSTM network as the input, so as to obtain the layer feature vector of the current layer of the stockpile, which is expressed as ,Right now middle It means that the layer entity set is input into BiLSTM for feature extraction, so as to map the layer entity combination into a pile layer feature vector.
5. A control system based on a three-dimensional real-time model of a stockpile layer, characterized in that: include: The 3D point cloud real-time acquisition module is used to collect the 3D point cloud data of each layer of the material pile in real time using the laser radar during the material pile or material removal process; The material pile working surface meshing module performs adaptive meshing based on the point cloud spatial coordinate data of each layer of the working surface of the material pile to obtain a three-dimensional mesh of each layer of the working surface of the material pile. The specific steps of the material pile working surface meshing module performing adaptive meshing for the three-dimensional point cloud spatial coordinate data of each layer of the working surface of the material pile include: The spatial coordinate information of the three-dimensional point cloud of each layer of the working surface is initially meshed; the initial mesh is a polygon formed by connecting the feature points of the point cloud of each layer of the working surface as mesh vertices, and the initial mesh is regularized, including supplementing the missing edges of the missing polygons, inserting new mesh vertices at the intersection of the edges of the polygons, and merging the edges of the overlapping polygons. For the initial mesh after regularization, adaptive mesh subdivision is performed according to the mesh size and mesh quality index. First, the mesh to be subdivided that needs to be further adaptively subdivided in the initial mesh after regularization is locked. The mesh to be subdivided is a mesh in which any side length of the initial mesh is greater than the set side length threshold, and any internal angle of the polygon is less than the angle threshold; then the center of the circumscribed circle of the mesh to be subdivided is inserted into the mesh to be subdivided as an inserted mesh vertex, and then the inserted mesh vertex and other mesh vertices of the mesh to be subdivided are connected to achieve adaptive mesh subdivision, and the above steps are repeated until all the meshes to be subdivided are adaptively subdivided; The three-dimensional grid representation processing module is used to represent the three-dimensional grids contained in each layer of the working surface of the stockpile as a layer entity set, including: The three-dimensional grid after adaptive subdivision is used as the basic unit and represented as a layer entity. ,in, Represents the layer entity set of the current layer working surface of the pile, and maps each 3D grid of the current layer working surface of the pile to a layer entity in the set according to its spatial position and The layer entity sets representing the lower adjacent layer and the upper adjacent layer of the current layer of the stockpile respectively; The stockpile layer feature extraction module is used to extract features from the layer entity set of each layer working surface of the stockpile using the BiLSTM network to obtain the stockpile layer feature vector; The stockpile layer feature category classification module obtains the feature category of the current layer of the stockpile based on the prototype network model trained with the support set and the layer feature vector of the current layer of the stockpile; The action control strategy module is used to match the control strategy of the stacker-reclaimer's execution of actions according to the feature category of the current layer working surface of the material pile, so as to realize the control of the stacker-reclaimer's execution of actions.
6. The control system based on the three-dimensional real-time model of stockpile stratification according to claim 5 is characterized in that: The stockpile working surface meshing module determines mesh vertices based on a Delaunay triangulated mesh algorithm to form an initial triangular mesh.
7. The control system based on the three-dimensional real-time model of stockpile stratification according to claim 6 is characterized in that: The stockpile layer feature extraction module uses the BiLSTM network to extract features from the layer entity set of each layer of the stockpile working surface to obtain the stockpile layer feature vector, which specifically includes: by inputting the layer entity set of each layer of the stockpile working surface into the BiLSTM network as an input, the layer feature vector of the current layer of the stockpile can be obtained, which is expressed as ,Right now in It means that the layer entity set is input into BiLSTM for feature extraction, so as to map the layer entity combination into a pile layer feature vector.
8. The control system based on the three-dimensional real-time model of stockpile stratification according to claim 7 is characterized in that: The prototype network of the pile layer feature category classification module calculates the layer feature vector of the current layer of the input pile Relative to Each of the categories Distribution: in express categories do not belong to the category Other categories of Represents the feature vector obtained by extracting features from sample features. As a prototype, The average representation of the sample feature vector in the category, so the prototype network can be based on the layer feature vector of the current layer of the material pile Relative to Each of the categories The distribution of the material pile is used to determine the characteristic category of the current layer.
