An intelligent goaf filling planning method based on machine vision and autonomous navigation

Abstract

The present application relates to the field of intelligent filling planning, and more particularly to a goaf intelligent filling planning method based on machine vision and autonomous navigation. The content includes: collecting point cloud data of the goaf environment and registering to obtain point cloud data under the global coordinate system; based on the point cloud data under the global coordinate system, introducing a local spectral weighted index algorithm to generate a structure weighted geometric reconstruction index; based on the structure weighted geometric reconstruction index, identifying the filling points and constructing the filling point set; based on the filling point set, combining the structure weighted geometric reconstruction index, introducing a progressive filling planning algorithm to select the next filling point and generate an ordered filling path. The problems of traditional point cloud processing methods that are difficult to automatically identify structural abnormalities or high-risk areas, and filling path planning that fails to make comprehensive decisions combining structural risk and spatial distance, and is difficult to achieve continuous, controllable and risk-prior intelligent filling planning are solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent filling planning, and more particularly to an intelligent filling planning method for goaf areas based on machine vision and autonomous navigation. Background Technology

[0002] With the continuous increase in the intensity of coal and metal mining, the number of underground goaf areas is constantly increasing. Due to the complex spatial structure and poor roof stability of goaf areas, they are prone to safety hazards such as collapse, caving, and surface subsidence. Therefore, carrying out scientific and reasonable backfilling operation planning is of great significance for ensuring safe production and efficient utilization of resources in mines.

[0003] In recent years, with the development of computer and sensor technologies, spatial perception technologies based on machine vision and multi-sensor fusion have been gradually applied to unmanned mining operations. Acquiring environmental data through lidar and vision systems, and performing point cloud registration, has become an important technical approach for constructing 3D models of underground spaces. However, traditional point cloud processing methods often focus on geometric reconstruction or simple planar extraction, lacking comprehensive analysis of local structural stability, spatial dispersion, and attitude characteristics. This makes it difficult to accurately characterize the risk level of local structures in goaf areas. Furthermore, large-scale point cloud data processing suffers from high computational load, insufficient real-time performance, and low automation, failing to meet the high efficiency and stability requirements of intelligent filling planning in the field of computer information processing. In addition, regarding filling path planning, traditional goaf filling methods rely heavily on manual experience or simple rules for area judgment and path arrangement, lacking intelligent perception and data-driven decision-making mechanisms based on machine vision and autonomous navigation. This makes it difficult to perform high-precision 3D modeling and risk identification of complex goaf areas.

[0004] Therefore, there is an urgent need for an intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation. On the basis of completing high-precision point cloud registration and global modeling, an evaluation index that can reflect the geometric discreteness and attitude relationship of local structures should be introduced to realize the automatic identification of abnormal structural areas. In addition, progressive path planning should be carried out in combination with structural weights, thereby improving the intelligence level and engineering safety of backfilling operations. Summary of the Invention

[0005] This invention provides an intelligent filling planning method for goaf areas based on machine vision and autonomous navigation, which solves the technical problems of traditional point cloud processing methods lacking evaluation indicators that can quantify the stability of local structures and the degree of spatial dispersion, making it difficult to automatically identify structural anomalies or high-risk areas; and filling path planning failing to combine structural risk and spatial distance for comprehensive decision-making, making it difficult to achieve continuous, controllable and risk-prioritized intelligent filling planning.

[0006] The present invention provides an intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation, comprising the following steps:

[0007] S1. Collect point cloud data of the goaf environment and register it to obtain point cloud data in the global coordinate system; based on the point cloud data in the global coordinate system, introduce the local spectral weighted index algorithm to generate the structural weighted geometric reconstruction index; based on the structural weighted geometric reconstruction index, identify the points to be filled and construct the set of points to be filled.

[0008] S2. Based on the set of points to be filled, and combined with the structural weighted geometric reconstruction index, an incremental filling planning algorithm is introduced to select the next filling point and generate an ordered filling path.

[0009] Preferably, S1 specifically includes:

[0010] Using the current frame point cloud and the reference point cloud as registration objects, the corresponding point pair search, decentralization processing, cross-covariance matrix construction and singular value decomposition are performed sequentially through the iterative nearest point algorithm to obtain the rotation matrix and translation vector, and construct the homogeneous transformation matrix; based on the homogeneous transformation matrix, the current frame point cloud is transformed into the global coordinate system to obtain the point cloud data in the global coordinate system.

[0011] Preferably, S1 specifically includes:

[0012] In the implementation of the local spectral weighted index algorithm, a neighborhood is constructed for the points in the point cloud data in the global coordinate system, and the offset of all points in the neighborhood relative to the current point is statistically analyzed to construct a covariance matrix; the covariance matrix is ​​decomposed into eigenvalues ​​to obtain eigenvalues ​​and corresponding eigenvectors; based on the eigenvalues, the geometric discrete ratio is calculated.

[0013] Preferably, S1 specifically includes:

[0014] In the implementation of the local spectral weighted index algorithm, a directional weighting mechanism is introduced. Based on the eigenvector corresponding to the minimum eigenvalue, directional consistency analysis is performed to calculate the directional consistency weight. Based on the geometric discretization ratio and the directional consistency weight, the structural weighted geometric reconstruction index is generated.

[0015] Preferably, S1 specifically includes:

[0016] Based on the structural weighted geometric reconstruction index, a structural index threshold is set to filter the points that need to be filled and construct a set of points that need to be filled.

[0017] Preferably, S2 specifically includes:

[0018] The progressive filling planning algorithm uses the structure-weighted geometric reconstruction index as the dominant factor and generates an ordered filling path through a stepwise weighted selection mechanism triggered by a single source starting point.

[0019] Preferably, S2 specifically includes:

[0020] In the implementation of the incremental filling planning algorithm, each iteration takes the currently filled point as the center and combines it with the set of points to be filled to construct the next filling point candidate set; based on each candidate point in the next filling point candidate set, combined with the spatial distance factor and the structural weighted geometric reconstruction index, the comprehensive selection cost is calculated.

[0021] Preferably, S2 specifically includes:

[0022] In the current set of candidates for the next fill point, select the point with the lowest overall selection cost as the next fill point. After the next fill point is filled, mark it as filled and update the current point position. Continue iterating until all points in the set of points to be filled are filled.

[0023] The beneficial effects of the technical solution of the present invention are:

[0024] 1. By performing local neighborhood statistical modeling on point clouds in the global coordinate system and constructing a covariance matrix for spectral decomposition, the main direction features are extracted, and a quantitative expression of the local structural geometry of the goaf is realized. This enables accurate characterization of spatial dispersion and structural stability. By generating geometric dispersion ratios, effective distinction between flat and broken areas is achieved.

[0025] 2. Further analysis of the consistency between the normal direction and the gravity direction was conducted to realize the perception and quantification of the local structural posture characteristics, thereby enabling the identification of different spatial forms such as horizontal, inclined and vertical structures, improving the ability to identify complex geological structures in goaf areas. By coupling the geometric discretization ratio with the directional consistency weight, a structural weighted geometric reconstruction index was constructed, realizing a comprehensive evaluation of structural volatility and potential risk areas, thereby improving the accuracy and reliability of abnormal area identification.

[0026] 3. In the path planning stage, a progressive filling planning algorithm is constructed with the structural weighted geometric reconstruction index as the dominant factor. This algorithm realizes a risk-first-oriented single-source continuous expansion path generation mechanism, thereby ensuring that the filling operation prioritizes coverage of areas with significant structural fluctuations and improving engineering safety. Through the coupled adjustment of the spatial distance factor and the structural weighted geometric reconstruction index, continuous spatial advancement driven by local optima is achieved, avoiding skip-path selection and improving the coherence and executability of path planning. Under the premise of ensuring the rationality of engineering logic, the algorithm effectively reduces the computational complexity caused by global search and improves the computational efficiency and system stability in large-scale point cloud scenarios. Attached Figure Description

[0027] Figure 1 This is a flowchart of an intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation, as described in this invention. Detailed Implementation

[0028] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0030] The following description, in conjunction with the accompanying drawings, details a specific scheme for an intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation provided by the present invention.

[0031] See attached document Figure 1 The diagram illustrates a flowchart of an intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation, according to an embodiment of the present invention. The method includes the following steps:

[0032] S1. Acquire local point cloud data of the frame and align it to obtain point cloud data in the global coordinate system; based on the point cloud data in the global coordinate system, introduce the local spectral weighted index algorithm to generate the structure-weighted geometric reconstruction index; based on the structure-weighted geometric reconstruction index, identify the points to be filled and construct the set of points to be filled.

[0033] During scanning, a lidar system measures the time difference between laser emission and echo, and combines this with the emission angle information to convert each ranging result into three-dimensional spatial point coordinates. The data collected at the same time constitutes a point cloud frame. All points in each frame are expressed using the lidar's own coordinate system as a reference, and this is called the current frame local point cloud. During continuous motion acquisition, multiple frames of local point clouds are obtained sequentially. To achieve spatial stitching, the current frame local point cloud needs to be aligned with an existing reference point cloud. The reference point cloud can be a point cloud whose coordinates have been unified in the previous frame, or it can be a globally constructed point cloud map.

[0034] At the same time, the visual SLAM module outputs the initial pose estimate of the current frame relative to the previous frame or the global coordinate system, which includes spatial position and planar pose information, and is used as the initial transformation parameters when the current frame point cloud is accurately registered with the reference point cloud.

[0035] During the registration stage, the Iterative Nearest Point (ICP) algorithm is employed. This algorithm iteratively optimizes the distance between corresponding points in the current frame point cloud and the reference point cloud, further calculating a more precise rigid transformation to achieve accurate point cloud alignment. Specifically, for each point in the current frame point cloud, a nearest neighbor search based on Euclidean distance is performed in the reference point cloud using a KD-Tree data structure to obtain a pair of corresponding points. Based on these pairs, the mean coordinates of the corresponding point sets in the current frame and the reference point cloud are calculated to obtain the centroids of the two sets of corresponding points. Then, the mean coordinates of each corresponding point are subtracted from the mean coordinates of its corresponding point set. The centroid is decentralized. Then, a 3x3 matrix, the cross-covariance matrix, is constructed by multiplying and summing the coordinates of each pair of points. Singular value decomposition is performed on the cross-covariance matrix, and the rotation matrix and translation vector are calculated based on the decomposition results. A homogeneous transformation matrix is ​​then constructed to describe the rigid transformation relationship between the current frame point cloud and the global coordinate system. Based on the homogeneous transformation matrix, rotation and translation operations are performed on each point in the current frame point cloud to obtain new coordinate values. The point cloud after the transformation is the point cloud data unified in the global coordinate system, or simply point cloud data in the global coordinate system.

[0036] To construct a comprehensive evaluation index that characterizes local structural stability, spatial discretization, and attitude features in a discrete point cloud environment, a local spectral weighted index algorithm is introduced. Taking point cloud data in a global coordinate system as input, local statistical modeling is performed for each point. The algorithm extracts the principal orientation information of the structure through spectral decomposition and combines it with the gravity direction to construct an orientation weighting mechanism, thereby generating a structural weighted geometric reconstruction index. The specific implementation process of the local spectral weighted index algorithm is as follows:

[0037] First, the points in the point cloud data in the global coordinate system are constructed using a fixed-scale nearest neighbor selection strategy. That is, the Euclidean distance between any point and all points except itself is calculated, and the points are sorted in ascending order of Euclidean distance. The top few nearest points are selected to form a local neighborhood set, i.e., the neighborhood.

[0038] After the neighborhood is determined, the offset vectors of all points in the local neighborhood set relative to the current point are calculated, and second-order statistical processing is performed on the offset vectors to construct a covariance matrix that describes the local spatial distribution characteristics. The covariance matrix essentially characterizes the extent of expansion and directional coupling of the point cloud in the local neighborhood set in three orthogonal directions. It is a mathematical expression of the local geometry. The diagonal components of the covariance matrix are used to reflect the discrete intensity of each direction, and the off-diagonal components are used to reflect the directional correlation.

[0039] The formula for calculating the covariance matrix is:

[0040]

[0041] in, Indicates the first The neighborhood covariance matrix of each point; Indicates the first The number of points within the local neighborhood set of a given point is a positive integer. In engineering, it is usually taken to be between 10 and 50; Indicates the first A set of local neighborhoods constructed with each point as the center; Indicates the first In the neighborhood of the nth point Three-dimensional coordinate vectors of points; Indicates the first Three-dimensional coordinate vectors of points; Represents the offset vector; Indicates the transpose operation;

[0042] The covariance matrix is ​​then decomposed into eigenvalues, yielding three eigenvalues ​​ordered by size and their corresponding eigenvectors. The eigenvalues ​​represent the variance of the neighborhood point cloud along the direction of the corresponding eigenvector, which can be understood as the extension intensity of the structure along the eigenvector direction. The largest eigenvalue... Corresponding to the local main expansion direction, it represents the structural extension trend, and the second largest eigenvalue. Reflects secondary expansion, minimum eigenvalue Corresponding to the normal direction or thickness direction, it is used to describe the degree of local surface undulation or spatial disturbance;

[0043] Furthermore, the relative proportion between the minimum eigenvalue and the total of the three eigenvalues ​​is calculated and used as the geometric discretization ratio to measure the three-dimensional discretization of the local structure: when the proportion of the minimum eigenvalue is extremely low, it indicates that the neighborhood point cloud is mainly distributed in the two-dimensional plane and the structure is flat; when the proportion of the minimum eigenvalue increases, it indicates that the point cloud has significant expansion in all three directions, the spatial discretization is improved, and the local structure tends to be irregular or fragmented.

[0044] Furthermore, a direction weighting mechanism is introduced, selecting the eigenvector corresponding to the smallest eigenvalue as the normal vector, and performing direction consistency analysis with the preset gravity direction vector. By calculating the direction angle between the normal vector and the gravity direction vector and performing normalization, the direction consistency weight is obtained. The direction consistency weight can reflect the spatial relationship between the local structure posture and the gravity direction: when the normal vector and the gravity direction are highly consistent, it indicates that the structure is close to a horizontal plane structure; when the consistency is low, it indicates that the structure is tilted or vertical.

[0045] Furthermore, the geometric discretization ratio is coupled with the directional consistency weight to generate a structurally weighted geometric reconstruction index, calculated as follows:

[0046]

[0047] in, Indicates the first The structural weighted geometric reconstruction index of each point; Indicates the geometric discrete ratio; Indicates the first The largest eigenvalue of the neighborhood covariance matrix of each point; Indicates the first The second largest eigenvalue of the neighborhood covariance matrix of each point; Indicates the first The smallest eigenvalue of the neighborhood covariance matrix of points; Indicates the weight of directional consistency; Indicates the first The eigenvector corresponding to the smallest eigenvalue of the neighborhood covariance matrix of a point, i.e., the normal vector, can reflect the normal direction; This represents the direction vector of gravity. ;

[0048] To ensure that only regions with significant structural fluctuations are processed, a structural index threshold is set. The structural index threshold is adaptively determined through statistical analysis of the weighted geometric reconstruction index of all structures, preferably using a combination of the mean and one standard deviation or the 75th percentile method, to achieve automatic identification of structural anomaly regions. The screening rules are as follows: When When, the point is determined to be a point that needs to be filled. When a point is determined to be a stable point, the points to be filled are combined to construct a set of points to be filled.

[0049] S2. Based on the set of points to be filled, and combined with the structural weighted geometric reconstruction index, an incremental filling planning algorithm is introduced to select the next filling point and generate an ordered filling path.

[0050] Based on the set of points to be filled, an incremental filling planning algorithm is introduced. Using a structurally weighted geometric reconstruction index as the dominant factor, and through a step-by-step weighted selection mechanism triggered by a single-source starting point, an ordered filling path that satisfies the engineering filling logic is generated. The specific implementation process of the incremental filling planning algorithm is as follows:

[0051] The structural weighted geometric reconstruction index of all points in the set of points to be filled is sorted, and the point with the largest structural weighted geometric reconstruction index is selected as the unique starting point to ensure that the areas with the greatest structural volatility or the highest stability risk are processed first, thereby achieving risk priority control. If there are multiple points with the same maximum structural weighted geometric reconstruction index, the unique point is determined according to the numbering order to ensure single source and execution determinism.

[0052] In each iteration, using the currently filled point as the center, points within the neighborhood that belong to the set of points to be filled but have not yet been filled are used to construct the next set of candidate points for filling. This next set of candidate points represents the leading edge region of the current spatial expansion, ensuring that the path generation process always proceeds along a continuous local spatial region, avoiding skip-selection. For each candidate point in the next set of candidate points, a comprehensive selection cost is calculated. This cost consists of two parts: the first is a spatial distance factor, reflecting the geometric proximity between the current point and the candidate point; the second is a structurally weighted geometric reconstruction index, reflecting the structural importance of the candidate point. A smaller spatial distance factor indicates lower movement costs, while a larger structurally weighted geometric reconstruction index indicates stronger structural risk or geometric volatility.

[0053] In the current candidate set of next fill points, select the point with the lowest comprehensive selection cost as the next fill point. After the next fill point is filled, mark it as filled and update the current point position. Continue iterating until all points in the set of points to be filled are filled.

[0054] The formula for selecting the next fill point is expressed as follows:

[0055]

[0056] in, Indicates the next fill point; This indicates the current filled point. The minimum value is searched by combining the selection costs in the candidate set of the next filling point; Indicates the currently filled points The candidate set of the next filling point; Indicates the currently filled points; Indicates the currently filled points Next fill point candidate set The first in Points The structural weighted geometric reconstruction index; Indicates the overall cost of the choice;

[0057] By coupling and adjusting the structural weighted geometric reconstruction index with spatial distance, a continuous expansion process driven by local optima is achieved. This not only ensures the engineering rationality of the filling plan but also significantly reduces computational complexity and improves applicability and stability in large-scale point cloud scenarios.

[0058] In summary, a method for intelligent backfilling planning of goaf areas based on machine vision and autonomous navigation has been developed.

[0059] The order of the embodiments is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0060] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0061] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for intelligent backfilling planning in goaf areas based on machine vision and autonomous navigation, characterized in that, Includes the following steps: S1. Collect point cloud data of the goaf environment and register it to obtain point cloud data in the global coordinate system. Based on the point cloud data in the global coordinate system, introduce the local spectral weighted index algorithm to construct the neighborhood of the points in the point cloud data in the global coordinate system, and perform statistical analysis on the offset of all points in the neighborhood relative to the current point to construct the covariance matrix. Eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors; based on The eigenvalues ​​are used to calculate the geometric discretization ratio; A direction weighting mechanism is introduced, and direction consistency analysis is performed based on the eigenvector corresponding to the minimum eigenvalue to calculate the direction consistency weight; Based on the geometric discretization ratio and directional consistency weight, a structural weighted geometric reconstruction index is generated; based on the structural weighted geometric reconstruction index, points that need to be filled are identified, and a set of points that need to be filled is constructed. S2. Based on the set of points to be filled, combined with the structural weighted geometric reconstruction index, an incremental filling planning algorithm is introduced to select the next filling point and generate an ordered filling path.

2. The intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation according to claim 1, characterized in that, S1 specifically includes: Using the current frame point cloud and the reference point cloud as registration objects, the corresponding point pair search, decentralization processing, cross-covariance matrix construction and singular value decomposition are performed sequentially through the iterative nearest point algorithm to obtain the rotation matrix and translation vector, and construct the homogeneous transformation matrix; based on the homogeneous transformation matrix, the current frame point cloud is transformed into the global coordinate system to obtain the point cloud data in the global coordinate system.

3. The intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation according to claim 1, characterized in that, S1 specifically includes: Based on the structural weighted geometric reconstruction index, a structural index threshold is set to filter the points that need to be filled and construct a set of points that need to be filled.

4. The intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation according to claim 1, characterized in that, S2 specifically includes: The progressive filling planning algorithm uses the structure-weighted geometric reconstruction index as the dominant factor and generates an ordered filling path through a stepwise weighted selection mechanism triggered by a single source starting point.

5. The intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation according to claim 4, characterized in that, S2 specifically includes: In the implementation of the incremental filling planning algorithm, each iteration takes the currently filled point as the center and combines it with the set of points to be filled to construct the next filling point candidate set; based on each candidate point in the next filling point candidate set, combined with the spatial distance factor and the structural weighted geometric reconstruction index, the comprehensive selection cost is calculated.

6. The intelligent backfilling planning method for goaf areas based on machine vision and autonomous navigation according to claim 5, characterized in that, S2 specifically includes: In the current set of candidates for the next fill point, select the point with the lowest overall selection cost as the next fill point. After the next fill point is filled, mark it as filled and update the current point position. Continue iterating until all points in the set of points to be filled are filled.

Images