Laser radar-based fully-mechanized mining control method and system, terminal device and medium

Abstract

The application relates to a kind of based on laser radar's fully mechanized control method, system, terminal equipment and medium, method includes: by the laser radar of preset track being set obtains point cloud data, and constructs the three-dimensional point cloud model of fully mechanized working face according to the point cloud data;The preset track is arranged in parallel with scraper conveyor;The three-dimensional point cloud model is dynamically processed to obtain fully mechanized point cloud model;The roof area point cloud and floor area point cloud in the fully mechanized point cloud model are eliminated and handled;Fully mechanized point cloud model after elimination and handling is identified to obtain identification result, and the identification result is used to assist control fully mechanized working face.The application can realize the perception and understanding of global information of working face, to improve the efficiency and safety of fully mechanized working.

Description

Technical Field

[0001] This invention relates to a fully mechanized mining control method, system, terminal equipment, and medium based on lidar. Background Technology

[0002] In the environment of fully mechanized coal mining faces, the working face layout is characterized by narrow space, pervasive dust and fog, low visibility, and high risk. Traditional methods relying on manual visual monitoring and two-dimensional imaging are insufficient to meet the safety and efficiency requirements of high-speed automated mining.

[0003] With the continuous improvement of the level of intelligence in coal mines, environmental perception technologies such as lidar and machine vision are gradually being introduced into coal mine production systems to achieve functions such as equipment obstacle avoidance, position detection, and safety monitoring. Currently, these technologies mainly include the following categories: 1) Fixed or localized laser scanning solutions For example, by installing laser scanning devices at key locations in coal mining machines, roadways, or working faces, two-dimensional or three-dimensional scans can be performed on localized areas to detect the distance between obstacles or measuring equipment and surrounding structures. However, this type of solution has a limited scanning range, making it difficult to cover the entire longwall mining face and unable to create a continuous, global three-dimensional environmental model.

[0004] 2) SLAM-based moving scan modeling scheme For example, lidar can be mounted on mobile equipment to construct environmental maps through feature matching and pose estimation. However, fully mechanized mining faces have obvious linear structural characteristics, and there are many repetitive structures in the scene (such as hydraulic supports). SLAM algorithms are prone to problems such as feature degradation and amplification of cumulative errors, making it difficult to obtain stable and reliable modeling results.

[0005] Although existing technologies have incorporated methods such as lidar to some extent, it is still difficult to achieve stable 3D modeling, effective semantic understanding, and reliable auxiliary control in long-distance, structurally repetitive, and highly interfering environments such as fully mechanized mining faces. Summary of the Invention

[0006] The purpose of this invention is to provide a fully mechanized mining control method, system, terminal equipment, and medium based on lidar, so as to solve the problem of poor environmental perception in existing fully mechanized mining operations.

[0007] To address the aforementioned problems, the integrated mining control method based on lidar involved in this invention adopts the following technical solution: Point cloud data is acquired by a laser radar set on a preset track, and a three-dimensional point cloud model of the fully mechanized mining face is constructed based on the point cloud data; the preset track is set parallel to the scraper conveyor. The three-dimensional point cloud model is dynamically processed to obtain the fully mechanized mining point cloud model; The point clouds of the top plate area and the bottom plate area in the fully mechanized mining point cloud model are removed. The identification results are obtained by performing identification processing on the fully mechanized mining point cloud model after the rejection process. The identification results are used to assist in the control of the fully mechanized mining face.

[0008] In some embodiments, the process of acquiring point cloud data using a lidar mounted on a preset track and constructing a three-dimensional point cloud model of the fully mechanized mining face based on the point cloud data includes: (1) Divide the fully mechanized mining face into multiple scanning segments along the length direction of the preset track; and acquire the stage point cloud data corresponding to each scanning segment through the lidar; the stage point cloud data includes point cloud data within the segment and point cloud data outside the segment; (2) Register the intra-segment point cloud data of each scan segment using a preset algorithm; (3) Align all adjacent scan segments after registration to obtain a three-dimensional point cloud model of the fully mechanized mining face.

[0009] In some embodiments, the process of dynamically processing the three-dimensional point cloud model to obtain the fully mechanized mining point cloud model includes: The three-dimensional point cloud model is divided into regionalized point cloud models based on spatial location and functional attributes. The fully mechanized mining point cloud model is obtained by processing the point clouds of different regions of the regionalized point cloud model using a preset method.

[0010] In some embodiments, the regionalized point cloud model includes a roof regional point cloud, a floor regional point cloud, a pre-defined regional point cloud of the coal wall and working face, an equipment regional point cloud, and an unstructured spatial point cloud; the equipment in the equipment region includes a coal mining machine, a hydraulic support, and a scraper conveyor.

[0011] In some embodiments, the processing of the point cloud of the device region includes: Based on the real-time pose information of the device, the pre-established device geometric model is mapped to the regionalized point cloud model; Using the coal mining machine as the reference center, the equipment area point cloud of the regionalized point cloud model is further divided into the front area, the middle area, and the rear area. The front region, the middle region, and the rear region are processed separately to update the device region point cloud.

[0012] In some embodiments, the process of removing the point cloud of the roof region and the point cloud of the floor region in the fully mechanized mining point cloud model includes: (1) Project the three-dimensional point cloud data of the fully mechanized point cloud model onto a two-dimensional plane, and divide the two-dimensional grid into multiple grid units according to the first grid size; (2) Statistically determine the candidate values ​​for the bottom plate height and the top plate height by analyzing the point cloud height information of each grid cell; (3) Determine the rate of change of the top plate height and the rate of change of the bottom plate height based on the candidate values ​​of the bottom plate height and the candidate values ​​of the top plate height, respectively; (4) Determine the target grid cell based on the height change rate of the top plate and the height change rate of the bottom plate, and perform a rejection operation on the target grid cell.

[0013] In some embodiments, the recognition result includes semantic content and multidimensional geometric feature information; The semantic content includes equipment, personnel, vehicles, and obstacles; The multidimensional geometric feature information includes geometric morphological features, spatial positional relationship features, and density distribution features.

[0014] To address the aforementioned problems, the present invention relates to a comprehensive mining control system based on lidar, comprising a data acquisition unit, a dynamic processing unit, a rejection processing unit, and an identification unit. The data acquisition unit is used to acquire point cloud data through a lidar set on a preset track, and to construct a three-dimensional point cloud model of the fully mechanized mining face based on the point cloud data; the preset track is set parallel to the scraper conveyor. The dynamic processing unit is used to dynamically process the three-dimensional point cloud model to obtain a fully mechanized point cloud model. The elimination processing unit is used to eliminate the point cloud of the top plate area and the point cloud of the bottom plate area in the fully mechanized mining point cloud model. The identification unit is used to identify the fully mechanized mining point cloud model after the rejection process to obtain the identification result, and the identification result is used to assist in the control of the fully mechanized mining face.

[0015] To address the aforementioned problems, the present invention relates to a terminal device comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned integrated mining control method based on lidar.

[0016] To address the aforementioned problems, the present invention relates to a computer-readable storage medium storing a computer program, which, when executed on a processor, implements the aforementioned integrated mining control method based on lidar.

[0017] The beneficial effects of this invention are as follows: This invention presents a fully mechanized mining control method based on lidar. By installing a mobile 3D lidar on the parallel track of the scraper conveyor at the fully mechanized mining face, it can scan the entire face back and forth along the track, achieving perception and understanding of global information about the face. Simultaneously, due to the lidar's strong active emission capability, the system can operate stably in the low-light, high-dust environment of coal mines, overcoming the limitations of traditional cameras and fixed scanning. Mobile acquisition improves measurement efficiency and safety, enabling rapid response to environmental changes. This invention, through dynamic processing, reduces the data volume of the fully mechanized mining point cloud model while ensuring its accuracy. Therefore, based on this fully mechanized mining point cloud model, auxiliary control of fully mechanized mining operations can be achieved, thereby improving the efficiency and safety of fully mechanized mining work. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below: Figure 1 This is a flowchart illustrating the integrated mining control method based on lidar in a specific embodiment of the present invention. Figure 2 This is a schematic diagram of the process for dynamically processing and obtaining the fully mechanized mining point cloud model in a specific embodiment of the present invention. Detailed Implementation

[0019] To make the technical objectives, technical solutions, and beneficial effects of the present invention clearer, the technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention; that is, the described embodiments are merely some embodiments of the present invention, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] In the following text, the terms "comprising," "having," and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more combinations thereof. Furthermore, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0021] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0022] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0023] This invention addresses the current lack of real-time perception and semantic understanding of the global 3D environment in fully mechanized mining systems. This leads to inaccurate coal mining machine path planning, delayed support position adjustments, and difficulty in timely obstacle detection and early warning, thus affecting production efficiency and personnel safety. The invention proposes an environmental interpretation and detection (EID) auxiliary control technology based on lidar in fully mechanized mining faces. By setting a lidar track along the scraper conveyor direction on the fully mechanized mining face, the lidar reciprocates and scans within the face to acquire continuous, full-coverage point cloud data. Combined with point cloud modeling, preprocessing, segmentation, and semantic analysis methods designed specifically for the characteristics of the fully mechanized mining environment, the invention achieves 3D reconstruction and semantic-level understanding of the working face environment. The environmental interpretation and detection results are then used for assisted driving, assisted mining, and obstacle avoidance early warning, thereby improving the safety and intelligence level of the fully mechanized mining system.

[0024] The following describes the lidar-based integrated mining control method, system, terminal equipment, and medium using specific embodiments.

[0025] Specific embodiments of the integrated mining control method based on lidar involved in this invention are as follows: Figure 1 As shown, the method includes: The S100 acquires point cloud data through a laser radar set on a preset track, and constructs a three-dimensional point cloud model of the fully mechanized mining face based on the point cloud data; the preset track is set parallel to the scraper conveyor.

[0026] In a specific embodiment of the present invention, the lidar is not fixedly installed or moves freely with the coal mining machine, but is installed on a preset track that is parallel to the scraper conveyor. The preset track is laid out along the direction of the fully mechanized mining face, so that the lidar has a clear main direction of motion under the constraint of the preset track, and the pose change of the lidar on the preset track is measurable and controllable.

[0027] The lidar of this invention is installed on a preset track. The resulting point cloud sequence naturally has a topological order along the working face direction. It performs reciprocating scanning along the preset track throughout the entire longwall mining face, ensuring panoramic data acquisition capability. This scanning method differs from existing free-moving SLAM mapping or fixed-point scanning methods, and is particularly suitable for the special environment of longwall mining faces, which are narrow, linear, and involve continuous operations.

[0028] In a specific embodiment of the present invention, the system establishes a three-dimensional coordinate system based on a preset track, with the length direction along the longwall mining face and the preset track as the X-axis, the direction perpendicular to the preset track and on the same horizontal plane as the Y-axis, and the direction perpendicular to the horizontal plane where the preset track is located as the Z-axis.

[0029] During the movement of the lidar on the preset track, the lidar acquires its position information in real time along the preset track direction through a track encoder, travel sensor, or equivalent measurement device. The relative pose information of the point cloud data scanned by the lidar in the X-axis direction is the same as the lidar's position information on the preset track. This invention determines the relative position of the point cloud data in the X-axis direction through the track position information on the X-axis to complete the initial spatiotemporal registration in one dimension. Furthermore, the method of determining the point cloud data in the X-axis avoids the problem of relying entirely on ICP or NDT algorithms for full-degree-of-freedom point cloud registration in the prior art, significantly reducing the registration difficulty and the risk of error accumulation.

[0030] In a specific embodiment of the present invention, to improve the overall stability and accuracy of the working surface point cloud model, a segmented point cloud modeling strategy is adopted; the segmented point cloud modeling process includes: (1) Divide the fully mechanized mining face into multiple scanning segments along the length of the preset track; and acquire the stage point cloud data corresponding to each scanning segment; the stage point cloud data includes point cloud data within the segment and point cloud data outside the segment; In actual data acquisition, point cloud data includes point cloud data of all stages corresponding to the scanning segments. When acquiring point cloud data of each scanning segment, in order to ensure the integrity of the segment point data, point cloud data of local areas outside the segment points are generally scanned. Therefore, the point cloud data of this invention includes point cloud data within the segment and point cloud data outside the segment.

[0031] (2) Register the intra-segment point cloud data of each scan segment using a preset algorithm; Specifically, within each scanning segment, the improved ICP algorithm or NDT algorithm is used to perform fine registration of the intra-segment point cloud data. Since the relative pose in the X direction of the point cloud data is directly determined by the position of the preset track, the fine registration mainly corrects minor deviations in the Y and Z directions and attitude.

[0032] (3) Align all adjacent scan segments after registration to obtain a three-dimensional point cloud model of the fully mechanized mining face.

[0033] Specifically, inter-segment alignment is achieved through a pre-defined overlapping area of ​​segment points in adjacent scan segments. This pre-defined overlapping area includes point cloud data outside the segment. The adjacent scan segments are the first and second scan segments. Inter-segment alignment using the pre-defined overlapping area includes aligning the point cloud data within the first scan segment with the point cloud data outside the second scan segment, and aligning the point cloud data outside the first scan segment with the point cloud data within the second scan segment. The pre-defined overlapping area is essentially an introduced pre-defined orbital geometric constraint to suppress long-distance drift. It should be noted that the point cloud outside the segment included in the pre-defined overlapping area can include some or all of the point cloud data outside the adjacent scan segments.

[0034] This invention constructs and merges segments to form a complete three-dimensional point cloud model of a fully mechanized mining face. This process does not rely on a global SLAM loop closure detection mechanism and is particularly suitable for scenarios such as fully mechanized mining faces where loop closure conditions are weak and environmental repeatability is high.

[0035] S200 is used to dynamically process a 3D point cloud model to obtain a fully mechanized mining point cloud model.

[0036] In this embodiment of the invention, for point cloud data with large volume and significant regional functional differences in fully mechanized mining faces, the point cloud of the three-dimensional point cloud model is partitioned based on spatial location and functional attributes. This facilitates the use of different processing strategies to process point clouds in different partitions, avoiding the application of a uniform downsampling strategy to all point clouds.

[0037] It should be noted that the point cloud division is dynamically updated according to the position of the coal mining machine.

[0038] In this embodiment of the invention, the process of dynamically processing a three-dimensional point cloud model to obtain a fully mechanized mining point cloud model includes: S210, based on spatial location and functional attributes, divides the 3D point cloud model into regionalized point cloud models.

[0039] In this embodiment of the invention, based on actual working conditions and experience, the three-dimensional point cloud model is initially divided. The resulting regionalized point cloud model includes roof area point cloud, floor area point cloud, coal wall and working face preset area point cloud, equipment area point cloud, and unstructured spatial point cloud. The equipment includes a coal mining machine, hydraulic supports, and scraper conveyors.

[0040] It should be noted that unstructured spatial point cloud is a set of point clouds that are spatially irregular and difficult to describe by a predefined geometric model after removing point clouds of roof, floor, coal wall and equipment with obvious geometric continuity and regular structural features. This set of point clouds corresponds to the candidate area of ​​obstacles that may contain personnel, temporary equipment or other foreign objects, and can also be understood as the obstacle concentration area.

[0041] S220 uses a preset method to process the point clouds of different regions of the regionalized point cloud model to obtain the fully mechanized mining point cloud model.

[0042] In this embodiment of the invention, for the point clouds of the top and bottom plate regions, which are mainly used for environmental background description, the point cloud range is large and the geometric changes are relatively gentle. Therefore, a voxel grid downsampling method is adopted, and the grid size is a first preset value, so as to significantly reduce the amount of data while preserving the overall geometric trend. The first preset value is preferably in the range of 0.3m to 1.0m, and can be 0.4, 0.8, 0.9, etc.

[0043] In this embodiment of the invention, for the point cloud of the coal wall and working face preset area mainly used for environmental background description, the point cloud of this area has a large range and relatively gentle geometric changes. Therefore, a voxel grid downsampling method is also adopted, and the grid size is a second preset value, so as to significantly reduce the amount of data while preserving the overall geometric trend. The second preset value is preferably in the range of 0.3m to 1.0m. Since this area is directly related to the cutting height, path and safety of the coal mining machine, the second preset value actually needs to be smaller than the first preset value. For example, the second preset value is 0.4, 0.5, 0.6, etc.

[0044] In this embodiment of the invention, in order to avoid interference from the device's own point cloud to environmental modeling and obstacle recognition, the invention introduces a point cloud occlusion mechanism based on the device's geometric model to process the point cloud of the device area.

[0045] First, based on the device's real-time pose information, the pre-established device geometric model is mapped to the regionalized point cloud model; Then, using the coal mining machine as the reference center, the equipment area point cloud of the regionalized point cloud model is further divided into the front area, the middle area, and the rear area. Finally, the front, middle, and rear regions are processed separately to update the device area point cloud.

[0046] Specifically, three-dimensional geometric models or envelope models of the coal mining machine, hydraulic support, and scraper conveyor are pre-established in the system. Then, based on the real-time pose information of the equipment, the three-dimensional geometric models, i.e., the equipment geometric models, are mapped to the point cloud coordinate system. Point clouds falling within the three-dimensional geometric models are directly marked as equipment point clouds and do not participate in obstacle detection and environmental modeling. At the same time, with the coal mining machine as the reference center, the point cloud is divided into a front region, a middle region, and a rear region: the front region uses a radius filtering and density preservation strategy to ensure the integrity of the point cloud of obstacles in front; the middle region combines model occlusion and strong downsampling; and the rear region uses large voxel downsampling or is directly discarded.

[0047] S300 performs a process to remove point clouds from the roof and floor regions of the fully mechanized mining point cloud model.

[0048] Due to the complex working conditions of the actual fully mechanized mining face, such as the undulation of the floor and the unevenness of the roof, it is necessary to perform fine processing on the point clouds of the roof and floor regions obtained in the initial division to improve their accuracy and thus ensure the accuracy of semantic analysis.

[0049] In this embodiment of the invention, the process of removing point clouds from the roof and floor regions of the fully mechanized mining point cloud model includes: (1) Project the three-dimensional point cloud data of the fully mechanized point cloud model onto a two-dimensional plane, and divide the two-dimensional grid according to the first grid size to obtain multiple grid units; Specifically, the acquired 3D point cloud data is projected horizontally onto the X–Y plane and then rasterized into two dimensions according to a preset first grid size. Alternatively, it can be projected onto the YZ plane.

[0050] (2) Statistically determine the candidate values ​​for the bottom plate height and the top plate height by analyzing the point cloud height information of each grid cell; Specifically, within each grid cell, the height information of the point cloud falling into that grid cell is statistically analyzed, and the minimum height value is preferably used as the candidate value for the bottom plate height, and the maximum height value is used as the candidate value for the top plate height.

[0051] (3) Determine the rate of change of the height of the top plate and the rate of change of the height of the bottom plate based on the candidate values ​​of the bottom plate height and the candidate values ​​of the top plate height, respectively; This invention defines the height change rate as the ratio of the height difference between adjacent grid cells to the horizontal distance between the centers of the corresponding grid cells, used to characterize the slope variation in a local area; therefore, the height change rate between candidate height values ​​of adjacent grid cells is calculated based on the height variation between adjacent grid cells. Specifically, the top plate height change rate is the ratio of the difference between the candidate top plate height values ​​of adjacent grid cells to the difference between the center distances of adjacent grid cells; the bottom plate height change rate is the ratio of the difference between the candidate bottom plate height values ​​of adjacent grid cells to the difference between the center distances of adjacent grid cells.

[0052] (4) Determine the target grid cell based on the top plate height change rate and the bottom plate height change rate, and perform a rejection operation on the target grid cell.

[0053] In this embodiment of the invention, when the change trend of the top plate height change rate is the same and the change rate of the top plate height is less than a preset threshold, the grid cell corresponding to the height change rate being less than the preset threshold is taken as the target grid cell.

[0054] When the height change rate between multiple adjacent grid cells changes continuously and the corresponding height change rate is less than a preset threshold, the point cloud in the area is determined to be a continuous structural point cloud of the top or bottom plate, and the point cloud in the area is removed from the subsequent obstacle recognition process. The preset threshold is set according to the working surface tilt angle and the flatness of the top / bottom plate, preferably within the range of 5° to 15°. When there is a significant abrupt change in the height change rate between adjacent grid cells, or when the corresponding height change rate is greater than a preset threshold, the point cloud in that area is determined not to be a continuous structure of the top or bottom plate, and is retained as a non-top and non-bottom plate structure point cloud as candidate data for subsequent obstacle recognition and semantic analysis.

[0055] The present invention does not simply use a fixed height threshold to judge the removal of point clouds on the top and bottom plates. Instead, it constructs an adaptive removal method based on the local geometric continuity and slope characteristics of the point cloud in spatial distribution. This method can effectively adapt to the actual working conditions of the bottom plate undulation and the local unevenness of the top plate in the fully mechanized mining face, and reduce the phenomenon of false removal while ensuring the accuracy of removal.

[0056] S400 performs identification processing on the fully mechanized mining point cloud model after the rejection process to obtain the identification results, which are used to assist in the control of the fully mechanized mining face.

[0057] In this embodiment of the invention, after removing the point clouds of the roof and floor, the system performs two-dimensional raster density analysis on the remaining point clouds. The remaining point clouds mainly include supports, coal walls, equipment, and obstacles. Among them, areas with high density and strong continuity are identified as coal walls or large structures. The remaining point clouds are used as candidate obstacle point clouds, and are further segmented using a Euclidean distance clustering algorithm. The clustering results generate three-dimensional bounding boxes, which are then used as independent obstacle targets for semantic analysis to locate various obstacles in the fully mechanized mining face and determine the spatial positions of components such as the coal mining machine and supports. For example, obstacles include fallen coal blocks, equipment displacement, and missed objects.

[0058] Therefore, the identification results obtained by the present invention through the identification processing of candidate point clouds in the fully mechanized mining point cloud model include: (1) Perform two-dimensional density analysis on the fully mechanized mining point cloud model after the removal process to determine the point cloud of the coal wall and the working face in the preset area; and take the remaining point cloud after deleting the points of the coal wall and the working face in the preset area as the candidate point cloud; (2) Extract multidimensional geometric feature information of candidate point clouds. Multidimensional geometric feature information includes geometric morphology features, spatial position relationship features and density distribution features.

[0059] For example, geometric morphological features include the size range of point cloud clusters, length-width-height ratio, voxel occupancy rate, local curvature distribution, and surface normal vector consistency; spatial positional features include the relative position of point cloud clusters with respect to the coal wall, floor, and scraper conveyor, height distribution range, and extension characteristics along the working face direction; density distribution features include the density, aggregation, and boundary clarity of point clouds in space.

[0060] The present invention further includes the following steps for identifying candidate point clouds in a fully mechanized mining point cloud model: (1) The Euclidean distance clustering algorithm was used to segment the fully mechanized mining point cloud model after the removal process to obtain multiple point cloud clusters; (2) Based on the preset semantic segmentation network model, the semantic content of each point cloud cluster is identified and processed; the semantic content includes equipment, personnel, vehicles and obstacles.

[0061] In practical implementation, various semantics are taken as input, and a semantic segmentation network model based on deep learning (e.g., PointNet++, DGCNN, LFE-Net, etc.) is used to classify point clouds. Customized training is performed for the coal mine fully mechanized mining environment, enabling the network to output point-level or cluster-level semantic category labels. The stability and reliability of semantic recognition can be further improved by fusing and verifying the network output with rule-based spatial constraints.

[0062] For example, semantic judgment: point cloud areas located in front of the coal face, with continuous planar distribution and obvious height variation patterns, are identified as coal face or coal-rock interface; point cloud areas located above the floor, with regular geometric shape and fixed height distribution characteristics, are identified as fixed equipment components such as hydraulic supports or scraper conveyors; point cloud areas located in non-fixed areas, with irregular point cloud distribution, small volume and large height variation are identified as personnel, vehicles or other temporary obstacles.

[0063] It should be noted that the focus of this invention is on the process of constructing a fully mechanized mining point cloud model, and on how this model enables auxiliary control, thereby improving the efficiency and safety of fully mechanized mining operations. The specific semantic content recognition process is not the focus of this invention, and therefore will not be elaborated upon in detail.

[0064] Therefore, the recognition results of this invention include semantic content and multi-dimensional geometric feature information. Based on the recognition results, it can assist in coal mining machine operation, auxiliary mining control, and obstacle avoidance and early warning for personnel and equipment, providing a high-level environmental understanding capability for fully mechanized mining systems. Specifically, it can achieve the following functions: Assisted driving – Based on geological models and the shape of the segmented coal seam, the forward path of the coal mining machine is planned and adjusted to maintain its advance along the coal seam and improve cutting efficiency; Collision avoidance warning and control - When a potential collision risk is detected between the cutting part of the coal mining machine and the hydraulic support guard plate, the system calculates the relative angle between the two in real time and automatically issues a command to retract the guard plate or appropriately lower the cutting part of the coal mining machine to avoid interference accidents. Scraper conveyor straightening—monitors the straightness of the conveyor track. If it is found to be non-compliant, the decision module calculates the required support push compensation distance and controls the support to perform straightening operation. Obstacle alarm – If an obstacle is detected in front of the workpiece or personnel enter the danger zone, the system will immediately issue an alarm and can control the coal mining machine to slow down or stop. Mining optimization suggestions – combining real-time point cloud models and semantic information to provide operators or automated systems with auxiliary mining decisions, such as adjusting support positions and optimizing coal face proximity.

[0065] The integrated mining control method based on lidar of the present invention has the following advantages: 1. Comprehensive environmental perception: This invention uses a mobile 3D lidar to scan the entire fully mechanized mining face, which can build a cloud environment model of the entire site in real time, realizing the perception and understanding of global information of the working face, rather than relying solely on local 2D images.

[0066] 2. All-weather reliability: Due to the strong active emission capability of lidar, the system can operate stably in the low-light and high-dust environment of coal mines, overcoming the limitations of traditional cameras and fixed scanning. Mobile acquisition improves measurement efficiency and safety, and can quickly respond to environmental changes.

[0067] 3. Semantic-level environmental understanding: By introducing deep learning semantic segmentation technology, the system can not only reconstruct geometric shapes but also identify semantic targets such as coal roofs, coal walls, supports, and people and vehicles, achieving true environmental interpretation. This semantic perception capability greatly improves the accuracy and relevance of decision support.

[0068] 4. Real-time Intelligent Assisted Decision Making: By integrating an obstacle detection and control decision-making module, the system can respond quickly when dangerous situations are detected, such as automatically retracting the side guards to avoid collisions, alerting or controlling the coal mining machine to avoid obstacles, thereby effectively improving production safety and efficiency. Compared with existing solutions that only focus on single functions, this invention provides more comprehensive auxiliary control performance.

[0069] 5. Improved Mining Safety and Efficiency: Through full-process environmental perception and decision-making, this invention supports reduced or even unmanned operation of fully mechanized mining faces. It ensures that equipment such as coal mining machines operate along the optimal path and that supports are accurately coordinated, thereby improving mining quality, reducing downtime, and minimizing personnel exposure in hazardous areas.

[0070] An embodiment of the present invention provides a fully mechanized mining control system based on lidar, comprising a data acquisition unit, a dynamic processing unit, a rejection processing unit, and an identification unit. The data acquisition unit acquires point cloud data using lidar mounted on a preset track and constructs a three-dimensional point cloud model of the fully mechanized mining face based on the point cloud data. The preset track is parallel to the scraper conveyor. The dynamic processing unit dynamically processes the three-dimensional point cloud model to obtain a fully mechanized mining point cloud model. The rejection processing unit rejects point clouds from the roof and floor regions of the fully mechanized mining point cloud model. The identification unit identifies the rejected fully mechanized mining point cloud model to obtain an identification result, which is used to assist in controlling the fully mechanized mining face.

[0071] This invention relates to a fully mechanized coal mining control system based on lidar, designed for the complex, dynamic, and highly disruptive working environment of fully mechanized coal mining faces. Through reciprocating scanning of lidar under track constraints, it achieves stable construction of a three-dimensional point cloud of the working face. Based on this, it completes point cloud preprocessing, obstacle identification, environmental semantic understanding, and auxiliary control decision-making, thereby providing reliable environmental perception and decision-making basis for assisted driving, assisted mining, and obstacle avoidance early warning of coal mining machines.

[0072] This application also provides a terminal device, exemplary of which includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to enable the terminal device to perform the aforementioned lidar-based integrated mining control method.

[0073] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0074] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving execution instructions.

[0075] This application also provides a computer-readable storage medium for storing the computer program used in the aforementioned terminal device. For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0077] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0078] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0079] Finally, it should be noted that the above embodiments are only for illustration and not for limiting the technical solutions of the present invention. Any equivalent substitutions, modifications or partial substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A fully mechanized mining control method based on lidar, characterized in that, include: Point cloud data is acquired by a laser radar set on a preset track, and a three-dimensional point cloud model of the fully mechanized mining face is constructed based on the point cloud data. The preset track is set parallel to the scraper conveyor; The three-dimensional point cloud model is dynamically processed to obtain the fully mechanized mining point cloud model; The point clouds of the top plate area and the bottom plate area in the fully mechanized mining point cloud model are removed. The identification results are obtained by performing identification processing on the fully mechanized mining point cloud model after the rejection process. The identification results are used to assist in the control of the fully mechanized mining face.

2. The integrated mining control method based on lidar according to claim 1, characterized in that, The process of acquiring point cloud data using a lidar system set on a preset track and constructing a three-dimensional point cloud model of the fully mechanized mining face based on the point cloud data includes: (1) Divide the fully mechanized mining face into multiple scanning segments along the length direction of the preset track; and acquire the stage point cloud data corresponding to each scanning segment through the lidar; the stage point cloud data includes point cloud data within the segment and point cloud data outside the segment; (2) Register the intra-segment point cloud data of each scan segment using a preset algorithm; (3) Align all adjacent scan segments after registration to obtain a three-dimensional point cloud model of the fully mechanized mining face.

3. The integrated mining control method based on lidar according to claim 2, characterized in that, The process of dynamically processing the three-dimensional point cloud model to obtain the fully mechanized mining point cloud model includes: The three-dimensional point cloud model is divided into regionalized point cloud models based on spatial location and functional attributes. The fully mechanized mining point cloud model is obtained by processing the point clouds of different regions of the regionalized point cloud model using a preset method.

4. The integrated mining control method based on lidar according to claim 3, characterized in that, The regionalized point cloud model includes roof regional point cloud, floor regional point cloud, coal wall and working face preset regional point cloud, equipment regional point cloud and unstructured spatial point cloud; the equipment in the equipment region includes coal mining machine, hydraulic support and scraper conveyor.

5. The integrated mining control method based on lidar according to claim 4, characterized in that, The processing of the point cloud in the device area includes: Based on the real-time pose information of the device, the pre-established device geometric model is mapped to the regionalized point cloud model; Using the coal mining machine as the reference center, the equipment area point cloud of the regionalized point cloud model is further divided into the front area, the middle area, and the rear area. The front region, the middle region, and the rear region are processed separately to update the device region point cloud.

6. The integrated mining control method based on lidar according to claim 5, characterized in that, The process of removing the point clouds of the top plate region and the bottom plate region from the fully mechanized mining point cloud model includes: (1) Project the three-dimensional point cloud data of the fully mechanized point cloud model onto a two-dimensional plane, and divide the two-dimensional grid into multiple grid units according to the first grid size; (2) Statistically determine the candidate values ​​for the bottom plate height and the top plate height by analyzing the point cloud height information of each grid cell; (3) Determine the rate of change of the top plate height and the rate of change of the bottom plate height based on the candidate values ​​of the bottom plate height and the candidate values ​​of the top plate height, respectively; (4) Determine the target grid cell based on the height change rate of the top plate and the height change rate of the bottom plate, and perform a rejection operation on the target grid cell.

7. The integrated mining control method based on lidar according to claim 1, characterized in that, The recognition results include semantic content and multidimensional geometric feature information; The semantic content includes equipment, personnel, vehicles, and obstacles; The multidimensional geometric feature information includes geometric morphological features, spatial positional relationship features, and density distribution features.

8. A fully mechanized mining control system based on lidar, characterized in that, It includes a data acquisition unit, a dynamic processing unit, a rejection processing unit, and an identification unit; The data acquisition unit is used to acquire point cloud data through a lidar set on a preset track, and to construct a three-dimensional point cloud model of the fully mechanized mining face based on the point cloud data; the preset track is set parallel to the scraper conveyor. The dynamic processing unit is used to dynamically process the three-dimensional point cloud model to obtain a fully mechanized point cloud model. The elimination processing unit is used to eliminate the point cloud of the top plate area and the point cloud of the bottom plate area in the fully mechanized mining point cloud model. The identification unit is used to identify the fully mechanized mining point cloud model after the rejection process to obtain the identification result, and the identification result is used to assist in the control of the fully mechanized mining face.

9. A terminal device, characterized in that, The terminal device includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the integrated mining control method based on lidar as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed on a processor, implements the integrated mining control method based on lidar according to any one of claims 1-7.

Images