Coal mining machine cutting planning method and device based on inspection vision perception

Abstract

A coal mining machine cutting planning method based on inspection vision perception, comprising: on the basis of a space mapping relationship between a mobile platform coordinate system and a geological coordinate system of a stope, digitally constructing a stope space and automatically detecting and recognizing a coal mining machine; perceiving and analyzing the motion state of the coal mining machine to obtain motion state information of the coal mining machine, and capturing and tracking the coal mining machine; and carrying out state perception on a target object of a coal mining machine part in a cutting space, and determining an optimal safe cutting curve on the basis of state perception data. A coal mining machine cutting planning device (300) based on inspection vision perception, comprising a space mapping module (310), a stope space digital construction module (320), a coal mining machine capturing and tracking module (330), and a cutting planning module (340). An electronic device, comprising at least one processor and a memory communicationally connected thereto, for use in executing the steps in the planning method. A non-transitory computer readable storage medium (800) storing computer instructions, for use in causing a computer to execute the steps in the planning method. By means of such configurations, for the fluctuation of a coal seam, the stope and the spatial state of operation equipment can be perceived and captured in real time by means of inspection vision, and the cutting curve can be adaptively adjusted, thereby achieving continuous operation cutting planning, and improving the automated production efficiency of the working face.

Description

Coal mining machine cutting planning method and equipment based on inspection visual perception Technical Field

[0001] The present disclosure relates to the technical field of coal mining, and in particular to a method, device, equipment, and storage medium for coal mining machine cutting planning based on inspection visual perception. Background Art

[0002] Currently, memory-based cutting remains the primary method and means for automatically controlling the height adjustment of the shearer drum during fully mechanized coal mining. Typically, a preliminary plan for shearer cutting is developed based on prior geological survey data. Based on this plan, the shearer operator demonstrates the operation, and the system records relevant shearer status data during the demonstration. Finally, the shearer memory-based cutting system analyzes and processes the acquired data to generate a memory-based cutting curve, providing data support for subsequent shearer cutting operations. However, this current method of automatic shearer height control based on memory-based cutting requires frequent manual intervention to complete the entire cutting process when the coal seam fluctuates significantly, reducing automated production efficiency at the working face. Furthermore, this method and system cannot accurately and in real time perceive the spatial status of the shearer drum and the hydraulic support top beam. This can lead to dangerous anomalies during actual operation, such as the shearer drum cutting the hydraulic support top beam. This necessitates real-time manual monitoring and intervention.

[0003] Based on this, this solution will propose a safer, more accurate, and more efficient coal mining machine cutting planning method based on mobile platform (robot) vision, combined with the reality capture and perception system of coal mine underground scenes. This method can capture the mining site and the spatial status of the operating equipment in real time through patrol vision in response to the fluctuations of the coal seam, adaptively adjust the cutting curve, realize continuous operation cutting planning, and improve the automated production efficiency of the working face. In addition, this method can avoid the dangerous anomaly of the coal mining machine drum cutting the hydraulic support top beam during the autonomous following process based on the perception and capture of the spatial status of the coal mining machine drum and the hydraulic support top beam, thereby improving the level of intelligent safety production control of the working face.

[0004] Summary of the Invention

[0005] The present disclosure provides a coal mining machine cutting planning method, device, equipment, and storage medium based on patrol visual perception, aiming to achieve real-time visual dynamic monitoring of target objects in the mining field environment and provide a perception basis for intelligent mining.

[0006] To this end, the first purpose of this disclosure is to propose a coal mining machine cutting planning method based on inspection visual perception, including:

[0007] A mobile platform with a priori information identification is set up at a specific location in the stope environment to obtain the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system;

[0008] A mobile platform equipped with visual sensors and state perception sensors searches for shearers and digitally constructs the stope space based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system. It also detects and determines whether the shearer is within the visual monitoring range in the stope space, enabling automatic detection and identification of the shearer.

[0009] When a coal mining machine appears within the visual monitoring range, the machine's motion state is perceived and analyzed to obtain the machine's motion state information. Based on the motion state information, the machine is captured and tracked.

[0010] The state of the target objects of the coal mining machine components in the cutting space is perceived, and the optimal safe cutting curve is determined based on the state perception data.

[0011] Among them, a mobile platform equipped with visual sensors and state perception sensors searches for coal shearers and digitally constructs the mining field space based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system. At the same time, it detects and determines whether the coal shearer is within the visual monitoring range in the mining field space. The automatic detection and identification of coal shearers includes the following steps:

[0012] Initialize the mobile platform's motion position, and pre-plan the motion path based on the fully mechanized mining face's working conditions and the target objects detected and identified by the shearer;

[0013] During the movement process, the mobile platform obtains its absolute coordinate information in the geological coordinate system based on the system's spatial coordinate mapping relationship, and realizes real-time positioning of the mobile platform according to the real-time positioning and mapping system (SLAM) based on the visual sensor data and state perception sensor data. The visual sensor data includes data collected by various types of visual cameras and various types of lidars, and the state perception sensor includes data collected by various types of inertial navigation odometers, wheel speedometers, and UWB devices.

[0014] At the same time, the mining space is digitally constructed, and visual perception and intelligent detection are used to determine whether the coal mining machine is within the visual monitoring range, thereby realizing automatic detection and identification of the coal mining machine.

[0015] The steps for realizing automatic detection and identification of the coal mining machine include:

[0016] Using machine learning algorithms, we detect and identify shearers in real time based on visual sensor data. When a shearer appears within the visual monitoring range, the mobile platform senses and analyzes its motion status to obtain information about it.

[0017] Based on the status data of the shearer's positioning, operating speed, and acceleration, the mobile platform is controlled and planned for its motion status;

[0018] Adjust the visual sensor posture in real time to achieve stable capture and tracking of the target object, ensuring that the coal mining machine is always within the system's visual monitoring range;

[0019] When the mobile platform detects an obstacle in the mine during movement, it will perform local obstacle avoidance path planning based on the robot platform model and the local map of the mine, complete autonomous circumvention of the obstacle, and then re-execute the capture and tracking of the coal mining machine.

[0020] The step of performing state perception on the target object of the coal mining machine component in the cutting space includes:

[0021] By combining digitally constructed map data from the stope space, a physical model of the shearer, and the shearer's capture and tracking data, instance segmentation is performed on the shearer drum. Based on the instance segmentation results and combined with the drum's operating status, the positional information of the drum's upper and lower edges, as well as its center, is calculated.

[0022] Real-time detection and identification of the hydraulic support roof and the status of the support panels in the cutting space; combining the digital construction map data of the stope space, the hydraulic support physical model, and the hydraulic support spatial status data to perform instance segmentation of the hydraulic support roof and calculate and obtain the roof beam edge line roof spatial information data;

[0023] Based on the digital construction of the stope space, map data is constructed, instance segmentation is performed on the local coal wall area in the cutting space, and the coal wall spatial surface is calculated.

[0024] The steps of determining the optimal safety cutoff curve based on the state perception data include:

[0025] Combined with the cut space state perception data, artificial intelligence algorithms are used to desensitize, screen and optimize the data, and to build and optimize the digital twin physical simulation model and scenario model;

[0026] Based on the cutting space status perception data, the safety status of the cutting space is judged in real time; when a preset abnormal situation occurs during the tracking process, a cutting safety warning is issued;

[0027] Combining the perception information obtained by calculation in the cutting space, the physical simulation model of the digital twin target object, and the prior information of cutting planning, an optimization algorithm is used to construct a coal mining machine drum cutting height optimization model, and the optimal curve for safe cutting of the coal mining machine is obtained by real-time calculation.

[0028] In the step of performing movement planning prediction based on the moving direction and moving speed information of the target object based on the motion model, the state perception data includes:

[0029] The real-time spatial status of the shearer drum in the cutting space;

[0030] The real-time spatial status of the hydraulic support top plate in the cutting space;

[0031] The spatial surface of the coal wall in the cutting space.

[0032] Among them, the cutting curve optimization and analysis are carried out for the area where the coal mining machine has completed the operation, and the prior information of the cutting plan for the next cutting operation is saved and updated; during the first cutting operation, the cutting plan prior information can be calculated and obtained based on the geological exploration data;

[0033] Based on the perception result information and the optimal safe cutting curve obtained in the cutting space, the digital twin real-scene dynamic simulation is driven to be constructed in real time, and visual interaction is completed, so that staff can perform auxiliary intervention in cutting planning according to scene requirements.

[0034] The second object of the present disclosure is to provide a coal mining machine cutting planning device based on inspection visual perception, comprising:

[0035] The spatial mapping module is used to set up a mobile platform with a priori information identification at a specific location in the stope environment and obtain the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system;

[0036] The stope space digital construction module is used to search for shearers using a mobile platform equipped with visual sensors and state perception sensors. Based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system, the stope space is digitally constructed. At the same time, the module detects and determines whether the shearer is within the visual monitoring range in the stope space, enabling automatic detection and identification of the shearer.

[0037] The shearer capture and tracking module is used to perceive and analyze the shearer's motion state when the shearer appears within the visual monitoring range, obtain the shearer's motion state information, and then capture and track the shearer based on the shearer's motion state information;

[0038] The cutting planning module is used to perceive the state of the target objects of the coal mining machine components in the cutting space, and to determine the optimal safe cutting curve based on the state perception data by combining the digital twin physical simulation model and the scenario model.

[0039] The third object of the present disclosure is to propose an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute each step in the method of the aforementioned technical solution.

[0040] The fourth object of the present disclosure is to propose a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute each step in the method according to the aforementioned technical solution.

[0041] Different from the existing technology, the present disclosure provides a coal mining machine cutting planning method based on patrol visual perception, which obtains the spatial mapping relationship between the coordinate system of the mobile platform of the mining field and the geological coordinate system, performs digital construction of the mining field space, and automatically detects and identifies the coal mining machine in the mining field space; when the coal mining machine appears in the visual monitoring range, it performs perception analysis on the motion state of the coal mining machine, obtains the motion state information of the coal mining machine, and captures and tracks the coal mining machine; performs state perception on the target object of the coal mining machine component in the cutting space, and determines the optimal safe cutting curve based on the state perception data. Through the present disclosure, it is possible to complete real-time cutting planning through visual intelligent perception of the equipment and environmental status of the cutting space, with strong data information timeliness and engineering implementation continuity; it performs real-time visual interaction on the related equipment in the cutting space and the perception results of the surrounding environment status, so that the staff can perform cutting auxiliary intervention in real time and obtain more accurate cutting planning data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present disclosure and / or additional aspects and advantages will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0043] FIG1 is a flow chart of a method for planning coal mining machine cutting based on patrol visual perception provided by the present disclosure.

[0044] FIG2 is a logic diagram of a coal mining machine cutting planning method based on patrol visual perception provided by the present disclosure.

[0045] FIG3 is a block diagram of a coal mining machine cutting planning device based on patrol visual perception provided by the present disclosure.

[0046] FIG4 is a block diagram of an electronic device provided by the present disclosure. DETAILED DESCRIPTION

[0047] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0048] As shown in FIG1 , a method for planning coal cutting for a coal mining machine based on patrol visual perception is provided in an embodiment of the present disclosure. The logic for planning coal cutting for a coal mining machine is shown in FIG2 . The method includes:

[0049] S110: Setting up a mobile platform with a priori information identification at a specific location in the stope environment, and obtaining a spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system.

[0050] Based on the geological model, by setting up markers with prior information at specific locations in the mining environment, the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system is obtained, realizing the integration of the system spatial coordinates.

[0051] S120: A mobile platform equipped with visual sensors and state perception sensors searches for coal mining machines. At the same time, based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system, the mining field space is digitally constructed. In the mining field space, it is detected and determined whether the coal mining machine is within the visual monitoring range, thereby realizing automatic detection and identification of the coal mining machine.

[0052] Initialize the motion position of the mobile platform, combine the working conditions of the fully mechanized mining face, and pre-plan the motion path around the target object detected and identified by the coal mining machine in a specific way and with a specific strategy.

[0053] During the movement, the mobile platform obtains its absolute coordinate information in the geological coordinate system based on the system's spatial coordinate mapping relationship, and realizes real-time positioning of the mobile platform according to the real-time positioning and mapping system (SLAM) based on the visual sensor data and state perception sensor data; wherein, the visual sensor data includes data collected by various types of visual cameras and various types of lidars, and the state perception sensor includes data collected by various types of inertial navigation odometers, wheel speedometers, and UWB devices.

[0054] At the same time, the mining space is digitally constructed, and visual perception and intelligent detection are used to determine whether the coal mining machine is within the visual monitoring range, thereby realizing automatic detection and identification of the coal mining machine.

[0055] S130: When the coal mining machine appears within the visual monitoring range, perception analysis is performed on the movement state of the coal mining machine to obtain the movement state information of the coal mining machine, and the coal mining machine is captured and tracked based on the movement state information of the coal mining machine.

[0056] Using machine learning algorithms, the system detects and identifies shearers in real time based on visual sensor data. When a shearer appears within the visual monitoring range, the mobile platform senses and analyzes its motion state, acquiring this information. The mobile platform then controls and plans its motion based on the shearer's positioning, operating speed, acceleration, and other status data. The system also adjusts the visual sensor's posture in real time to ensure stable target capture and tracking, ensuring the shearer remains within the system's visual monitoring range. If the mobile platform detects an obstacle in the mine during movement, the system performs local obstacle avoidance path planning based on the robot platform model and a local map of the mine, autonomously circumventing the obstacle and then resuming capture and tracking of the shearer.

[0057] S140: Performing state perception on the target object of the coal mining machine component in the cutting space, and determining the optimal safe cutting curve based on the state perception data.

[0058] Based on the stable capture and tracking of the shearer by the mobile platform, the state of target objects such as the shearer drum and the top beam of the hydraulic support in the cutting space is perceived as follows:

[0059] By combining digitally constructed map data from the stope space, the shearer physical model, and the shearer capture and tracking data, we perform instance segmentation on the shearer drum. Based on the instance segmentation results and combined with the drum's operating status, we calculate and obtain information such as the drum's upper and lower edges, as well as its center point.

[0060] Real-time detection and identification of the hydraulic support roof and the status of the support panels in the cutting space. Also, by combining digital construction of the stope space map data, the hydraulic support physical model, and the hydraulic support spatial status data, the hydraulic support roof instance segmentation is performed, and roof spatial information data such as the top beam edge line is calculated.

[0061] Based on the digital construction of the stope space, map data is constructed, instance segmentation is performed on the local coal wall area in the cutting space, and the coal wall spatial surface is calculated.

[0062] Artificial intelligence algorithms are used to perform desensitization, screening and other optimization processing on the data of the cut space state perception data;

[0063] Combined with the cutting space status perception data, the safety status of the cutting space is judged in real time. When abnormal conditions such as the hydraulic support not closing properly, the distance between the shearer drum and the top beam being too close, or the shearer tilting occur during the following process, a cutting safety warning is issued;

[0064] Combining the sensory information obtained by calculation in the cutting space, the physical simulation model of the digital twin target object, and the prior information of the cutting plan, we use machine learning, particle filtering and other optimization algorithms to build a coal mining machine drum cutting height optimization model, and calculate the optimal curve for safe cutting of the coal mining machine in real time. The sensory information used includes:

[0065] ①The real-time spatial status of the shearer drum in the cutting space;

[0066] ②The real-time spatial status of the hydraulic support top plate in the cutting space;

[0067] ③ The spatial surface of the coal wall in the cutting space.

[0068] Optimize and analyze the cutting curve for the area where the shearer has completed its operation, and save and update the prior information for the next cutting operation plan. During the first cutting operation, the cutting plan prior information can be calculated based on geological exploration data;

[0069] Based on the perception result information and the optimal safe cutting curve obtained in the cutting space, the digital twin real-scene dynamic simulation is driven to be constructed in real time, and visual interaction is completed, so that staff can perform auxiliary intervention in cutting planning according to scene requirements.

[0070] Different from the existing technology, the solution disclosed in this disclosure has the following advantages:

[0071] 1. This solution does not require any modification of the operating equipment, and the project implementation is easy and the maintenance cost is low;

[0072] 2. This solution uses the mobile platform (robot) to intelligently perceive the equipment and environmental conditions in the cutting space to complete real-time cutting planning, ensuring strong data and information timeliness and project implementation continuity.

[0073] 3. This solution enables real-time visual monitoring of associated equipment in the cutting space and the surrounding environment, preventing dangerous anomalies such as the shearer drum cutting the top beam of the hydraulic support during autonomous tracking, achieving a high level of intelligent production safety management and control.

[0074] 4. This solution enables the system to conduct real-time visual interaction with the associated equipment in the cutting space and the surrounding environment status perception results, so that staff can conduct cutting assistance intervention in real time and obtain more accurate cutting planning data;

[0075] 5. This system is highly scalable and can be linked with other perception systems to achieve more robust and accurate cutting planning.

[0076] As shown in FIG3 , the present disclosure provides a coal mining machine cutting planning device 300 based on patrol visual perception, comprising:

[0077] The spatial mapping module 310 is used to set up a mobile platform with a priori information identification at a specific location in the stope environment and obtain a spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system;

[0078] The stope space digital construction module 320 is used to digitally construct the stope space based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system during the search for a coal mining machine using a mobile platform equipped with visual sensors and state perception sensors. It also detects and determines whether the coal mining machine is within the visual monitoring range in the stope space, thereby realizing automatic detection and identification of the coal mining machine.

[0079] The shearer capture and tracking module 330 is used to perform perception analysis on the movement state of the shearer when the shearer appears in the visual monitoring range, obtain the movement state information of the shearer, and capture and track the shearer based on the movement state information of the shearer;

[0080] The cutting planning module 340 is used to perform state perception on target objects of the shearer components in the cutting space and determine the optimal safe cutting curve based on the state perception data.

[0081] In order to implement the embodiment, the present disclosure also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute each step of the coal mining machine cutting planning method based on patrol visual perception of the aforementioned technical solution.

[0082] As shown in Figure 4, electronic device 800 includes an instruction memory 810, a processor 820, and an interface 830. The instructions are executed by processor 820 to implement a method for planning coal mining machine cutting based on patrol visual perception. Memory 810, processor 820, and interface 830 are connected via a bus. Interface 830 provides an interface between processor 820 and peripheral interface modules, such as a keyboard, click wheel, and buttons.

[0083] The electronic device 800 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are intended to be examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0084] To implement the embodiments, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the coal mining machine cutting planning based on patrol visual perception, as described in the embodiments of the present disclosure. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0085] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0086] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0087] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0088] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0089] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the embodiments described, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0090] Those skilled in the art will understand that all or part of the steps of the method for implementing the embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0091] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0092] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0093] Although embodiments of the present disclosure have been shown and described above, it is understood that the embodiments are illustrative and are not to be construed as limitations of the present disclosure. A person skilled in the art may change, modify, replace, and alter the embodiments within the scope of the present disclosure.

Claims

1. A coal mining machine cutting planning method based on inspection visual perception, comprising: Setting up a mobile platform with a priori information identification at a specific location in the stope environment, and obtaining a spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system; A mobile platform equipped with visual sensors and state perception sensors searches for shearers and digitally constructs the stope space based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system. It also detects and determines whether the shearer is within the visual monitoring range in the stope space, thus achieving automatic detection and identification of the shearer. When a coal mining machine appears within the visual monitoring range, the machine's motion state is perceived and analyzed to obtain the machine's motion state information. The machine is then captured and tracked based on the motion state information of the coal mining machine, combined with the digital twin physical simulation model and the scene model. The state of the target objects of the coal mining machine components in the cutting space is perceived, and the optimal safe cutting curve is determined based on the state perception data.

2. The method for planning coal mining machine cutting based on patrol visual perception according to claim 1, wherein a mobile platform equipped with a visual sensor and a state perception sensor searches for the coal mining machine, and digitally constructs the mining field space based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system. At the same time, the method detects and determines whether the coal mining machine is within the visual monitoring range in the mining field space, and realizes automatic detection and identification of the coal mining machine, including the following steps: Initialize the mobile platform's motion position, and pre-plan the motion path based on the fully mechanized mining face's working conditions and the target objects detected and identified by the shearer; During the movement, the mobile platform obtains its absolute coordinate information in the geological coordinate system by combining the spatial coordinate mapping relationship of the system. Based on the visual sensor data and the state perception sensor data, the real-time positioning of the mobile platform is realized according to the real-time positioning and mapping system (SLAM). The visual sensor data includes data collected by various types of visual cameras and various types of laser radars, and the state perception sensor includes data collected by various types of inertial navigation odometers, wheel speedometers, and UWB devices; At the same time, the mining space is digitally constructed, and visual perception and intelligent detection are used to determine whether the coal mining machine is within the visual monitoring range, thereby realizing automatic detection and identification of the coal mining machine.

3. The method for coal mining machine cutting planning based on patrol visual perception according to claim 2, wherein the step of realizing automatic detection and identification of the coal mining machine includes: Using machine learning algorithms, we detect and identify shearers in real time based on visual sensor data. When a shearer appears within the visual monitoring range, the mobile platform senses and analyzes its motion status to obtain information about it. Based on the status data of the shearer's positioning, operating speed, and acceleration, the mobile platform is controlled and planned for its motion status; Adjust the visual sensor posture in real time to achieve stable capture and tracking of the target object, ensuring that the coal mining machine is always within the system's visual monitoring range; When the mobile platform detects an obstacle in the mine during movement, it will perform local obstacle avoidance path planning based on the robot platform model and the local map of the mine, complete autonomous circumvention of the obstacle, and then re-execute the capture and tracking of the coal mining machine.

4. The method for planning coal mining machine cutting based on patrol visual perception according to any one of claims 1 to 3, wherein the step of performing state perception on target objects of coal mining machine components in the cutting space comprises: By combining the digital construction of the stope space map data, the shearer physical model, and the shearer capture and tracking result data, instance segmentation of the shearer drum is performed; Based on the instance segmentation results, combined with the roller's operating status, the position information of the roller's upper edge, lower edge, and center point is calculated and obtained; Real-time detection and identification of the hydraulic support top plate and the mutual support plate status in the cutting space; Combined with the digital construction map data of the stope space, the physical model of the hydraulic support, and the spatial status data of the hydraulic support, the hydraulic support roof instance segmentation is performed, and the roof beam edge line roof spatial information data is calculated; Based on the digital construction of the stope space, map data is constructed, instance segmentation is performed on the local coal wall area in the cutting space, and the coal wall spatial surface is calculated.

5. The method for coal mining machine cutting planning based on patrol visual perception according to any one of claims 1 to 4, wherein the step of determining the optimal safe cutting curve based on state perception data comprises: Combined with the cut space state perception data, artificial intelligence algorithms are used to desensitize, screen and optimize the data, and to build and optimize the digital twin physical simulation model and scenario model; Based on the cutting space status perception data, the safety status of the cutting space is judged in real time; When a preset abnormal situation occurs during the machine tracking process, a cutting safety warning will be issued; Combining the perception information obtained by calculation in the cutting space, the physical simulation model of the digital twin target object, and the prior information of cutting planning, an optimization algorithm is used to construct a coal mining machine drum cutting height optimization model, and the optimal curve for safe cutting of the coal mining machine is obtained by real-time calculation.

6. The method for coal mining machine cutting planning based on inspection visual perception according to claim 5, wherein the state perception data includes: The real-time spatial status of the shearer drum in the cutting space; The real-time spatial status of the hydraulic support top plate in the cutting space; The spatial surface of the coal wall in the cutting space.

7. The method for shearer cutting planning based on patrol visual perception according to claim 6, wherein the cutting curve is optimized and analyzed for the area where the shearer has completed its operation, and the prior information is saved and updated as the cutting plan for the next cutting operation; during the first cutting operation, the cutting plan prior information can be calculated based on geological exploration data; Based on the perception result information and the optimal safe cutting curve obtained in the cutting space, the digital twin real scene dynamic simulation is driven Build in real time and complete visual interaction so that staff can assist in cutting and planning interventions based on scenario requirements.

8. A coal mining machine cutting planning device based on inspection visual perception, comprising: A spatial mapping module is used to set up a mobile platform with a priori information identification at a specific location in the stope environment and obtain a spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system; The stope space digital construction module is used to search for shearers using a mobile platform equipped with visual sensors and state perception sensors. Based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system, the stope space is digitally constructed. At the same time, the module detects and determines whether the shearer is within the visual monitoring range in the stope space, thereby realizing automatic detection and identification of the shearer. A shearer capture and tracking module is used to perform perception analysis on the movement state of the shearer when the shearer appears within the visual monitoring range, obtain the movement state information of the shearer, and capture and track the shearer based on the movement state information of the shearer; The cutting planning module is used to perceive the state of the target objects of the coal mining machine components in the cutting space, and to determine the optimal safe cutting curve based on the state perception data by combining the digital twin physical simulation model and the scenario model.

9. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform each step in the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute each step of the method according to any one of claims 1 to 7.

Images