A shearer cutting planning method and device based on inspection visual perception

Abstract

The application provides a coal cutter cutting planning method and equipment based on inspection visual perception, which combines the space mapping relationship between the stope mobile platform coordinate system and the geological coordinate system, carries out stope space digitization construction and automatic detection and identification of the coal cutter, carries out perception analysis on the motion state of the coal cutter, obtains the motion state information of the coal cutter, carries out capture tracking of the coal cutter, carries out state perception on the target object of the coal cutter component in the cutting space, and determines the optimal safe cutting curve based on the state perception data. Through the application, the equipment and environment state of the cutting space can be intelligently perceived and visualized, real-time cutting planning can be completed, the data information timeliness and the engineering implementation continuity are strong, the real-time visual interaction is carried out according to the state perception results of the related equipment in the cutting space and the surrounding environment, so that the staff can carry out cutting auxiliary intervention in real time, and more accurate cutting planning data can be obtained.

Description

Technical Field

[0001] This invention relates to the field of coal mining technology, and in particular to a method, apparatus, equipment, and storage medium for coal mining machine cutting planning based on inspection visual perception. Background Technology

[0002] Currently, in fully mechanized coal mining, memory-based cutting remains the primary technology and method for automatic height control of the coal mining machine drum. Typically, preliminary cutting planning for the coal mining machine is first conducted based on prior geological survey data. Then, based on this, the coal mining machine operator demonstrates the operation, and the system records relevant status data of the coal mining machine during the demonstration operation. Finally, the memory-based cutting system analyzes and processes the acquired data to generate memory-based cutting curves, providing data support for subsequent coal mining machine cutting operations. However, this current method of automatic height control for coal mining machines based on memory-based cutting requires frequent manual intervention to complete the entire cutting process when there are significant variations in the coal seam, reducing the efficiency of automated production at the working face. Furthermore, this method and system cannot accurately sense and capture the spatial status of the coal mining machine drum and the hydraulic support top beam in real time, which may lead to dangerous abnormalities such as the coal mining machine drum cutting the hydraulic support top beam during actual operation, thus requiring real-time manual monitoring and intervention.

[0003] Based on this, this solution proposes a safer, more accurate, and more efficient coal mining machine cutting planning method based on mobile platform (robot) vision and combined with a coal mine underground scene reality capture and perception system. This method can adaptively adjust the cutting curve to adapt to changes in coal seam undulations, capturing real-time perception of the mining area and the spatial status of the operating equipment, achieving continuous operation cutting planning and improving the efficiency of automated production at the working face. Furthermore, based on the perception and capture of the spatial status of the coal mining machine drum and the hydraulic support top beam, this method avoids dangerous abnormalities caused by the coal mining machine drum cutting the hydraulic support top beam during autonomous following, thus improving the level of intelligent safety production management at the working face. Summary of the Invention

[0004] This invention provides a method, device, equipment, and storage medium for coal mining machine cutting planning based on inspection visual perception, aiming to achieve real-time visual dynamic monitoring of target objects in the mining environment and provide a perception basis for intelligent mining.

[0005] Therefore, the first objective of this invention is to propose a coal mining machine cutting planning method based on inspection visual perception, comprising:

[0006] A mobile platform with prior information identifiers is set up at a specific location in the mining environment to obtain the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system;

[0007] A mobile platform equipped with visual sensors and state perception sensors searches for coal mining machines and digitally constructs the mining area based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system. At the same time, it detects and determines whether the coal mining machine is within the visual monitoring range in the mining area, thereby realizing the automatic detection and identification of the coal mining machine.

[0008] Once the coal mining machine appears within the visual monitoring range, the motion status of the coal mining machine is perceived and analyzed to obtain its motion status information. Based on this information, the coal mining machine is captured and tracked.

[0009] The system performs state perception on target objects of the coal mining machine components in the cutting space, and determines the optimal safe cutting curve based on the state perception data.

[0010] The mobile platform, equipped with visual sensors and state perception sensors, searches for the coal mining machine and, based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system, digitally constructs the mining area. Simultaneously, it detects and determines whether the coal mining machine is within the visual monitoring range within the mining area, achieving automatic detection and identification of the coal mining machine. This process includes the following steps:

[0011] Initialize the movement position of the mobile platform, and based on the working conditions of the fully mechanized mining face, pre-plan the movement path around the target object detected and identified by the coal mining machine;

[0012] During the movement, the mobile platform obtains its absolute coordinate information in the geological coordinate system by combining the system's spatial coordinate mapping relationship. Based on visual sensor data and state perception sensor data, the mobile platform achieves real-time positioning according to the real-time positioning and mapping system (SLAM). The visual sensor data includes data collected by various types of visual cameras and various types of lidar, and the state perception sensor data includes data collected by various types of inertial navigation odometry, wheel speedometers, and UWB devices.

[0013] Simultaneously, the mining area is digitally constructed, and visual perception is used to intelligently detect and determine whether the coal mining machine is within the visual monitoring range, thereby achieving automatic detection and identification of the coal mining machine.

[0014] The steps for achieving automatic detection and identification of coal mining machines include:

[0015] The machine learning algorithm is used to detect and identify the coal mining machine in real time based on the visual sensor data. When the coal mining machine appears in the visual monitoring range, the mobile platform performs perception and analysis on the movement status of the coal mining machine to obtain the movement status information of the coal mining machine.

[0016] Based on the status data of the coal mining machine's positioning, operating speed, and acceleration, motion status control and planning are carried out for the mobile platform.

[0017] The posture of the vision sensor is adjusted in real time to achieve stable capture and tracking of the target object, ensuring that the coal mining machine is always within the system's vision monitoring range;

[0018] When the mobile platform detects an obstacle in the mining area during its movement, it will perform local obstacle avoidance path planning based on the robot platform model and the local map of the mining area, complete the autonomous detour around the obstacle, and then re-execute the capture and tracking of the coal mining machine.

[0019] The step of state perception for target objects of the coal mining machine components in the cutting space includes:

[0020] By combining the digital construction map data of the mining area space, the physical model of the coal mining machine, and the coal mining machine capture and tracking results data, instance segmentation is performed on the coal mining machine drum; based on the instance segmentation results, analysis is performed, and combined with the drum's operating status, the position information data of the upper edge, lower edge, and center point of the drum are calculated and obtained.

[0021] Real-time detection and identification of the hydraulic support roof and mutual support plate status in the cutting space; combined with the digital construction of map data of the mining space, the physical model of the hydraulic support, and the spatial status data of the hydraulic support, the hydraulic support roof instance segmentation is performed, and the spatial information data of the roof edge line of the top beam is calculated and obtained.

[0022] Based on the digital construction of map data of the mining space, the local coal wall area in the cutting space is segmented into instances, and the curved surface of the coal wall space is calculated and obtained.

[0023] The steps for determining the optimal safe cut-off curve based on state-aware data include:

[0024] By combining truncated spatial state perception data with artificial intelligence algorithms for data desensitization, screening and optimization, digital twin physical simulation models and scene models are constructed and optimized.

[0025] Based on the status perception data of the cutting space, the safety status of the cutting space is judged in real time; when a preset abnormal situation occurs during the follow-up process, a cutting safety warning is issued.

[0026] By combining the perception information obtained from calculations within the cutting space, the physical simulation model of the digital twin target object, and the prior information of the cutting plan, an optimization algorithm is used to construct an optimization model for the cutting height of the coal mining machine drum, and the optimal curve for safe cutting of the coal mining machine is calculated in real time.

[0027] In the step of motion planning and prediction based on the motion model to determine the target object's direction and speed, the state-aware data includes:

[0028] Real-time spatial status of the coal mining machine drum in the cutting space;

[0029] Real-time spatial status of the hydraulic support top plate in the cutting space;

[0030] The spatial curved surface of the coal wall in the sectional space.

[0031] Specifically, the cutting curve of the area already worked by the coal mining machine is optimized and analyzed, and the prior information for the next cutting operation is saved and updated; during the first cutting operation, the prior information for the cutting plan can be calculated and obtained based on geological survey data.

[0032] Based on the perception results information obtained in the cut space and the optimal safe cut curve, the digital twin real-scene dynamic simulation is driven to be constructed in real time, and the visualization interaction is completed so that the staff can make auxiliary interventions for cut planning according to the needs of the scene.

[0033] The second objective of this invention is to provide a coal mining machine cutting planning device based on inspection visual perception, comprising:

[0034] The spatial mapping module is used to set up a mobile platform with prior information identifiers at a specific location in the mining environment and obtain the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system.

[0035] The mining space digital construction module is used to search for coal mining machines through a mobile platform equipped with visual sensors and status perception sensors, and to digitally construct the mining space based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system. At the same time, it detects and determines whether the coal mining machine is within the visual monitoring range in the mining space, thereby realizing the automatic detection and identification of the coal mining machine.

[0036] The coal mining machine capture and tracking module is used to perceive and analyze the movement status of the coal mining machine when it appears in the visual monitoring range, obtain the movement status information of the coal mining machine, and capture and track the coal mining machine based on the movement status information.

[0037] The cutting planning module is used to perform state perception on target objects of coal mining machine components in the cutting space, and to determine the optimal safe cutting curve based on the state perception data by combining the digital twin physical simulation model and the scene model.

[0038] A third objective of the present invention is to provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method described above.

[0039] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the steps of the method according to the foregoing technical solution.

[0040] Unlike existing technologies, this invention provides a coal mining machine cutting planning method based on visual perception during inspection. It acquires the spatial mapping relationship between the coordinate system of the moving platform in the mining area and the geological coordinate system, digitally constructs the mining area space, and automatically detects and identifies the coal mining machine within this space. Once the coal mining machine appears within the visual monitoring range, its movement status is perceived and analyzed to obtain its movement status information, enabling capture and tracking. The method also perceives the status of target objects within the cutting space and determines the optimal safe cutting curve based on the perceived status data. This invention enables real-time cutting planning by visually and intelligently perceiving the equipment and environmental status of the cutting space, providing strong data timeliness and engineering continuity. Real-time visualization and interaction of the perceived status of related equipment and the surrounding environment within the cutting space allow for real-time intervention by personnel to assist in cutting and obtain more accurate cutting planning data. Attached Figure Description

[0041] The present invention and / or its additional aspects and advantages will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0042] Figure 1 This is a flowchart illustrating a coal mining machine cutting planning method based on inspection visual perception provided by the present invention.

[0043] Figure 2 This is a logical schematic diagram of a coal mining machine cutting planning method based on inspection visual perception provided by the present invention.

[0044] Figure 3 This is a schematic diagram of the structure of a coal mining machine cutting and planning device based on inspection visual perception provided by the present invention.

[0045] Figure 4 This is a schematic diagram of the structure of a non-transitory computer-readable storage medium storing computer instructions provided by the present invention. Detailed Implementation

[0046] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0047] like Figure 1 As shown, this is an embodiment of the coal mining machine cutting planning method based on inspection visual perception provided by the present invention. The logic of the present invention for coal mining machine cutting planning is as follows: Figure 2 As shown, the method includes:

[0048] S110: Establish a mobile platform with prior information identifiers at a specific location in the mining environment, and obtain the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system.

[0049] Based on the geological model, the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system is obtained by setting up prior information markers at specific locations in the mining environment, thereby realizing the integration of system spatial coordinates.

[0050] S120: A mobile platform equipped with visual sensors and status perception sensors searches for coal mining machines. At the same time, based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system, it digitally constructs the mining area space and detects and determines whether the coal mining machine is within the visual monitoring range in the mining area, thereby realizing the automatic detection and identification of the coal mining machine.

[0051] Initialize the movement position of the mobile platform, and based on the working conditions of the fully mechanized mining face, pre-plan the movement path around the target object detected and identified by the coal mining machine in a specific way and with a specific strategy.

[0052] During the movement, the mobile platform obtains its absolute coordinate information in the geological coordinate system by combining the system's spatial coordinate mapping relationship. Based on visual sensor data and state perception sensor data, the mobile platform achieves real-time positioning according to the real-time positioning and mapping system (SLAM). The visual sensor data includes data collected by various types of visual cameras and various types of lidar, and the state perception sensor data includes data collected by various types of inertial navigation odometers, wheel speedometers, and UWB devices.

[0053] Simultaneously, the mining area is digitally constructed, and visual perception is used to intelligently detect and determine whether the coal mining machine is within the visual monitoring range, thereby achieving automatic detection and identification of the coal mining machine.

[0054] S130: When the coal mining machine appears within the visual monitoring range, the motion status of the coal mining machine is perceived and analyzed to obtain the motion status information of the coal mining machine. Based on the motion status information of the coal mining machine, the coal mining machine is captured and tracked.

[0055] Machine learning algorithms are used to detect and identify the coal mining machine in real time based on visual sensor data. Once the coal mining machine appears within the visual monitoring range, the mobile platform perceives and analyzes its motion status to obtain motion status information. Based on the coal mining machine's positioning, operating speed, acceleration, and other status data, the mobile platform performs corresponding motion control and planning. Simultaneously, the posture of the visual sensors is adjusted in real time to achieve stable target acquisition and tracking, ensuring the coal mining machine remains within the system's visual monitoring range. When the mobile platform detects obstacles in the mining area during movement, the system plans a local obstacle avoidance path based on the robot platform model and the local map of the mining area, autonomously avoiding the obstacle, and then re-executes the coal mining machine acquisition and tracking.

[0056] S140: Perform state perception on target objects of coal mining machine components in the cutting space, and determine the optimal safe cutting curve based on the state perception data.

[0057] Based on the stable capture and tracking of the coal mining machine on the mobile platform, status perception is performed on target objects such as the coal mining machine drum and the hydraulic support top beam in the cutting space, as detailed below:

[0058] By combining digitally constructed map data of the mining area, physical model of the coal mining machine, and tracking data of the coal mining machine, instance segmentation is performed on the coal mining machine drum. Based on the instance segmentation results, analysis is conducted, and information such as the upper edge, lower edge, and center point position of the drum are calculated and obtained in conjunction with the drum's operating status.

[0059] Real-time detection and identification of the hydraulic support roof and interlocking plate status in the cutting space. Similarly, by combining the digital construction map data of the stope space, the physical model of the hydraulic support, and the spatial status data of the hydraulic support, the hydraulic support roof instance is segmented, and the roof spatial information data such as the edge line of the top beam is calculated and obtained;

[0060] Based on the digital construction of map data of the mining space, the local coal wall area in the cutting space is segmented into instances, and the curved surface of the coal wall space is calculated and obtained.

[0061] Artificial intelligence algorithms are used to optimize and process truncated spatial state perception data, including data desensitization and filtering.

[0062] By combining the status perception data of the cutting space, the safety status of the cutting space is judged in real time. When abnormal situations occur during the operation, such as the hydraulic support not closing properly, the distance between the coal mining machine drum and the top beam being too close, or the coal mining machine tilting, a cutting safety warning is issued.

[0063] Combining sensory information obtained from calculations within the cutting space, a digital twin physical simulation model of the target object, and prior information on cutting planning, an optimization model for the cutting height of the coal mining machine's drum is constructed using optimization algorithms such as machine learning and particle filtering. This allows for real-time calculation of the optimal safe cutting curve for the coal mining machine. The sensory information used includes:

[0064] ① The real-time spatial status of the coal mining machine drum in the cutting space;

[0065] ② The real-time spatial status of the hydraulic support top plate in the cutting space;

[0066] ③The spatial curved surface of the coal wall in the sectional space.

[0067] The cutting curve of the area already worked by the coal mining machine is optimized and analyzed, and the prior information for the next cutting operation is saved and updated. For the first cutting operation, the prior information for the cutting plan can be calculated based on geological survey data.

[0068] Based on the perception results information obtained in the cut space and the optimal safe cut curve, the digital twin real-scene dynamic simulation is driven to be constructed in real time, and the visualization interaction is completed so that the staff can make auxiliary interventions for cut planning according to the needs of the scene.

[0069] Unlike existing technologies, the solution of this invention has the following advantages:

[0070] 1. This solution requires no modification to the operating equipment, making the project easy to implement and low in maintenance costs;

[0071] 2. This solution uses a mobile platform (robot) to visually and intelligently perceive the equipment and environmental conditions of the cutting space to complete real-time cutting planning, which has strong data information timeliness and engineering implementation continuity;

[0072] 3. This solution can monitor the associated equipment in the cutting space and the surrounding environment in real time, avoiding dangerous abnormalities such as the coal mining machine drum cutting the top beam of the hydraulic support during autonomous follow-up, and has a high level of intelligent management and control for safe production;

[0073] 4. This solution enables the system to visualize and interact in real time with the perception results of the associated equipment in the cut space and the surrounding environment, so that staff can make real-time cut-off assistance interventions and obtain more accurate cut-off planning data;

[0074] 5. This solution system has high scalability and can be linked with other sensing systems to achieve more robust and accurate cut-off planning.

[0075] like Figure 3 As shown, the present invention provides a coal mining machine cutting and planning device 300 based on inspection visual perception, comprising:

[0076] The spatial mapping module 310 is used to set up a mobile platform with prior information identification at a specific location in the mining environment and obtain the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system.

[0077] The mining space digital construction module 320 is used to construct the mining space digitally based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system during the coal mining machine search process using a mobile platform equipped with a visual sensor and a status perception sensor. It can also detect and determine whether the coal mining machine is within the visual monitoring range in the mining space, thereby realizing the automatic detection and identification of the coal mining machine.

[0078] The coal mining machine capture and tracking module 330 is used to perceive and analyze the movement status of the coal mining machine when it appears in the visual monitoring range, obtain the movement status information of the coal mining machine, and capture and track the coal mining machine based on the movement status information of the coal mining machine.

[0079] The cutting planning module 340 is used to perform state perception on target objects of coal mining machine components in the cutting space and determine the optimal safe cutting curve based on the state perception data.

[0080] To implement the embodiments, the present invention also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps in the coal cutting planning method based on inspection visual perception of the aforementioned technical solution.

[0081] The non-transitory computer-readable storage medium 800 includes an instruction memory 810 and an interface 830, the instructions of which can be executed by a coal mining machine cutting planning processor 820 based on inspection vision perception to complete the method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0082] To implement the embodiments, the present invention also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the coal mining machine cutting plan based on inspection visual perception as described in the embodiments of the present invention.

[0083] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0084] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0085] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0086] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0087] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the described embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0088] Those skilled in the art will understand that all or part of the steps of the method described in the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0089] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0090] The storage medium mentioned may be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the embodiments within the scope of the present invention.

Claims

1. A shearer cutting planning method based on inspection visual perception, characterized in that, The method comprises the following steps: setting a mobile platform with prior information identification at a specific position in a mining environment, and obtaining a spatial mapping relationship between a coordinate system of the mobile platform and a geological coordinate system; the mobile platform carrying a visual sensor and a state perception sensor searches for a coal mining machine, and based on the spatial mapping relationship between the coordinate system of the mobile platform and the geological coordinate system, a mining space is digitally constructed, and it is detected and judged whether the coal mining machine is in the visual monitoring range in the mining space, so as to realize automatic detection and identification of the coal mining machine, wherein the state perception sensor comprises various types of inertial navigation odometry, wheel speed meter and UWB device; when the coal mining machine appears in the visual monitoring range, the motion state of the coal mining machine is perceived and analyzed, the motion state information of the coal mining machine is obtained, a digital twin physical simulation model and a scene model are combined, and based on the motion state information of the coal mining machine, the coal mining machine is captured and tracked; the target object of the coal mining machine component in the cutting space is perceived, and the optimal safe cutting curve is determined based on the state perception data.

2. The shearer cutting planning method based on patrol visual perception according to claim 1, characterized in that, The mobile platform carrying a visual sensor and a state perception sensor searches for a coal mining machine, and based on the spatial mapping relationship between the coordinate system of the mobile platform and the geological coordinate system, a mining space is digitally constructed, and it is detected and judged whether the coal mining machine is in the visual monitoring range in the mining space, so as to realize automatic detection and identification of the coal mining machine, which comprises the following steps: initializing the motion position of the mobile platform, combining the working conditions of the fully-mechanized coal mining face, and pre-planning the motion path around the detection and identification target object of the coal mining machine; in the motion process, the mobile platform obtains the absolute coordinate information of itself in the geological coordinate system based on the spatial coordinate mapping relationship of the system, realizes real-time positioning of the mobile platform based on visual sensor data and state perception sensor data according to a real-time positioning and mapping system (SLAM); wherein the visual sensor data comprises data collected by various types of visual cameras and various types of laser radars, and the state perception sensor data comprises data collected by various types of inertial navigation odometry, wheel speed meter and UWB device; at the same time, the mining space is digitally constructed, it is intelligently detected and judged whether the coal mining machine is in the visual monitoring range by visual perception, and automatic detection and identification of the coal mining machine are realized.

3. The shearer cutting planning method based on patrol visual perception according to claim 2, characterized in that, In the step of realizing automatic detection and identification of the coal mining machine, the following steps are included: the coal mining machine is detected and identified in real time by a machine learning algorithm based on the visual sensor data, when the coal mining machine appears in the visual monitoring range, the motion state of the coal mining machine is perceived and analyzed by the mobile platform, and the motion state information of the coal mining machine is obtained; according to the state data of the positioning, running speed and acceleration of the coal mining machine, the motion state control and planning of the mobile platform are carried out; the posture of the visual sensor is adjusted in real time, the stable capture and tracking of the target object are realized, and the coal mining machine is ensured to be always in the visual monitoring range of the system; when the mobile platform detects an obstacle in the mining field during the movement, local obstacle avoidance path planning is carried out based on the robot platform model and the local map of the mining field, the obstacle is autonomously bypassed, and then the capture and tracking of the coal mining machine are re-executed.

4. The shearer cutting planning method based on patrol visual perception according to claim 1, characterized in that, In the step of perceiving the target object of the coal mining machine component in the cutting space, the following steps are included: By combining the digital construction map data of the mining area space, the physical model of the coal mining machine, and the coal mining machine capture and tracking results data, instance segmentation is performed on the coal mining machine drum; based on the instance segmentation results, analysis is performed, and combined with the drum's operating status, the position information data of the upper edge, lower edge, and center point of the drum are calculated and obtained. Real-time detection and identification of the hydraulic support roof and side protection plate status in the cutting space; combined with the digital construction map data of the mining space, the physical model of the hydraulic support, and the spatial status data of the hydraulic support, the hydraulic support roof instance segmentation is performed, and the spatial information data of the roof edge line of the top beam is calculated and obtained. Based on the digital construction of map data of the mining space, the local coal wall area in the cutting space is segmented into instances, and the curved surface of the coal wall space is calculated and obtained.

5. The shearer cutting planning method based on patrol visual perception according to claim 3, characterized in that, The steps for determining the optimal safe cutoff curve based on state-aware data include: By combining truncated spatial state perception data with artificial intelligence algorithms for data desensitization, screening and optimization, digital twin physical simulation models and scene models are constructed and optimized. Based on the status perception data of the cutting space, the safety status of the cutting space is judged in real time; when a preset abnormal situation occurs during the follow-up process, a cutting safety warning is issued. By combining the perception information obtained from calculations within the cutting space, the physical simulation model of the digital twin target object, and the prior information of the cutting plan, an optimization algorithm is used to construct an optimization model for the cutting height of the coal mining machine drum, and the optimal curve for safe cutting of the coal mining machine is calculated in real time.

6. The shearer cutting planning method based on patrol visual perception according to claim 5, characterized in that, The state-aware data includes: Real-time spatial status of the coal mining machine drum in the cutting space; Real-time spatial status of the hydraulic support top plate in the cutting space; The spatial curved surface of the coal wall in the sectional space.

7. The shearer cutting planning method based on patrol visual perception according to claim 6, characterized in that, For the coal mining machine's completed work area, the cutting curve is optimized and analyzed, and the prior information for the next cutting operation is saved and updated; during the first cutting operation, the prior information for the cutting plan can be calculated and obtained based on geological survey data; Based on the perception results information obtained in the cut space and the optimal safe cut curve, the digital twin real-scene dynamic simulation is driven to be constructed in real time, and the visualization interaction is completed so that the staff can make auxiliary interventions for cut planning according to the needs of the scene.

8. A shearer cutting planning device based on visual perception of inspection, characterized in that, include: The spatial mapping module is used to set up a mobile platform with prior information identifiers at a specific location in the mining environment and obtain the spatial mapping relationship between the coordinate system of the mobile platform and the geological coordinate system. The mining area space digital construction module is used to search for coal mining machines through a mobile platform equipped with visual sensors and state perception sensors. Based on the spatial mapping relationship between the mobile platform coordinate system and the geological coordinate system, it performs digital construction of the mining area space. At the same time, it detects and determines whether the coal mining machine is within the visual monitoring range in the mining area space, realizing the automatic detection and identification of the coal mining machine. The state perception sensors include various types of inertial navigation odometers, wheel speed meters, and UWB devices. The coal mining machine capture and tracking module is used to perceive and analyze the movement status of the coal mining machine when it appears in the visual monitoring range, obtain the movement status information of the coal mining machine, and capture and track the coal mining machine based on the movement status information of the coal mining machine. The cutting planning module is configured to perform state sensing on a target object of a component of the coal mining machine in a cutting space, and combine a digital twin physical simulation model and a scene model to determine an optimal safe cutting curve based on state sensing data.

9. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform each step in the method of any one of claims 1-7.

10. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are configured to cause the computer to perform each step in the method of any one of claims 1-7.

Images