Mining and transporting equipment double-person-double-machine measuring method

Abstract

The invention discloses a mining and transportation equipment double-person-double-machine measurement method. An intelligent measurement module, a measurement user module and a measurement task module are included. The intelligent measurement module comprises two inspection personnel which carry out inspection work in opposite directions and AR glasses carried by the inspection personnel, and collects data in a manual matching and automatic tracking mode based on Hololens equipment; the measurement user module is arranged in an edge computer carried by inspection personnel and comprises a multi-user cooperation platform; a multi-user system platform measures working face mining and transporting pose data under the assistance of AR and carries out data processing operation, and the multi-user system platform coordinates measurement view angles and data among multiple users in real time; and the measurement task module is responsible for defining a measurement task for the multi-user cooperation platform, resolving data fed back by the measurement task, and synchronously feeding a resolving result back to the inspection AR end in a task form, thereby realizing rapid and efficient measurement of the posture of the mining and transportation equipment based on the dual-Hololens equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent inspection of fully mechanized mining working faces, and in particular to a two-person-two-machine measurement method for mining and transportation equipment driven by the fusion of manual matching and automatic tracking. Background Art

[0002] Intelligent coal mining is the core technical support for the high-quality development of the coal industry and the only way for the transformation and upgrading of the coal mining industry. At present, coal mining is still in the stage of partial unmanned development, and people still play an important role in the comprehensive mining system. The coal industry has a high demand for the development of intelligent coal mine comprehensive mining working faces, especially in the comprehensive mining inspection end. The existing mechanical sensors and SLAM point cloud technology measurement technologies are difficult to guarantee the measurement effect in narrow working faces, which will cause the comprehensive mining equipment to be difficult to locate and the posture to be difficult to measure. Due to the existence of this situation, not only the inspection efficiency is difficult to guarantee, but also the subsequent dynamic inspection work is difficult to promote and develop. Therefore, the development of AR measurement plays a vital role in the promotion of future intelligent inspections.

[0003] The invention patent with the publication number CN114186403A discloses a real-time and accurate virtual deduction method for the position and posture of the floating connection mechanism of comprehensive mining and supporting equipment. This invention proposes a real-time and accurate virtual deduction method for the position and posture of the floating connection mechanism of comprehensive mining and supporting equipment. First, based on conformal geometry, the spatial combined motion of the floating connection mechanism is converted into the transformation relationship between points, lines and surfaces to obtain the motion law of the floating connection mechanism; then, by changing the rotation of each middle slot of the scraper conveyor, the change of the direction vector is realized, and then the motion value of each structure of the floating connection mechanism is updated, and finally the adaptive propulsion of the hydraulic support group and the scraper conveyor is realized; the scraper conveyor is discretely decomposed into several middle slots, driven based on the rotation of each middle slot, and the scraper conveyor posture reconstruction based on coordinate information is established. This invention gets rid of the influence of the movement order of different degrees of freedom of the floating connection mechanism, and can directly solve the movement of each structure. After virtual application, it can reconstruct the movement of the floating connection mechanism more quickly, and then can deduce the posture relationship between the hydraulic support and the scraper conveyor more quickly. However, complete virtual deduction is difficult to cope with the real complex conditions underground. This patent proposes a measurement method of manual matching and automatic tracking fusion drive, which eliminates traditional physical sensors, improves human subjectivity, and more accurately presents the real situation underground.

[0004] The invention patent with the publication number CN113051756A discloses a virtual-real fusion coal mining machine memory cutting test system and method, and provides a virtual-real fusion coal mining machine memory cutting test system and method, which includes a virtual test scene, a virtual-real interaction system and a test bench. The coal mining machine is installed on the test bench, and the worker operates the coal mining machine to demonstrate the knife operation. The sensor installed on the coal mining machine transmits the signal to the virtual test scene in real time, drives the virtual coal mining machine to run in the virtual coal seam and displays various data for the workers' reference. The autonomous prediction data of memory cutting comes from the real coal mining machine controller, drives the virtual coal mining machine to run and presents the memory cutting effect on the test bench. The prediction data can be exported for comparison and analysis with the manual operation data to verify the effect of memory cutting. This solution solves many problems that the current test memory cutting can only be carried out in the actual underground environment, the test process consumes huge manpower and material resources, the actual underground environment has many risk factors, the risk factor is high, the test is difficult and the feasibility is low. This patent uses virtual-reality fusion technology to construct different comprehensive mining scenes, but it is still far from the real comprehensive mining scenes; this patent gives full play to people's subjective initiative on the basis of virtual-reality fusion, and experienced inspection personnel can flexibly calibrate and measure on the working surface to make up for the deficiencies in virtual scene simulation.

[0005] The invention patent with the publication number CN117540518A discloses an underground pipeline inspection equipment and method based on the fusion of three-dimensional real scene and virtual reality. The invention discloses an underground pipeline inspection equipment and method based on the fusion of three-dimensional real scene and virtual reality, which relates to the field of three-dimensional real scene and virtual reality fusion inspection technology, including a three-dimensional real scene acquisition device, a data processing unit, a virtual model generation unit and a fusion display unit. The three-dimensional real scene acquisition device is used to collect the real scene data of the underground pipeline, and the real scene data includes environmental three-dimensional parameters, underground pipeline location data and underground pipeline three-dimensional contour data. The three-dimensional real scene acquisition device transmits the real scene data to the data processing unit. The present invention realizes the comprehensive and accurate acquisition and modeling of the actual scene of the underground pipeline through the three-dimensional real scene acquisition device and the data processing unit; through the virtual model generation unit, the type, location and direction information of the underground pipeline are automatically identified, and a virtual model corresponding to the actual scene is generated; through the fusion display unit, the three-dimensional real scene and the virtual model are fused and displayed, realizing real-time and accurate underground pipeline inspection. This patent uses an electric car as the inspection executor, which is equivalent to a single-user inspection. This patent adopts a two-person and multi-user inspection strategy and designs a multi-user collaborative platform to provide assistance for future promotion to multiple people or multiple cars.

[0006] The invention patent with the publication number CN115456361A discloses a patrol inspector abnormal state detection management method, system and storage medium. The invention relates to a patrol inspector abnormal state detection management method, system and storage medium, which belongs to the field of patrol inspection. The method includes obtaining the position of the patrol inspector in real time based on the patrol inspector's intelligent terminal; judging whether the position of the patrol inspector has changed within a preset time period; if the position of the patrol inspector has not changed, judging whether the intelligent terminal is online; if the intelligent terminal is online, obtaining the preset terminal position; judging whether the interval distance between the position and the terminal position is less than the preset distance threshold; if the interval distance is not less than the distance threshold, judging that the patrol inspector has an abnormal situation, and sending a prompt message to the intelligent terminal; generating and displaying the working trajectory based on the working time node based on the position obtained in real time before the abnormal situation of the patrol inspector occurs. The patent uses the terminal power and running memory occupancy rate of the intelligent terminal to determine whether the patrol inspection condition is abnormal, while this patent proposes a patrol inspector posture calculation method to determine the patrol inspector's working path, which is more intuitive than the above patent solution logic.

[0007] The invention patent with publication number CN118017461A discloses a distribution station area topology identification and line loss analysis method based on cloud-edge collaboration. The invention discloses a distribution station area topology identification and line loss analysis method based on cloud-edge collaboration. It includes the following steps: the cloud server establishes an initial topology identification and line loss analysis model, and sends the initial parameters of the model to each intelligent fusion terminal participating in the joint modeling; the intelligent fusion terminal completes the training of the lightweight computing model locally, and after the training is completed, the model parameters are uploaded to the cloud server; the cloud server aggregates and optimizes the model parameters of each intelligent fusion terminal; the cloud server updates the global model once according to the aggregated results, and returns the updated model parameters to the intelligent fusion terminals participating in the modeling; the intelligent fusion terminal updates and iterates and optimizes the local model again until the performance reaches the target. The method of the present invention integrates distributed collaborative computing with cloud-edge collaborative computing, which can be applied to the application scenarios of efficient topology identification and real-time line loss analysis of large-scale low-voltage distribution networks, and supports the construction of new power systems. This patent still adopts the cloud-edge collaborative strategy, with the AR end as the measurement end to measure and integrate the initial parameters; the VR end is the subsequent solution end; after the AR end completes the basic data measurement and preprocessing locally, the cloud will calculate the mining and transportation equipment posture and the mining and transportation-support equipment coordination posture in detail, and finally feedback to the front-line inspection personnel; the biggest difference between this patent and the target patent is that it emphasizes the people-oriented concept, and the inspection personnel serve as the on-site decision-making terminal.

[0008] The posture detection mentioned in the above patents is all in the virtual simulation stage. If it is to be applied to real production, it is necessary to further promote the virtual simulation to the actual working surface. Summary of the invention

[0009] The purpose of the present invention is to provide a two-person-two-machine measurement method for mining and transportation equipment, which integrates manual matching with automatic tracking technology, adopts a multi-person inspection and synchronous inspection strategy, and realizes collaborative inspection between dual Hololens devices.

[0010] To achieve the above object, the technical solution adopted by the present invention is: A two-person-two-machine measurement method for mining and transportation equipment, comprising an intelligent measurement module, a measurement user module and a measurement task module; The intelligent measurement module includes two inspectors who conduct inspections in opposite directions and AR glasses carried by the inspectors, which collect the working face mining posture data by manual matching and automatic tracking based on Hololens equipment; The measurement user module is built into the edge computer carried by the inspection personnel, including a multi-user collaborative platform; the multi-user system platform measures the AR-assisted working face mining posture data and performs data processing operations, and the multi-user system platform coordinates the measurement perspectives and data between multiple users in real time; The measurement task module is responsible for clarifying the measurement tasks for the multi-user collaborative platform and for solving the data fed back by the measurement tasks. Driven by the task, a virtual solution platform is built according to the actual working status of the mining and transportation equipment. The posture information of the scraper conveyor and the coal mining machine is solved in real time according to the measurement data fed back by the user, and the posture information of the hydraulic support group in the multi-user collaborative platform is matched and solved, so as to integrate the relationship between the mining and transportation equipment and the support equipment, and synchronously feed back the solution results to the inspection AR terminal in the form of tasks.

[0011] As a preferred implementation, the multi-user system platform is based on the mining and transportation posture data of the working face assisted by AR, integrates multiple Hololens platform collaboration modules and network interaction modules, uses Vuforia tools to perform real-scene scanning of the comprehensive mining equipment on the working face, builds a virtual model on the virtual end and integrates it into the AR end, and uses the AR device to identify and track the comprehensive mining equipment running in the real working face, and measures the posture information of the corresponding parts in real time.

[0012] As a preferred implementation, it includes establishing a three-level coordinate system in the fully mechanized mining face; The primary coordinate system is the coal seam coordinate system, with the initial mining position of the coal seam as the origin of the coal seam coordinate system, the working face layout direction as the X-axis, the horizontal advancement direction as the Y-axis, and the vertical direction as the Z-axis, which is used to locate all coordinate systems in the working face; The secondary coordinate system is the working face coordinate system composed of three fully mechanized mining machines. The geometric center of the scraper conveyor head section of the central symmetric plane of the first end support in the scraper conveyor head direction is taken as the origin, and the coordinate axis direction is always parallel to the coal seam coordinate system. The three-level coordinate system is the local coordinate system of each equipment and the AR coordinate system. The coal mining machine takes the geometric center of the fuselage as the origin of the coordinate system, the direction parallel to the fuselage as the X-axis, the direction perpendicular to the front and rear plane of the fuselage as the Y-axis, and the direction perpendicular to the upper plane of the fuselage as the Z-axis; the scraper conveyor is first divided into several units according to the number of middle trough sections. Each middle trough unit takes the geometric center of the plane on the scraper trough as the origin of the coordinate system, the direction parallel to the bottom plate pointing to the tail of the machine as the X-axis, the direction parallel to the bottom plate pointing to the coal wall as the Y-axis, and the upward direction perpendicular to the floor as the Z-axis; the inspection AR coordinate system is composed of two inspectors, with the geometric center of the Hololens equipment as the origin of the coordinate system, the inspection route along the working face layout direction as the X-axis, the horizontal advancement direction along the working face as the Y-axis, and the vertical direction as the Y-axis.

[0013] As a preferred embodiment, the data processing flow of the multi-user collaborative platform includes anchor point calculation, including scene anchor points and space anchor points; The scene anchor point is calibrated in advance during the virtual development process. The anchor point is based on the static position of the three fully mechanized mining machines. The scene anchor point coordinate system composed of the scene anchor points belongs to the secondary coordinate system of the entire coordinate system. The spatial anchor points are calibrated according to the dynamic conditions of the three machines in the comprehensive mining process. The inspection personnel flexibly calibrate them according to the actual operation conditions of the working face during the inspection process. After the inspection task of each shift is completed, it is saved as the inspection data of the shift. As the number of inspection shifts increases, the spatial anchor point coordinate system formed by the spatial anchor points belongs to the tertiary coordinate system of the entire coordinate system.

[0014] As a preferred implementation, the data processing flow of the multi-user collaborative platform includes lightweight computing for filtering complex data. The filtered complex data is uploaded to the VR end via the local area network, and the VR end performs complex calculations. The remaining simpler calculations are quickly calculated by the AR end.

[0015] As a preferred implementation, for the posture solution of the AR end, with the help of the manipulator model, the posture of the comprehensive mining equipment and the inspection personnel is determined by changing the posture matrix of different objects at different times; for the posture solution of the comprehensive mining equipment, a base must first be determined, and the coordinate transformation matrix of each actuator point in the joint of the comprehensive mining equipment is expressed by the coordinates of the base, thereby feeding back the posture of the entire comprehensive mining equipment; for the posture solution of the inspection personnel, the inspection personnel are regarded as particles in space, and the position of each inspection personnel in space is determined based on the scene anchor point, and the coordinates of the inspection personnel are expressed by the scene anchor point coordinates, and then the corresponding posture matrix is ​​converted into a spatial position matrix based on the primary coordinate system, and the inspection path of the inspection personnel is fed back through the inspection posture.

[0016] As a preferred implementation, in the measurement task module, the data solution method is: first, the AR glasses worn by the inspection personnel are preliminarily developed, including the construction of the Hololens+Unity3D platform, the matching of three-dimensional model feature blocks, and the writing of solution scripts in C#. Finally, the pitch, yaw, roll angle and distance data of the scraper conveyor and the coal mining machine can be displayed in the AR task bar; after obtaining the position and posture of the mining and transportation equipment, the data is uploaded to the local LAN, and the coordinated solution of the mining and transportation equipment and the hydraulic support group is carried out in the cloud.

[0017] As a preferred embodiment, the coordination method for solving the posture relationship between the mining and transportation equipment and the hydraulic support group is as follows: for the scraper conveyor, the scraper conveyor is first divided into a number of units corresponding to the hydraulic supports according to the number of floating connection mechanisms, and then the posture between a single floating connection mechanism and the floating connection mechanism of the adjacent frame is analyzed to form the overall pushing and sliding space posture of the scraper conveyor, and then the posture analysis task between the scraper conveyor and the hydraulic support group is completed by matching the coordination relationship between the hydraulic cylinder of the floating connection mechanism and the base of the hydraulic support; for the coal mining machine, the overall posture of the coal mining machine is first analyzed to form the overall operating space posture of the coal mining machine, and then the posture analysis task between the coal mining machine and the hydraulic support group is completed by matching the coordination relationship between the cutting knife and the hydraulic support guard plate.

[0018] As a preferred implementation method, a two-way matching verification method is adopted to realize the interaction between the mining and transportation equipment and the hydraulic support group. The two-way matching verification is divided into two parts: a test set and a verification set. The test set determines the interaction result by judging the distance and angle parameters according to the limit parameters set in advance; the verification set performs virtual simulation in the virtual space, takes the relationship between the support equipment and the mining and transportation equipment as the basis for judgment, and verifies whether the operating relationship between the support equipment and the mining and transportation equipment is normal according to whether there is interference between the models in the virtual space.

[0019] As a preferred implementation method, for coal mining machines with more stable operation, a cloud-edge collaborative strategy is adopted, with the help of Vuforia tools for efficient feature recognition and tracking capabilities. The three-dimensional feature block model is matched with the actual coal mining machine through the built-in nearest neighbor search algorithm, random sampling consistency algorithm and RANSAC algorithm script. The accuracy of the matching results is verified on the AR side, and finally the complex data is uploaded to the cloud through a multi-user collaborative platform for precise calculation and cloud rendering, so that the feature blocks can move together with the coal mining machine.

[0020] The present invention mainly starts from the intelligent inspection of the mining and transportation equipment in the comprehensive mining equipment - scraper conveyor and coal mining machine, and proposes a "two-person-two-machine" inspection idea that integrates manual matching and automatic tracking technology. With the help of AR augmented reality technology, the underground comprehensive mining working face operation is automatically tracked; on this basis, a manual matching strategy is added to give full play to the accuracy and meticulousness of manual measurement. At the same time, manual measurement has irreplaceable advantages over traditional contact sensor measurement for data scanning measurement of underground flooded areas, and can quickly and accurately convert contact information into spatial information; in addition, in order to adapt to the rapid update of posture information during the operation of mining and transportation equipment, reduce inspection pressure, and improve inspection efficiency, a multi-person inspection and synchronous inspection strategy is proposed, a dual Hololens device inspection method for two users is designed, and with the help of a multi-user collaborative interaction platform for data transmission and rapid solution, immersive collaborative inspection between dual Hololens devices is well realized.

[0021] The present invention integrates the multi-user collaborative interaction technology, virtual-reality interaction technology, and data fusion technology in digital twin technology, and its technical effects include: (1) Extend AR inspection to multiple users. A two-person and multi-user inspection strategy is adopted, and a multi-user collaborative platform is designed, which provides a new concept for extending inspection to multiple people or multiple vehicles in the future. (2) Introducing manual matching on the basis of automatic tracking to give full play to human subjective initiative: Experienced inspection personnel use the edge computers they carry with them to convert AR measurement data into posture data in real time. This not only conforms to the intuitive cognitive process of the human eye, but also can effectively filter information and quickly present effective information, thus improving the intelligence level of comprehensive mining operations. (3) Wearing AR glasses can realize real-life immersive inspection of the working face, ensuring the safety of operators while improving the efficiency and accuracy of inspection operations. At the same time, it can avoid, to a certain extent, the situation where the positioning of comprehensive mining equipment and the measurement of posture are difficult due to the difficulty in ensuring the measurement effect of existing mechanical sensors and SLAM point cloud technology in narrow working faces; (4) The multi-user collaborative platform will communicate the real-time data of the working face with the centralized control and command center. The centralized control and command personnel can also issue instructions to the on-site inspection personnel and assign tasks to achieve all-round and full-process data interaction. At the same time, the two inspection personnel can also use the multi-user collaborative platform to communicate and discuss with each other, and discuss and judge abnormal situations in the working face in a timely manner; (5) A method for calculating the posture of patrol personnel based on a robot model is proposed. The working surface is used as the robot base, and the patrol personnel are used as the robot actuator. The patrol personnel's working path is directly determined through matrix posture transformation. The human operation is more flexible and does not require complex mechanism transmission. Not only is the matrix representation more convenient, but it can also more accurately reflect the patrol personnel's inspection path. (6) Adopting the cloud-edge collaboration strategy, the AR end is the measurement end, measuring and integrating the initial parameters; the VR end is the subsequent solution end; after the AR end completes the basic data measurement and preprocessing locally, the cloud will solve the mining and transportation equipment posture and the mining and transportation-support equipment coordination posture in detail, and finally feedback to the front-line inspection personnel, highlighting the people-oriented concept, and the inspection personnel serve as the on-site decision-making terminal; In summary, the dual Hololens fast and efficient measurement method for mining and transportation equipment driven by the fusion of manual matching and automatic tracking provided by this patent is innovative and has good results. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is the overall framework diagram of the process implemented by the present invention; Figure 2 It is the specific flow chart of measuring user module; Figure 3 It is the specific flow chart of the measurement task module; Figure 4 It is a data processing flow chart of a multi-user collaborative platform; Figure 5 It is an interactive flow chart of mining and transportation equipment and support equipment; Figure 6 It is an automatic tracking flowchart. DETAILED DESCRIPTION

[0023] The two-person-two-machine measurement method for mining and transportation equipment provided by the present invention relates to a dual Hololens fast and efficient measurement method for mining and transportation equipment driven by manual matching and automatic tracking fusion. The overall concept is to use the Hololens device to non-contactly measure the mining and transportation equipment on the working face in the mode of manual matching and automatic tracking fusion drive, and then use the edge computer to solve the data and provide real-time feedback. First, for mining and transportation equipment, we pioneered an intelligent measurement method driven by the fusion of manual matching and automatic tracking, giving full play to human subjective initiative, and using AR equipment and edge computers to correct the data of the real-time operation status of the working face and make timely adjustments according to the actual working conditions; secondly, on the basis of intelligent measurement, we designed the measurement user module and measurement task module respectively: the measurement user module is used to coordinate tasks among multiple users and work in a division of labor; the measurement task module is used to solve the posture relationship between the scraper conveyor and the coal mining machine, and to link the posture of the mining and transportation equipment with the posture of the support equipment to form the dynamic operation posture state of the working face; the data measured by AR is first processed by the measurement task module in the edge computer, and multi-user matching, data analysis and preprocessing operations are performed, and then the measurement task module is entered to perform a separate solution of the posture data of the mining and transportation equipment, and the coordinated solution of the posture of the support equipment is realized through two-way matching verification. Finally, the solution results are fed back in the field of vision of the inspection personnel, realizing fast and efficient measurement of the posture of the mining and transportation equipment based on the dual Hololens device.

[0024] To achieve the above functions, firstly, a three-level coordinate system is established in the fully mechanized mining face to locate the entire fully mechanized mining space. The three-level coordinate systems are interrelated, with the first-level coordinate system as the reference, and the coordinates of the third-level coordinate system are transformed with the second-level coordinate system through matrix transformation, and the second-level coordinate system is transformed with the first-level coordinate system through matrix transformation.

[0025] The primary coordinate system is the coal seam coordinate system, with the initial mining position of the coal seam as the origin of the coal seam coordinate system, the working face layout direction as the X-axis, the horizontal advancement direction as the Y-axis, and the vertical direction as the Z-axis. This coordinate system is the positioning coordinate system of the entire space. Each coordinate in the space is obtained by matrix transformation from the primary coordinate system, and is used to locate all coordinate systems in the working face. The secondary coordinate system is the working face coordinate system composed of the three fully mechanized mining machines. The geometric center of the scraper conveyor head section of the first end support center symmetric plane in the scraper conveyor head direction is taken as the origin. The coordinate axis direction is always parallel to the coal seam coordinate system. This coordinate system is used to express the working face advancement information and describe the horizontal and vertical operation of the three fully mechanized mining machines. The three-level coordinate system is the local coordinate system of each equipment and the AR coordinate system. The coal mining machine takes the geometric center of the fuselage as the origin of the coordinate system, the direction parallel to the fuselage as the X-axis, the direction perpendicular to the front and rear plane of the fuselage as the Y-axis, and the direction perpendicular to the upper plane of the fuselage as the Z-axis; the scraper conveyor is first divided into several units according to the number of middle trough sections. Each middle trough unit takes the geometric center of the plane on the scraper trough as the origin of the coordinate system, the direction parallel to the bottom plate pointing to the tail of the machine as the X-axis, the direction parallel to the bottom plate pointing to the coal wall as the Y-axis, and the upward direction perpendicular to the floor as the Z-axis; the inspection AR coordinate system is composed of two inspectors, with the geometric center of the Hololens device as the origin of the coordinate system, the inspection route along the working face layout direction as the X-axis, the horizontal advancement direction along the working face as the Y-axis, and the vertical direction as the Y-axis. This coordinate system is for the active parts and inspectors in the working face, and is used to describe the operation of each component and the inspection of the inspectors in detail.

[0026] like Figure 1 As shown, the overall process of this implementation includes an intelligent measurement module, a measurement user module and a measurement task module.

[0027] The intelligent measurement module is based on the Hololens equipment for intelligent measurement, and its content mainly includes two parts: manual matching and automatic tracking. Automatic tracking can improve inspection efficiency and simplify inspection operations; manual matching can improve inspection accuracy and ensure the accuracy of measurement data.

[0028] Two experienced inspectors carry AR glasses and edge calculators, and conduct inspections from the head and tail ends respectively. Through the AR glasses they wear, they measure the pitch, roll, and yaw angle information of the floating connection mechanism and the distance relationship between different devices. Then, the data will be uploaded to the edge calculator carried by the inspectors by the multi-user collaborative platform. After a preliminary judgment on the operating posture of the mining and transportation equipment, the data will be fed back to the AR glasses worn by the two inspectors at the inspection end in the form of tasks. Finally, the two inspectors will converge in the middle of the working face from two different starting points. All the information measured by the two inspectors will constitute the inspection results of this inspection task, which will be uploaded to the centralized control center by the multi-user collaborative platform to generate an inspection log, completing the intelligent and efficient inspection process.

[0029] like Figure 2 As shown in the figure, the measurement user module is used to coordinate tasks among multiple users and work in a division of labor. This module mainly includes a multi-user collaboration platform, which is responsible for real-time coordination of measurement perspectives and measurement data among multiple users. All data is uniformly integrated and processed by the platform and then distributed to each user end, ensuring the consistency of measurement perspectives and measurement results displayed between different users, and effectively improving the efficiency of multi-person collaborative work.

[0030] The function of the multi-user collaborative platform is based on the working face mining and transportation posture data assisted by AR. It integrates multiple modules such as multi-Hololens platform collaborative cooperation module and network interaction module, and has built-in anchor point calculation script and lightweight calculation script: the anchor point calculation script is used to solve the position matrix of the anchor point. The anchor point group composed of multiple anchor points can effectively determine the spatial consistency of multi-user coordinates; the lightweight calculation script is used to filter complex data. The filtered complex data is uploaded to the VR end through the local LAN, and the VR end performs complex calculations. The remaining simpler calculations are quickly calculated by the AR end. These together constitute a complete multi-user collaborative system, which can not only upload and receive data through the local LAN, but also avoid data redundancy through the built-in posture solution script, ensure the accuracy of the measurement data, and truly improve the measurement efficiency.

[0031] The implementation process of the AR-assisted measurement method for the pose data of mining and transportation equipment is as follows: first, the Vuforia plug-in is used to scan and model the real scene of the mining and transportation equipment to facilitate the matching work in the subsequent measurement, and then the script written in the Hololens platform is used to run the program. After matching the virtual model with the real equipment, the script will automatically obtain the measurement data. Similarly, the above pose measurement method can be further extended to dual inspection users.

[0032] Use Vuforia tools to scan the working face fully mechanized mining equipment in real life, build a virtual model on the virtual end and integrate it into the AR end, and use the AR device to identify and track the fully mechanized mining equipment running in the real working face, and measure the posture information of the corresponding parts in real time. During the operation of the fully mechanized working face, the scraper conveyor has the characteristics of freedom and spatiality of its floating connection system. It is difficult to ensure the straightness during the pushing and sliding process, and the posture uncertainty is large. The manual matching method is selected to achieve more accurate positioning; while the operation of the coal mining machine is relatively more stable, so an AR automatic matching system can be added to achieve automatic matching of feature blocks.

[0033] In order to ensure the coordination between multiple users' AR glasses, the implementation process of the coordination between multiple users' AR glasses is as follows: First, a scene anchor group of scraper conveyors and coal mining machines is constructed in the coal seam space to locate the position of the comprehensive mining equipment in the working face before the operation; as the operation progresses, the position of the mining and transportation equipment changes relative to the scene anchor calibration position, and the on-site inspection personnel flexibly calibrate it according to the actual operation situation. After each shift inspection task is completed, it is saved as the inspection data of the shift. As the number of inspection shifts increases, it is gradually iterated to reflect the subtle changes on the working face. After locating the anchor point, the AR coordinate system position analysis method for multi-person collaborative operation is as follows: First, the position change of the anchor coordinate system composed of the scene anchor point and the space anchor point is analyzed, and the consistency of the observation angle of the two inspection personnel is determined through these anchor points; on the basis of this positioning, the coordinate AR coordinate system of the two inspection personnel is analyzed relative to the primary coordinate system to achieve the accuracy of positioning the same object under different perspectives.

[0034] like Figure 4 As shown in the figure, the data processing flow of the multi-user collaborative platform can be divided into two aspects: anchor point computing and lightweight computing.

[0035] For the positioning anchor point problem in multi-user coordinate consistency positioning, a method of jointly anchoring scene anchor points and space anchor points is adopted. The scene anchor points are calibrated first in the virtual development process. The anchor points are based on the static positions of the three machines in the comprehensive mining industry and cannot be modified. The scene anchor point coordinate system composed of scene anchor points belongs to the secondary coordinate system of the entire coordinate system; the space anchor points are calibrated for the dynamic conditions of the three machines in the comprehensive mining industry. The inspection personnel flexibly calibrate them according to the actual operation conditions of the working face during the inspection process. After the inspection task of each shift is completed, it is saved as the inspection data of the shift. As the number of inspection shifts increases, it is gradually iterated to reflect the subtle changes on the working face. The space anchor point coordinate system composed of the space anchor points calibrated on site belongs to the tertiary coordinate system of the entire coordinate system.

[0036] The anchor calculation script is used to locate the anchor point. By solving the anchor point group composed of scene anchor points and space anchor points, a unified reference of multiple user perspectives is defined to ensure the spatial consistency of the AR coordinates of multiple users in the same working surface. The lightweight calculation script is used for data screening to solve the problem that the AR end has fast feedback but weak computing power, and the VR end has strong computing power but slow feedback compared to the direct calculation of the AR end. This script is specifically used to quickly screen data. For example, when judging the displacement distance and deflection angle of the floating connection mechanism of the scraper conveyor and the cutting depth and angle of the coal mining machine, which are relatively simple and time-sensitive information, the AR end will quickly calculate and immediately feedback to the inspection personnel, while the overall swing angle of the scraper conveyor and the complex posture data of the coal mining machine and hydraulic support posture relationship are quickly uploaded to the VR end through the local LAN. With the help of the powerful computing power of the VR end, the data is finely calculated and then fed back to the inspection user through the LAN through the multi-user collaborative platform, maximizing the use of the computing power of the AR and VR ends while ensuring the timeliness of the inspection data acquisition.

[0037] For lightweight computing problems, the method of lightweight computing script screening is adopted. Since the AR side has fast feedback but weak computing power, while the VR side has strong computing power but slow feedback compared to the direct computing of the AR side, the lightweight computing script is inserted into the multi-user collaborative platform to avoid the disadvantage that the traditional Hololens computing must be calculated through the cloud VR platform. This script is specially used to quickly screen data. For example, when judging the displacement distance and deflection angle of the floating connection mechanism of the scraper conveyor and the cutting depth and angle of the coal mining machine, which are relatively simple and time-sensitive information, the AR side will quickly calculate and immediately feedback to the inspection personnel, while the overall swing angle of the scraper conveyor and the complex posture data of the coal mining machine and the hydraulic support posture relationship are quickly uploaded to the VR side through the local LAN. With the help of the powerful computing power of the VR side, the data is finely calculated and then fed back to the inspection user through the LAN through the multi-user collaborative platform, maximizing the use of the computing power of the AR and VR sides while ensuring the timeliness of the inspection data acquisition.

[0038] For the posture solution of the AR end, with the help of the manipulator model, the posture of the comprehensive mining equipment and the inspection personnel is determined by changing the posture matrix of different objects at different times. For the posture solution of the comprehensive mining equipment, a base must be determined first, and the coordinate transformation matrix of each actuator point in the joint of the comprehensive mining equipment is expressed by the coordinates of the base, so as to feedback the posture of the entire comprehensive mining equipment; for the posture solution of the inspection personnel, the inspection personnel are regarded as particles in space, and the position of each inspection personnel in space is determined based on the scene anchor point. The coordinates of the inspection personnel are expressed by the coordinates of the scene anchor point, and then the corresponding posture matrix is ​​converted into a spatial position matrix based on the primary coordinate system, and the inspection path of the inspection personnel is fed back through the inspection posture.

[0039] like Figure 3As shown, the measurement task module is used to solve the posture relationship between the scraper conveyor and the coal mining machine, and to connect the posture of the mining and transportation equipment with the posture of the support equipment to form the dynamic operation posture state of the working face. This module mainly performs data solution.

[0040] The data solution method in the measurement task module is as follows: First, the AR glasses worn by the inspection personnel must be preliminarily developed, including the construction of the Hololens+Unity3D platform, the matching of 3D model feature blocks, and the writing of solution scripts in C#. Finally, the pitch, yaw, roll angle and distance data of the scraper conveyor and coal mining machine can be displayed in the AR task bar; after obtaining the position and posture of the mining and transportation equipment, the data is uploaded to the local LAN, and the coordinated solution of the mining and transportation equipment and the hydraulic support group is carried out in the cloud.

[0041] The coordination solution method for the posture relationship between the mining and transportation equipment and the hydraulic support group is as follows: for the scraper conveyor, the scraper conveyor is first divided into several units corresponding to the hydraulic supports according to the number of floating connection mechanisms, and then the posture between a single floating connection mechanism and the floating connection mechanism of the adjacent frame is analyzed to form the overall pushing and sliding space posture of the scraper conveyor. Then, the coordination relationship between the hydraulic cylinder of the floating connection mechanism and the base of the hydraulic support is matched to complete the posture analysis task between the scraper conveyor and the hydraulic support group; for the coal mining machine, the overall posture analysis of the coal mining machine is first performed to form the overall operating space posture of the coal mining machine. Then, the coordination relationship between the cutting knife and the hydraulic support guard plate is matched to complete the posture analysis task between the coal mining machine and the hydraulic support group.

[0042] In order to make the interaction between mining and transportation equipment and support equipment more accurate, a two-way matching verification method is adopted. The two-way matching verification is divided into two parts: the test set and the verification set. The test set determines the interaction result by judging the distance and angle parameters according to the limit parameters set in advance. When the distance is too close or the angle overlaps, prompts to retract the support plate, move the frame and other operations. If the corresponding operation is not performed in the working face, the inspection personnel should contact the operator in time to make adjustments; the verification set performs virtual simulation in the virtual space, takes the relationship between the support equipment and the mining and transportation equipment as the judgment basis, and verifies whether the operating relationship between the support equipment and the mining and transportation equipment is normal according to whether there is interference with the model in the virtual space.

[0043] In a relatively specific implementation manner, Figure 5As shown in the figure, the interactive solution of mining and transportation equipment and support equipment adopts the two-way matching verification method. The test set is used to solve the complex data uploaded by the multi-user platform. Through the AR measurement script, the distance and angle parameters measured by the on-site inspection personnel can be input to solve the posture information of the scraper conveyor and the coal mining machine. Through the existing hydraulic support posture data in the test script, the interactive solution of mining and support equipment is carried out for the floating connection mechanism of the scraper conveyor and the guard plate of the coal mining machine. After the solution, the generated calculation results prompt the guard plate, frame shifting and other operations when the distance is too close or the angle overlaps. If the corresponding operation is carried out in time in the working face, the inspector should contact the operator in time for adjustment; the verification set is used to verify the calculation results of the test set to further ensure the accuracy of the solution results. The verification set continues to determine the relationship between the support equipment and the mining and transportation equipment in the virtual space of the VR end, and judges whether the solution result is correct according to whether the model in the virtual space has interference. If the verification result of the verification set matches the solution result of the test set, it means that the solution error is small; if not, the work log should be uploaded and the professional personnel should continue to debug.

[0044] like Figure 6 As shown in the figure, the implementation of AR automatic tracking of the coal mining machine adopts the cloud-edge collaboration method, with the help of Vuforia tools for efficient feature recognition and tracking capabilities, and through the built-in nearest neighbor search algorithm, random sampling consistency algorithm and RANSAC algorithm script, the three-dimensional feature block model is matched with the actual coal mining machine, and the accuracy of the matching results is verified on the AR side. Finally, through the multi-user collaborative platform, the complex data is uploaded to the cloud for precise calculation and cloud rendering, so that the feature blocks can move together with the coal mining machine. This can save some inspection personnel from the work of manual dragging and matching on site, further improving the inspection efficiency.

[0045] The solution is divided into two parts: fast edge computing and precise cloud computing. The edge computing first extracts the geometric center of the coal mining machine, the axis of the cutter, and the connection between the coal mining machine and the scraper conveyor as feature points, and constructs feature blocks for these feature points to achieve the feature description of the coal mining machine. After that, the coal mining machine feature block and the actual coal mining machine need to be feature matched. The inspection personnel manually calibrate the initial position before the coal mining machine starts working. After the feature block and the coal mining machine position are matched, the nearest neighbor search algorithm built into the edge computer can match the feature points in each frame with the known feature points, and can continue to calibrate automatically as the coal mining machine moves. After the feature matching is completed, the matching result is verified, and the random sampling consistency algorithm is used to eliminate the mismatching results and verify the accuracy of the matching results. After the matching result is verified, the RANSAC algorithm is used to identify the feature block position parameters and use the scene anchor point and space anchor point to correct the posture in real time, identify and eliminate the abnormal data position in time, and prevent the feature block from drifting.

[0046] The above four processes are quickly matched by edge computers at the edge, and the rendering of virtual content is completed in the cloud. According to the posture matching results, the corresponding position and angle data are uploaded for rendering in the cloud, and the virtual feature blocks are accurately aligned with the real objects in the real world, so that the feature blocks can move together with the real objects.

[0047] Before starting work on the working face, the two inspectors first walk to their respective inspection positions at the end and the end. After arriving at the designated position, they communicate through the intercom to inform the centralized control end that they are in place. At the same time, they turn on their respective Hololens devices at their respective inspection positions and connect to the multi-user collaborative platform. After completing the above operations, the two inspectors have completed the preliminary preparations for immersive inspection. Virtual feature blocks of comprehensive mining equipment at the end and the end appear in the field of vision. The two inspectors manually drag these feature blocks on the AR end to match them with the corresponding comprehensive mining equipment. After the AR glasses task bar prompts "Matching is complete, you can start the inspection operation", the preparations for the inspection operation are all completed. After confirmation by the centralized control center, the operator starts the machine and the comprehensive mining operation begins. The two inspectors obtain a new task "Perform inspection operation" in the task bar, and the inspection operation starts synchronously.

[0048] The inspector who starts at the end and the inspector who starts at the end walk towards each other to measure and check the data of the comprehensive mining equipment passing along the way. When the inspectors put on AR glasses, they can observe the simulation model of the comprehensive mining equipment. The AR glasses divide the colors of the comprehensive mining equipment into red, yellow and green. The red area represents the area that is greatly affected by terrain factors and has unstable posture changes, which is the key inspection object; the green area represents the area with relatively stable operation and relatively stable posture change trend, which is the secondary inspection object; the yellow area represents the inspection range of the inspectors. The inspectors should perform inspection tasks within the specified yellow range. Beyond the yellow range is a dangerous area. If the inspectors exceed this range, they will immediately receive a warning reminder and cannot continue the inspection task. Only when the inspectors return to the specified inspection area can they continue to perform the inspection task.

[0049] When the inspectors perform their inspection tasks, every time they walk past the corresponding comprehensive mining equipment, they touch the measurement button under the AR perspective, and the distance and angle information of the corresponding equipment will be generated in the measurement bar. For some places where data is missing and difficult to measure, the centralized control center will give unified command and send real-time tasks to the task bar of nearby inspectors. On the basis of routine inspections, the inspectors will conduct AR measurements of these parts that require special care at multiple times and different perspectives, and record multiple sets of posture data to facilitate subsequent prediction and planning at the centralized control center. The edge computers carried by the inspectors can quickly calculate the simpler posture data, such as the operating status of the floating connection mechanism of the scraper conveyor and the cutting status of the coal mining machine. At the same time, they can also use actual observations of the human eye to observe the actual terrain conditions in the working face and the operating conditions of the comprehensive mining equipment. For rugged and protruding positions that are not marked on the virtual end, they must be marked in time and the operator must be contacted in time for correction according to the actual situation.

[0050] The AR inspection and measurement end also includes emergency processing commands. When the inspection personnel encounter an emergency situation on site, they can immediately click the emergency stop command on the AR end. This command is the highest level of authority and can directly order the entire working face to stop working without the approval of the operation end and the centralized control end. The scope of use of the emergency stop command includes: when there are serious failures in mechanical equipment, such as motor overheating, mechanical component breakage or excessive wear, which may endanger the safety of on-site personnel or cause equipment damage; when on-site workers encounter safety threats; when there are unpredictable environmental factors such as elevated gas and fire in the comprehensive mining area; the scope of the emergency stop command usually includes but is not limited to the above three aspects. Inspection personnel must undergo special training before taking up their posts to respond to emergencies correctly and promptly.

[0051] After obtaining the independent posture conditions of the floating connection mechanism and the coal mining machine, with the help of the powerful computing power of the VR end, the relevant posture data is uploaded to the cloud through the multi-user collaborative platform, and the posture relationship between the scraper conveyor, coal mining machine and hydraulic support is solved, forming a complete posture relationship between the mining and transportation equipment and the support equipment.

[0052] After the two inspectors meet in the middle of the working surface, the AR task bar prompts "Inspection completed". The inspection task of the inspectors of this shift is basically completed, and they can prepare to hand over the work with the next inspectors. When the "Shift Handover" task appears in the task bar, the two inspectors continue to move in the inspection direction respectively. The inspector whose starting point is at the end of the inspection ends at the end of the inspection, and the inspector whose starting point is at the end of the inspection ends at the end of the inspection. When going from the meeting point to the inspection end, the inspection work along the way should also be carried out simultaneously. The inspection process is the same as above. This part of the work task is the content of the handover. At the same time, the inspectors of the next shift enter the working surface and go to the predetermined location to prepare for standby. After the inspection work of the previous shift is completed, the task bar prompts "Inspection task completed, shift handover". After the next shift inspectors wear AR glasses, they continue the inspection task.

[0053] The key to maximizing AR measurement efficiency is to use a multi-user collaborative platform to perform lightweight calculations and data interaction between AR and VR. The lightweight calculation script built into the AR end is used to filter complex data. Simple data is calculated locally in real time, and complex data is uploaded to the VR end for precise calculation, avoiding the disadvantage that traditional Hololens calculations must be calculated through a cloud VR platform. Such as judging the relatively simple and time-sensitive information such as the displacement distance and deflection angle of the floating connection mechanism of the scraper conveyor, and the cutting depth and angle of the coal mining machine. The AR end quickly calculates the position of the floating connection mechanism, coal mining machine, and inspection personnel by formula and immediately feeds back to the inspection personnel. The formula is as follows: Among them, the pitch angle is defined as θ, the roll angle is defined as α, the yaw angle is defined as γ, and the pose matrix between adjacent floating connection mechanisms is expressed by the following formula:

[0054] The pose matrix of the coal mining machine is expressed by the following formula:

[0055] The inspection posture of the inspection personnel is expressed by the following formula:

[0056] Among them, A represents the coal seam coordinate system, B represents the motion coordinate system of the inspector on the moving rigid body, and B0 represents the coordinate origin of the motion coordinate system.

[0057] The overall swing angle of the scraper conveyor and the complex posture data of the position relationship between the coal mining machine and the hydraulic support are quickly uploaded to the VR side through the local LAN. A virtual model is built on the VR side based on Unity3D, and a real-time data transmission framework between the scraper conveyor and Unity3D is built through TCP / IP. A data storage method based on XML and SQLServer database is established, so that these data can be flexibly called for further planning and solving the complex posture relationship between mining and transportation equipment and support equipment. With the help of the powerful computing power of the VR side, the detailed calculation is then fed back to the inspection user through the multi-user collaborative platform through the local area network, maximizing the use of the computing power of the AR and VR sides while ensuring the timeliness of the inspection data acquisition.

[0058] When performing AR measurement, there is often a problem of inconsistent inspection rhythm, that is, the speed of manual matching by the inspectors is lower than the speed of the fully mechanized mining equipment, resulting in a disconnected inspection. The solution to this problem is to combine manual matching with automatic matching. For scraper conveyors with more complex operation, manual matching is used for precise matching. For coal mining machines with more stable operation, automatic matching is used to improve inspection efficiency.

Claims

1. A two-person-two-machine measurement method for mining and transportation equipment, characterized in that: It includes intelligent measurement module, measurement user module and measurement task module; The intelligent measurement module includes two inspectors who conduct inspections in opposite directions and AR glasses carried by the inspectors, which collect the working face mining posture data by manual matching and automatic tracking based on Hololens equipment; The measurement user module is built into the edge computer carried by the inspection personnel, including a multi-user collaborative platform; The multi-user system platform measures the AR-assisted working face mining posture data and performs data processing operations. The multi-user system platform coordinates the measurement perspectives and data between multiple users in real time; The measurement task module is responsible for clarifying the measurement tasks for the multi-user collaborative platform and for solving the data feedback from the measurement tasks; Driven by tasks, a virtual solution platform is built according to the actual working status of mining and transportation equipment. The position information of scraper conveyors and coal mining machines is solved in real time according to the measurement data fed back by users, and the position information of hydraulic support groups in the multi-user collaborative platform is matched and solved. The relationship between mining and transportation equipment and support equipment is integrated, and the solution results are synchronously fed back to the inspection AR end in the form of tasks.

2. The method according to claim 1, characterized in that: The multi-user system platform is based on the mining and operation posture data of the working face assisted by AR, integrates multiple Hololens platform collaboration modules and network interaction modules, uses Vuforia tools to perform real-scene scanning of the comprehensive mining equipment on the working face, builds a virtual model on the virtual end and integrates it into the AR end, and uses the AR device to identify and track the comprehensive mining equipment running in the real working face, and measures the posture information of the corresponding parts in real time.

3. The method according to claim 1 or 2, characterized in that: Including establishing a three-level coordinate system in the fully mechanized mining face; The primary coordinate system is the coal seam coordinate system, with the initial mining position of the coal seam as the origin of the coal seam coordinate system, the working face layout direction as the X-axis, the horizontal advancement direction as the Y-axis, and the vertical direction as the Z-axis, which is used to locate all coordinate systems in the working face; The secondary coordinate system is the working face coordinate system composed of three fully mechanized mining machines. The geometric center of the scraper conveyor head section of the central symmetric plane of the first end support in the scraper conveyor head direction is taken as the origin, and the coordinate axis direction is always parallel to the coal seam coordinate system. The three-level coordinate system is the local coordinate system of each equipment and the AR coordinate system. The coal mining machine takes the geometric center of the fuselage as the origin of the coordinate system, the direction parallel to the fuselage as the X-axis, the direction perpendicular to the front and rear plane of the fuselage as the Y-axis, and the direction perpendicular to the plane on the fuselage as the Z-axis; the scraper conveyor is first divided into several units according to the number of middle trough sections. Each middle trough unit takes the geometric center of the plane on the scraper trough as the origin of the coordinate system, the direction parallel to the bottom plate pointing to the tail of the machine as the X-axis, the direction parallel to the bottom plate pointing to the coal wall as the Y-axis, and the upward direction perpendicular to the floor as the Z-axis; the inspection AR coordinate system is composed of two inspection personnel, with the geometric center of the Hololens equipment as the origin of the coordinate system, the inspection route along the working face layout direction as the X-axis, the horizontal advancement direction along the working face as the Y-axis, and the vertical direction as the Y-axis.

4. The method according to claim 3, characterized in that: The data processing flow of the multi-user collaborative platform includes anchor point calculation, including scene anchor points and space anchor points; The scene anchor point is calibrated in advance during the virtual development process. The anchor point is based on the static position of the three fully mechanized mining machines. The scene anchor point coordinate system composed of the scene anchor points belongs to the secondary coordinate system of the entire coordinate system. The spatial anchor points are calibrated according to the dynamic conditions of the three machines in the comprehensive mining process. The inspection personnel flexibly calibrate them according to the actual operation conditions of the working face during the inspection process. After the inspection task of each shift is completed, it is saved as the inspection data of the shift. As the number of inspection shifts increases, the spatial anchor point coordinate system formed by the spatial anchor points belongs to the tertiary coordinate system of the entire coordinate system.

5. The method according to claim 1 or 4, characterized in that: The data processing flow of the multi-user collaborative platform includes lightweight computing, which is used to filter complex data. The filtered complex data is uploaded to the VR end via the local area network, and the VR end performs complex calculations. The remaining simpler calculations are quickly calculated by the AR end.

6. The method according to claim 5, characterized in that: For the posture solution of the AR end, with the help of the manipulator model, the posture of the comprehensive mining equipment and the inspection personnel is determined by changing the posture matrix of different objects at different times; for the posture solution of the comprehensive mining equipment, a base must be determined first, and the coordinate transformation matrix of each actuator point in the joint of the comprehensive mining equipment is expressed by the coordinates of the base, thereby feeding back the posture of the entire comprehensive mining equipment; for the posture solution of the inspection personnel, the inspection personnel are regarded as particles in space, and the position of each inspection personnel in space is determined with the scene anchor point as the reference, and the coordinates of the inspection personnel are expressed by the coordinates of the scene anchor point, and then the corresponding posture matrix is ​​converted into a spatial position matrix with the primary coordinate system as the reference, and the inspection path of the inspection personnel is fed back through the inspection posture.

7. The method according to claim 1 or 6, characterized in that: In the measurement task module, the data solution method is as follows: first, the AR glasses worn by the inspection personnel are preliminarily developed, including the construction of the Hololens+Unity3D platform, the matching of three-dimensional model feature blocks, and the writing of solution scripts in C#. Finally, the pitch, yaw, roll angle and distance data of the scraper conveyor and coal mining machine can be displayed in the AR task bar; after obtaining the position and posture of the mining and transportation equipment, the data is uploaded to the local LAN, and the coordinated solution of the mining and transportation equipment and the hydraulic support group is carried out in the cloud.

8. The method according to claim 7, characterized in that: The coordination solution method for the posture relationship between the mining and transportation equipment and the hydraulic support group is as follows: for the scraper conveyor, the scraper conveyor is first divided into several units corresponding to the hydraulic supports according to the number of floating connection mechanisms, and then the posture between a single floating connection mechanism and the floating connection mechanism of the adjacent frame is analyzed to form the overall pushing and sliding space posture of the scraper conveyor. Then, the coordination relationship between the hydraulic cylinder of the floating connection mechanism and the base of the hydraulic support is matched to complete the posture analysis task between the scraper conveyor and the hydraulic support group; for the coal mining machine, the overall posture analysis of the coal mining machine is first performed to form the overall operating space posture of the coal mining machine. Then, the coordination relationship between the cutting knife and the hydraulic support guard plate is matched to complete the posture analysis task between the coal mining machine and the hydraulic support group.

9. The method according to claim 1 or 8, characterized in that: The interaction between mining and transportation equipment and hydraulic support groups is realized by adopting the method of two-way matching verification. The two-way matching verification is divided into two parts: test set and verification set. The test set determines the interaction result by judging the distance and angle parameters according to the limit parameters set in advance; the verification set performs virtual simulation in the virtual space, takes the relationship between the support equipment and the mining and transportation equipment as the basis for judgment, and verifies whether the operating relationship between the support equipment and the mining and transportation equipment is normal according to whether there is interference between the models in the virtual space.

10. The method according to claim 9, characterized in that: For coal mining machines that run more stably, a cloud-edge collaborative strategy is adopted, with the help of Vuforia tools for efficient feature recognition and tracking capabilities. The three-dimensional feature block model is matched with the actual coal mining machine through the built-in nearest neighbor search algorithm, random sampling consistency algorithm and RANSAC algorithm script. The accuracy of the matching results is verified on the AR side, and finally the complex data is uploaded to the cloud through a multi-user collaborative platform for precise calculation and cloud rendering, so that the feature blocks can move together with the coal mining machine.

Images