Heterogeneous multi-sensor information fusion method and system for accurate positioning of underground coal mine mining equipment

Abstract

The invention provides a heterogeneous multi-sensor information fusion method and system for accurate positioning of coal mine underground mining equipment, and the method comprises the steps: S1, collecting inertial navigation information, and extracting the pose information of a heading machine; s2, inputting the pose information of the heading machine into a millimeter wave radar; s3, distance measurement information of the laser radar and the millimeter wave radar is collected; s4, extracting target position information from the target library; s5, constructing first branch information and second branch information; s6, performing information fusion on the collected inertial navigation information, the first branch information and the second branch information by adopting a fusion technology based on a factor graph to form fusion information; s7, returning and transmitting the fusion information to the inertial navigator; and S8, the inertial navigator carries out navigation information correction according to the fusion information. According to the invention, a multi-sensor fusion scheme which takes inertial navigation as a basis and takes a laser radar and a millimeter-wave radar as main auxiliary equipment is adopted, so that positioning equipment which can completely adapt to performance requirements of underground operation of a coal mine is provided.

Description

Technical Field

[0001] This application relates to the technical field of coal mine underground mining equipment positioning. Specifically, it relates to a heterogeneous multi-sensor information fusion method and system for precise positioning of coal mine underground mining equipment. Background Art

[0003] The intelligent mining mode represented by "intelligent control + remote intervention" with Huangling as an example has been built and operated in many mining areas across the country. The automatic straightening technology of the working face represented by the application of the inertial navigation system of the working face, and the intelligent mining technology of the "transparent working face" represented by building a digital coal seam through geological modeling represent the development of fully mechanized mining technology at different stages, which is an effective technical path to realize intelligent unmanned mining and has accumulated a large amount of technology and experience in the aspect of intelligence.

[0004] The intelligent development of fully mechanized mining faces is rapid, but the production equipment and technology of heading faces are backward, which are difficult to meet the production requirements of high-yield and high-efficiency coal mines, resulting in a serious contradiction of "mining and tunneling imbalance". Compared with fully mechanized mining faces, at present, the construction of most roadway headings using roadheader still adopts manual operation, with high labor intensity and low production efficiency. The autonomous precise positioning of roadheader is the key core technology for the construction of intelligent heading faces and has great research significance. Limited by the current production technology and heading quality specifications, and coupled with the limitations of high dust and complex geological conditions in the heading face, it is extremely difficult to sense the pose and working condition of the heading equipment. Among them, the dynamic positioning technology of coal mine underground roadway heading equipment has seriously restricted the development of intelligent heading equipment.

[0005] There are many positioning devices in coal mine underground. Table 1 shows the comprehensive performance comparison of several main positioning devices under the working conditions of coal mine mining and tunneling. At present, there is no positioning device that can fully meet the performance requirements of coal mine underground operations. Considering the advantages and disadvantages of each sensor technology, inertial navigation, lidar, and millimeter-wave radar are relatively reliable positioning devices under the working conditions of coal mine mining and tunneling. However, these positioning devices also have their own defects. The positioning error of inertial navigation accumulates over time, lidar is easily affected by dust or light in underground roadways, and millimeter-wave radar has the problem of low resolution.

[0006] Table 1 Comprehensive performance comparison of various positioning devices under the working conditions of coal mine mining and tunneling

[0007] Therefore, it is necessary to study the precise positioning technology for underground coal mining and excavation equipment. On the one hand, by integrating the advantages of various sensors, the influence of environmental factors such as light, dust, and water mist on the measurement information of the positioning equipment is suppressed, and the accurate position and attitude information of the equipment at the tunneling face are obtained. On the other hand, the configuration scheme suitable for various combinations of sensor devices is adapted to obtain the engineering application ability of plug-and-play. The research on these two aspects of technology is of great significance for solving the bottleneck problem of underground coal mine automation tunneling and realizing the safe, efficient, and less or unmanned mining of the tunneling face in underground coal mines. Summary of the Invention

[0008] In view of this, the present application provides a heterogeneous multi-sensor information fusion method and system for precise positioning of underground coal mining and excavation equipment, so as to provide a positioning device that can fully meet the performance requirements of underground coal mine operations.

[0009] To achieve the above object, the technical solution adopted by the present application is as follows: A heterogeneous multi-sensor information fusion method for precise positioning of underground coal mining and excavation equipment. In this method, a lidar is installed on a roadheader and assisted by multiple targets that advance with the coal mine roadway. At the same time, a millimeter-wave radar is installed on the roadheader, and the millimeter-wave radar obtains position information by using the target information library constructed by the lidar. And an inertial navigator is also installed on the roadheader. The method specifically includes the following steps: S1: Collect inertial navigation information for navigation solution, and extract the pose information of the roadheader from it. The inertial navigation information includes gyro information and accelerometer information of the inertial navigator. S2: Input the pose information of the roadheader into the millimeter-wave radar to assist the millimeter-wave radar in ranging. S3: Collect lidar information and millimeter-wave radar information. Both the lidar information and the millimeter-wave radar information include ranging information, and the ranging information is the distance information between the roadheader and the target measured by the lidar or the millimeter-wave radar. S4: Extract the target position information from the target library. S5: Subtract the target position information from the ranging information of the lidar to construct the first branch information, and subtract the target position information from the ranging information of the millimeter-wave radar to construct the second branch information. S6: Use the information fusion technology based on the factor graph to fuse the collected inertial navigation information, the first branch information, and the second branch information to form fusion information. S7: Transmit the fusion information back to the inertial navigator. S8: The inertial navigator corrects the navigation information according to the fusion information.

[0010] Further, in S1, the navigation solution based on the gyro information and accelerometer information in the inertial navigation information is specifically a dual-channel navigation solution based on the gyro information and accelerometer information in the inertial navigation information; the dual-channel navigation solution refers to simultaneously performing two-channel inertial navigation solutions, including an inertial navigation loop 1 and an inertial navigation loop 2; the inertial navigation loop 1 is used for information fusion; the inertial navigation loop 2 is used for Kalman filtering solution to perform filtering estimation of the celestial misalignment angle and celestial gyro drift, and then feedback the estimated values of the celestial misalignment angle and celestial gyro drift to the inertial navigation loop 1 to correct the inertial navigation error.

[0011] Further, before extracting the target position information from the target library in S4, it also includes: Performing target recognition based on the lidar information to determine whether the measured target belongs to a new target; if it is a new target, expanding the target library and updating the new target position information; if it is not a new target, directly execute step S4.

[0012] Further, S6 specifically includes: Fusing the inertial navigation information of the inertial navigation loop 1 in the dual-channel navigation solution with the first branch information and the second branch information by using the factor graph-based information fusion technology to form fusion information.

[0013] Further, during information fusion in S6, the observation information of the information fusion filter is optimized from the previous conventional position information to the original ranging information; and after performing information fusion by using the factor graph-based information fusion technology, the covariance intersection (CI) algorithm is used to improve the fusion accuracy; the CI fusion formula is shown in Equation (1): (1) Where ω is a constant that minimizes the trace of the fused covariance matrix, and the fused error covariance matrix given by the CI algorithm is the least upper bound on any error correlation between sensors; P represents the error covariance matrix; the subscript a and b represent the information identifiers participating in the fusion, and the subscript c represents the information identifier after fusion.

[0014] The observation equations based on the original ranging information can be divided into two categories: (I) Using the target point to calibrate the inertial navigation position error. Assuming the latitude, altitude, and longitude of the target point Si are Si ( , , ), and the latitude, altitude, and longitude of the inertial navigation (inertial navigator) are ( , , ), the sensor measures the distance between the two as L i ; the symbol represents the error, that is L i represents L i the error of, and its error equation should satisfy: (2) (II) Using the inertial navigation position information to calibrate the position error of the target point, at this time the error equation should satisfy: (3).

[0015] A heterogeneous multi-sensor information fusion system for precise positioning of coal mine underground excavation equipment, the system includes a lidar, a millimeter-wave radar and an inertial navigator installed on the roadheader; multiple target points advancing with the coal mine roadway; a target point library constructed by the lidar; the system also includes the following modules: The inertial navigation information acquisition module is used to acquire inertial navigation information for navigation calculation and extract the pose information of the roadheader from it. The inertial navigation information includes the gyro information and accelerometer information of the inertial navigator; The transmission module is used to input the pose information of the roadheader into the millimeter-wave radar to assist the millimeter-wave radar in ranging; The radar information acquisition module is used to acquire lidar information and millimeter-wave radar information. Both the lidar information and the millimeter-wave radar information include ranging information, and the ranging information is the distance information between the roadheader and the target point measured by the lidar or the millimeter-wave radar; The extraction module is used to extract the target point position information from the target point library; The calculation module is used to subtract the target point position information from the lidar ranging information to construct the first branch information, and subtract the target point position information from the millimeter-wave radar ranging information to construct the second branch information; The information fusion module is used to fuse the acquired inertial navigation information, the first branch information and the second branch information by using the information fusion technology based on the factor graph to form fusion information; The feedback module is used to return the fusion information to the inertial navigator; The correction module is used for the inertial navigator to correct the navigation information according to the fusion information.

[0016] Further, the inertial navigation information acquisition module includes a dual-channel navigation solution operator module for performing dual-channel navigation solution according to the gyro information and accelerometer information in the inertial navigation information; the dual-channel navigation solution refers to simultaneously performing two-channel inertial navigation solutions, including an inertial navigation loop 1 and an inertial navigation loop 2; the inertial navigation loop 1 is used for information fusion; the inertial navigation loop 2 is used for Kalman filter solution to perform filtering estimation of the celestial misalignment angle and celestial gyro drift, and then feeds the estimated values of the celestial misalignment angle and celestial gyro drift back into the inertial navigation loop 1 to correct the inertial navigation error.

[0017] Further, the system further includes an identification module for performing target identification based on lidar information before extracting target position information from the target library to determine whether the measured target belongs to a new target; if it is a new target, the target library is expanded and the new target position information is updated. If it is not a new target, the extraction module is started.

[0018] Further, the information fusion module is specifically configured to perform information fusion on the inertial navigation information of the inertial navigation loop 1 in the dual-channel navigation solution, the first branch information, and the second branch information by using an information fusion technology based on a factor graph to form fusion information.

[0019] Further, the information fusion module further includes an optimization sub-module for optimizing the observation information of the information fusion filter from the previous conventional position information to the original ranging information; and after performing information fusion by using the information fusion technology based on a factor graph, the covariance intersection (CI) algorithm is further used to improve the fusion accuracy; the CI fusion formula is shown in Equation (1): (1) where ω is a constant that minimizes the trace of the fused covariance matrix, and the fused error covariance matrix given by the CI algorithm is the least upper bound on any error correlation between sensors; The observation equations based on the original ranging information can be divided into two categories: (I)Using the target point to calibrate the position error of the inertial navigation (inertial navigator), assuming that the latitude, altitude, and longitude of the target point Si are Si ( , , ), the inertial navigation position information ( , , ), and the distance measured by the sensor between the two is L i , and its error equation should satisfy: (2) (2) Calibrate the position error of the calibration target point using the position information of the inertial navigation (inertial navigator). At this time, the error equation should satisfy: (3).

[0020] Compared with the prior art, the beneficial effects of the present application are as follows: 1. By adopting the factor graph information fusion method, the available navigation sensor measurement information is abstracted into corresponding factor nodes, and the state variable nodes are recursively updated according to the non-linear optimization theory. It can solve the interference of various situations such as multi-sensor information being simultaneously effective, time-sharing effective, and single information jump on the combined accuracy, and can make automatic decisions to improve the accuracy and robustness of the multi-sensor information fusion algorithm. Moreover, by adopting the information fusion technology based on the factor graph, the fusion system can have the engineering application ability of plug and play.

[0021] 2. By adopting the Covariance Intersection (CI) algorithm, this algorithm can effectively fuse the multi-sensor information with unknown error correlation, realize the tight constraint on the intersection area of the covariance ellipses of each sensor, and achieve the purpose of improving the fusion accuracy.

[0022] 3. By adopting a multi-sensor fusion scheme with inertial navigation as the basis and lidar and millimeter-wave radar as the main auxiliary devices, using the characteristics that the ranging or positioning information of the lidar or millimeter-wave radar does not diverge with time, it can effectively estimate the zero bias of the accelerometer of the inertial navigation system, the accelerometer scale factor, the inertial navigation installation error, etc. in real time, so as to achieve the purpose of suppressing the divergence speed of the inertial navigation.

[0023] 4. Give full play to the high autonomy and high sensitivity characteristics of inertial devices to provide real-time position measurement and boom attitude measurement information for the roadheader; auxiliary positioning devices such as lidar and millimeter-wave radar provide accurate position information for the inertial navigation device, playing a role in suppressing inertial errors and providing reference parameters for the control of the roadheader.

[0024] 5. By installing lidar and millimeter-wave radar devices on the roadheader and assisting with a configuration method of multiple fixed targets advancing with the coal mine roadway, it can realize the self-calibration of the lidar position information and automatically locate new targets, laying a foundation for realizing the automated work process of the roadheader.

[0025] 6. By adopting a dual-channel inertial navigation solution and filtering scheme, the influence of some poorly observable parameters (such as the celestial misalignment angle, celestial gyro drift, etc.) in the inertial navigation system is eliminated.

[0026] 7. The multi-sensor information fusion scheme of the present application not only has the ability to fuse inertial navigation, lidar, and millimeter-wave radar information, but also has the ability to fuse the information of other auxiliary positioning devices.

[0027] 8. Optimize the observation information of the information fusion filter from the previous conventional position information to the original ranging information, improving the anti-interference ability of the information fusion algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0029] Figure 1 It is a flowchart of a heterogeneous multi-sensor information fusion method for precise positioning of coal mine underground excavation equipment according to the present application; Figure 2 It is a structural block diagram of a heterogeneous multi-sensor information fusion system for precise positioning of coal mine underground excavation equipment according to the present application; Figure 3 It is a schematic diagram of a multi-sensor information fusion scheme such as lidar, millimeter-wave radar, and inertial navigation according to the present application; Figure 4 It is a schematic diagram of a dual-channel navigation solution scheme according to the present application; Figure 5 In (a), it is a simulation estimation result diagram of the inertial navigation gyro drift according to the present application, Figure 5 In (b), it is a simulation estimation result diagram of the misalignment angle; Figure 6 It is a schematic diagram of the principle of multi-sensor information fusion technology based on factor graph; Figure 7 It is a research idea diagram of "Analysis of positioning performance based on heterogeneous multi-sensor information fusion" in the specific implementation manner of the present application; Figure 8 It is a schematic diagram of the hardware configuration scheme of the precise positioning system for roadheaders in the specific implementation manner of the present application; Figure 9 It is a schematic diagram of the inertial navigation device in the specific implementation manner of the present application; Figure 10 In (a), it is a 24-hour attitude error diagram based on the multi-sensor information fusion positioning system of the present application, Figure 10 In (b), it is a 24-hour positioning error diagram based on the multi-sensor information fusion positioning system of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application.

[0031] This application comprehensively compares the advantages and disadvantages and operational convenience of available positioning devices for mining equipment (including lidar, millimeter-wave radar, inertial navigation devices, total stations, odometers, cameras, etc.) under the mining conditions in coal mines, and proposes a system solution for the precise positioning system for mining equipment that is based on inertial navigation and uses lidar and millimeter-wave radar as main auxiliary devices.

[0032] The specific implementation manner of this application is as follows: A heterogeneous multi-sensor information fusion method for precise positioning of mining equipment in coal mines. In this method, a lidar is installed on a roadheader and assisted by multiple targets that advance with the coal mine roadway; at the same time, a millimeter-wave radar is installed on the roadheader, and the millimeter-wave radar obtains position information by using the target information library constructed by the lidar; and an inertial navigator is also installed on the roadheader.

[0033] Installing the lidar on the roadheader and assisting with multiple targets that advance with the coal mine roadway is mainly based on two considerations: First, under actual working conditions, the roadway scene in the coal mine changes dynamically and the texture features degrade, so it is impossible to directly use the SLAM matching positioning algorithm to obtain position information, and it is necessary to manually configure targets to provide feature information for the lidar; Second, the method of placing the lidar on the ground and the targets on the roadheader has problems of complex operation and low accuracy: on the one hand, as the roadway deepens, it is necessary to adjust and calibrate the position information of the lidar from time to time, and on the other hand, the targets are placed in the narrow space of the roadheader body, and it is difficult to obtain high-precision positioning information.

[0034] The method of installing the lidar on the roadheader can not only calibrate the position information of the lidar, but also automatically locate new targets. The lidar sweeps the roadway backward, captures and identifies the targets, and the position information of the targets can be analyzed by integrating the pose information of the lidar and inertial navigation. If the target is a known target, the target position information in the target information library, the inertial navigation information, and the radar ranging information can be used to estimate and correct the inertial navigation error parameters, and then the position information of the lidar can be obtained (the relative position of the inertial navigation and the lidar is fixed). If the target is an unknown new target, it is necessary to correct the position of the target by integrating the information of several other known targets and then record and expand it into the target information library. In the actual operation process, new targets should be arranged one by one to avoid large position errors of new targets caused by too little information of known targets.

[0035] To suppress the influence of factors such as light and dust on the measurement accuracy of lidar, it is necessary to monitor the target ranging information obtained by the lidar: on the one hand, monitor the lidar ranging information by comparing with the inertial navigation information, and on the other hand, monitor the target point cloud information measured by the lidar.

[0036] The millimeter-wave radar is to be installed on the roadheader, and the position information is obtained by using the target information library constructed by the lidar. Guided by the real-time inertial navigation information, the millimeter-wave radar scans the target, and estimates and corrects the inertial navigation error parameters by fusing the target position information, inertial navigation information, and radar ranging information in the target information library.

[0037] The inertial navigation system is installed on the roadheader, which can provide high-frequency continuous and reliable navigation information for the control system of the roadheader, but has the disadvantage of long-term navigation divergence. According to the principle of inertial navigation error, in the application environment of coal mine excavation, when the speed or position is used as the observable quantity, the related error parameters of the accelerometer (including accelerometer zero bias, accelerometer scale factor, inertial navigation installation error, etc.) have good observability. The ranging or positioning information of the lidar or millimeter-wave radar has the characteristic of not diverging with time, and can effectively estimate these error parameters in real time, so as to achieve the purpose of suppressing the divergence of inertial navigation.

[0038] As Figure 1 and Figure 3 shown, the information fusion method specifically includes the following steps: S1: Collect inertial navigation information for navigation solution, and extract the pose information of the roadheader from it. The inertial navigation information includes gyro information and accelerometer information of the inertial navigator; S2: Input the pose information of the roadheader into the millimeter-wave radar to assist the millimeter-wave radar in ranging; S3: Collect lidar information and millimeter-wave radar information. Both the lidar information and the millimeter-wave radar information include ranging information, and the ranging information is the distance information between the roadheader and the target measured by the lidar or the millimeter-wave radar; S4: Extract the target position information from the target library; S5: Subtract the target position information from the lidar ranging information to construct the first branch information, and subtract the target position information from the millimeter-wave radar ranging information to construct the second branch information; S6: Use the information fusion technology based on the factor graph to fuse the collected inertial navigation information, the first branch information, and the second branch information to form fusion information; S7: Transmit the fusion information back to the inertial navigator; S8: The inertial navigator corrects the navigation information according to the fusion information.

[0039] As a further implementation, in S1, the navigation solution based on the gyro information and accelerometer information in the inertial navigation information is specifically a dual-channel navigation solution based on the gyro information and accelerometer information in the inertial navigation information; the dual-channel navigation solution refers to performing two-channel inertial navigation solutions simultaneously, including an inertial navigation loop 1 and an inertial navigation loop 2; the inertial navigation loop 1 is used for information fusion; the inertial navigation loop 2 is used for Kalman filtering solution to perform filtering estimation of the vertical misalignment angle and vertical gyro drift, and then feedback the estimated values of the vertical misalignment angle and vertical gyro drift to the inertial navigation loop 1 to correct the inertial navigation error.

[0040] To obtain accurate positioning information, the precise positioning system for mining equipment needs to achieve two capabilities: on the one hand, it needs to achieve accurate estimation of the error parameters with poor observability of the inertial navigation, and on the other hand, it needs to achieve a multi-sensor information fusion algorithm with anti-interference and plug-and-play capabilities.

[0041] Due to the influence of some poorly observable parameters in the inertial navigation system, these parameters include the vertical misalignment angle, vertical gyro drift, etc. Taking a navigation system with a gyro accuracy of 0.01° / h as an example, the self-alignment accuracy is about 0.053°. If it works 24 hours a day according to the requirements of the international advanced level, the vertical misalignment angle has exceeded 0.1°, which does not meet the requirements for the heading and attitude accuracy. Therefore, it is necessary to perform online estimation of the poorly observable parameters of the inertial navigation system. Using the Figure 4 dual-channel navigation solution shown, the inertial navigation uses the measurement information of the IMU (Inertial Measuring Unit) (gyro and accelerometer measurement information) to perform two-channel navigation solutions simultaneously. The inertial navigation information 1 is used for information fusion with multiple sensors, and the inertial navigation information 2 performs pure inertial navigation solution for Kalman filtering solution to perform filtering estimation of the vertical misalignment angle and vertical gyro drift. The estimated values of the vertical misalignment angle and vertical gyro drift will be fed back to the inertial navigation information 1 loop to correct the relevant inertial navigation errors, achieving the purpose of improving the information fusion accuracy.

[0042] Figure 5 is the simulation result of the Kalman filtering algorithm, Figure 5 in (a) is the simulation estimation result graph of the inertial navigation gyro drift, Figure 5 in (b) is the simulation estimation result graph of the misalignment angle, where Figure 5The initial set values have been deducted from the estimated values of the gyro drifts in Fig. (a). It can be seen from the simulation results that the north gyro drift Gn of the inertial navigation has good observability, and the convergence accuracy within 10 minutes is better than 0.001° / h; the east gyro drift Ge of the inertial navigation is unobservable, and this error is coupled with the vertical gyro drift; the vertical gyro drift Gu of the inertial navigation has poor observability, and the convergence accuracy within 10 hours can reach 0.0007° / h. The north and east misalignment angles of the inertial navigation have shown good convergence effects within 1 minute, while the heading misalignment angle has obtained a better convergence effect only after 10 minutes. It can be known from the simulation analysis that it is feasible to obtain accurate estimated values of the inertial navigation error parameters with poor observability through long-term filtering estimation.

[0043] As a further implementation manner, before extracting the target position information from the target library in S4, it further includes: Performing target recognition based on the lidar information to determine whether the measured target belongs to a new target; if it is a new target, expanding the target library and updating the new target position information. If it is not a new target, directly execute step S4.

[0044] The lidar rear-view sweeps the roadway to capture and identify the target. The target position information can be resolved by integrating the lidar and inertial navigation pose information. If the target is a known target, the target position information, inertial navigation information, and radar ranging information in the target information library can be fused to estimate and correct the inertial navigation error parameters, and then the lidar position information can be obtained (the relative position of the inertial navigation and the lidar is fixed). If the target is an unknown new target, it is necessary to correct the target position by integrating the information of several other known targets and then record and expand it into the target information library. In the actual operation process, new targets should be laid out one by one to avoid large errors in the new target positions due to too little information of known targets.

[0045] As a further implementation manner, S6 specifically includes: Fusing the inertial navigation information of the inertial navigation loop 1 in the dual-channel navigation solution with the first branch information and the second branch information by using the information fusion technology based on the factor graph to form fusion information.

[0046] As a further implementation manner, in terms of the anti-interference ability of information fusion, the observation information of the information fusion filter is optimized from the previous conventional position information to the original ranging information, which is based on the following considerations: First, obtaining position information requires integrating multiple ranging information. If some ranging information is contaminated, the positioning accuracy will be reduced or even unable to position. Second, if the situation of unable to position occurs, some effective ranging information will also be discarded, and this processing method seriously affects the adaptability of the positioning system to the harsh application environment of the shearer. Third, the original ranging information is convenient for effective detection.

[0047] The observation equations based on the original ranging information can be divided into two categories: (1) Using the calibration target points to calibrate the inertial navigation position error. Assuming that the latitude, altitude, and longitude of the calibration target point Si are Si( , , ), and the inertial navigation position information is ( , , ). The distance measured by the sensor between the two is L i . Its error equation should satisfy: (2) (2) Using the inertial navigation position information to calibrate the calibration target point position error. At this time, the error equation should satisfy: (3).

[0048] In the aspect of the plug-and-play engineering application of information fusion, the information fusion technology based on the factor graph is adopted.

[0049] The factor graph is a probabilistic graphical model and can be used for the information fusion of the navigation system. Abstracting the available navigation sensor measurement information into corresponding factor nodes, and recursively updating the state variable nodes according to the nonlinear optimization theory can solve the interference of the combined accuracy caused by various situations such as the simultaneous effectiveness, time-sharing effectiveness, and single information jump of multi-sensor information, and can make automatic decisions to improve the accuracy and robustness of the multi-sensor information fusion algorithm.

[0050] The factor graph is a bipartite graph model used to express the joint probability distribution of random variables. It includes two types of nodes: factor nodes, which refer to the local functions in the factorization; variable nodes, which refer to the variables in the global multivariate functions. Based on inertial navigation, it is combined with the measurement information from each sensor to abstract into corresponding factor nodes: the transfer function of the inertial / LiDAR combination sub-node , the transfer function of the inertial / mmWave radar combination sub-node , the transfer function of the inertial / odometer combination sub-node , the transfer function of the inertial / terrain matching combination sub-node … As Figure 6 shown, the transfer function between the state vectors from time k to time k + 1 is obtained by using the constraints between two adjacent time variable nodes, and for each time factor node, the allocation factor (i = 1, 2, … m) of each factor node will be obtained according to the credibility evaluation of each combined navigation factor at the current time , automatically adjusting the weight of information distribution, so as to realize the multi-sensor fusion navigation across scenarios.

[0051] The measurement information of the positioning sensors for the roadheader is related to the movement and pose state of the roadheader. Therefore, there is a strong correlation between the measurement information of each positioning sensor. The state vector fusion algorithm is an effective fusion algorithm for processing correlated data. However, this algorithm requires obtaining the correlation matrix between the estimation errors of all sensors, and it is difficult to obtain the correlation matrix in advance in the actual sensor network. Therefore, it is difficult to implement the state vector fusion algorithm. In this regard, the present application adopts the Covariance Intersection (CI) algorithm, which can effectively fuse the information of multiple sensors with unknown error correlations. The CI fusion formula is as follows:

[0052]

[0053] Among them, ω is a constant that minimizes the trace of the fused covariance matrix. The fused error covariance matrix given by the CI algorithm is the minimum upper bound for any error correlation between sensors, realizing a tight constraint on the intersection region of the covariance ellipses of each sensor and achieving the purpose of improving the fusion accuracy.

[0054] As a further implementation manner, the S3 further includes: collecting the information of other sensors and measurement devices; the S6 further includes: fusing the information of the collected other sensors and measurement devices, inertial navigation information, first branch information, and second branch information by using the information fusion technology based on the factor graph to form fused information.

[0055] The multi-sensor information fusion scheme of the present application, in addition to being able to fuse inertial navigation, lidar, and millimeter-wave radar information, also has the ability to fuse the information of other auxiliary positioning devices. Considering from the aspect of engineering application, the information fusion filter has the ability of plug-and-play for the integration of various auxiliary information.

[0056] As Figure 7 shown in the schematic diagram of the research idea of "Positioning Performance Analysis Based on Heterogeneous Multi-Sensor Information Fusion", based on the theoretical analysis of the error characteristics of single-sensor devices, the error characteristics of single devices under the working environment conditions in coal mines are studied. Combining the algorithm scheme of the present application, further carry out the research on the information fusion positioning error analysis of various auxiliary positioning device configuration methods with inertial navigation as the basis under the working environment conditions in coal mines, fully analyze the positioning performance of the precise positioning system for roadheaders under different working conditions, and provide theoretical guidance for the design and manufacture of mining equipment.

[0057] When studying the error characteristics in combination with environmental conditions, there is a problem that accurate modeling cannot be carried out when the error mechanism of the sensor in a complex environment is not clear. An artificial neural network-based sensor error modeling method is used for modeling. Based on the sample data of the current and past error information of each sensor, an artificial neural network is used to simulate the sensor error trend, and processing such as adaptive dynamic reference reconstruction, error model optimization, and reference feature extraction is carried out, so as to complete the non-linear estimation and modeling of the sensor error. The artificial neural network was born from the research on the working mechanism of the brain. It is a large-scale parallel distributed processing system composed of a large number of simple neurons with knowledge storage functions. It obtains knowledge from the outside world through learning, and then stores the knowledge through the weights connecting each neuron. It has a strong non-linear mapping ability, and at the same time has the advantages of real-time signal, self-adaptation, self-learning, and strong fault tolerance ability, which can provide a new idea for the research on the dynamic error modeling of sensors in complex environments.

[0058] Aiming at the problems of harsh working conditions in coal mining underground, such as a large amount of dust, curved roadways, and complex lighting conditions, this specific implementation method proposes a precise positioning system scheme for a roadheader as shown in Figure 8 Figure [Figure number not provided], adopting a multi-sensor fusion strategy with inertial navigation as the basis and lidar and millimeter-wave radar as the main auxiliary equipment to meet the requirements of less personnel and unmanned automated operation in coal mine underground roadway tunneling.

[0059] To reach the international advanced level and have the ability to support 24-hour all-day operation, the key equipment technical solutions of the positioning system are as follows.

[0060] Inertial navigation equipment The inertial navigation equipment is planned to adopt a single-axis fiber optic rotation modulation scheme. As shown in Figure 9, the single-axis fiber optic rotation modulation scheme fixes the IMU body on the equipment chassis through a single-axis rotation mechanism, so that the rotation mechanism (composed of a motor, an angle sensor, and a frame) can be driven to make the IMU rotate in the navigation coordinate system space according to a predetermined scheme. The IMU moving on a symmetric trajectory in space can have the ability of IMU parameter self-compensation, greatly improving the equipment's self-alignment accuracy and long-term navigation accuracy retention ability. The main performance of the selected single-axis fiber optic rotation modulation equipment is shown in Table 2.

[0061] Table 2 Performance indicators of the inertial navigation equipment to be selected

[0062] Lidar equipment The performance indicators of the selected lidar equipment are shown in Table 3.

[0063] Table 3 Performance indicators of the selected lidar equipment

[0064] Millimeter-wave radar device The main performance of the selected millimeter-wave radar device is shown in Table 4.

[0065] Table 4 Main performance indicators of the selected millimeter-wave device

[0066] During the actual working process of the precise positioning system for roadheaders, it gives full play to the high autonomy and high sensitivity characteristics of inertial devices to provide real-time position measurement and boom attitude measurement information for roadheaders. Auxiliary positioning devices such as lidar and millimeter-wave radar provide accurate position information for inertial navigation devices, playing a role in suppressing inertial errors and providing reference parameters for the control of roadheaders; to solve the problems of dynamic changes in underground coal mine roadway scenes and degradation of texture features, the method of artificially configuring targets is adopted to provide feature information for lidar and millimeter-wave radar devices; by installing lidar and millimeter-wave radar devices on roadheaders and assisting with multiple target configuration methods along with the advancement of coal mine roadways, the self-calibration of lidar position information can be achieved, and new targets can be automatically located, laying a foundation for realizing the automated work process of roadheaders.

[0067] In terms of information fusion algorithms, a dual-line filtering fusion scheme is adopted. The Kalman filtering channel mainly uses the inertial velocity and position information after information fusion as observables to accurately estimate the error parameters with poor observability of inertial navigation through long-term filtering estimation, and uses the estimated information to correct the inertial navigation parameters of the information fusion channel; the information fusion channel is based on inertial navigation information, fuses multiple sensor information for error estimation and correction. To improve the anti-interference ability of the information fusion algorithm, it is planned to optimize the observation information of the information fusion filter from the previous conventional position information to the original ranging information, and adopt the information fusion technology based on factor graph to make it have the engineering application ability of plug-and-play; considering that the measurement information of the positioning sensors for roadheaders is all related to the movement and pose state of roadheaders, and there is a strong correlation between the information, the CI algorithm is adopted to achieve the purpose of improving the fusion accuracy.

[0068] The algorithm implementation of the precise positioning system for roadheaders is all based on the geographical coordinate system. The underground roadway coordinate system can be obtained through the path of geographical coordinate system -> geocentric rectangular coordinate system -> underground roadway coordinate system. The conversion relationships between coordinate systems are all mature algorithms and will not be elaborated here.

[0069] According to the main equipment performance indicators selected above, the positioning and attitude accuracy simulation analysis of the precise positioning system for roadheaders is carried out. Assume that the working process and environmental conditions of the system are as follows:

[0070] 1) Preparation time: 10 min; 2) During the working process, the roadheader advances 20 m in 24 h; 3) During the working process, target layout is carried out every 20 m, and the targets are distributed at five positions of 12 o'clock, 2 o'clock, 4 o'clock, 8 o'clock and 10 o'clock in the roadway; 4) Considering the problem of environmental interference, the effective rate of the ranging information of the lidar is 70%, and the effective rate of the ranging information of the millimeter-wave radar is 90%; 5) The operation time is 24 h, and the estimation and correction of the lower error parameters of observability (heading misalignment angle and celestial gyro drift) are completed once every 10 h of operation.

[0071] According to the above operation process, the attitude and positioning errors of the precise positioning system of the roadheader are as shown in (a) and Figure 10 in (b). It can be seen from the information in the figure that under the conditions of this operation condition, the heading attitude error of the positioning system does not exceed 0.043°, the horizontal attitude error does not exceed 0.005°, the positioning error in the X direction of the roadway does not exceed ±4.1 cm, the positioning error in the Y direction of the roadway does not exceed ±7.9 cm, and the positioning error in the Z direction of the roadway does not exceed ±4.4 cm. Figure 10

[0072] Based on the above analysis, the satisfaction of the indicators of this embodiment is shown in Table 5. It can be seen from the information in the table that the index requirements of this embodiment are expected to be met.

[0073] Table 5 Satisfaction of the indicators of this embodiment

[0074] As Figure 2 shown, the present application also provides a heterogeneous multi-sensor information fusion system for precise positioning of coal mine underground mining equipment. The system includes a lidar, a millimeter-wave radar and an inertial navigator all installed on the roadheader. The lidar is configured with multiple targets that advance along with the coal mine roadway. The millimeter-wave radar obtains position information by using the target information library constructed by the lidar. The system further includes the following modules: An inertial navigation information acquisition module 210, which is used to collect inertial navigation information for navigation solution and extract the pose information of the roadheader from it. The inertial navigation information includes gyro information and accelerometer information of the inertial navigator; A transmission module 220, which is used to input the pose information of the roadheader into the millimeter-wave radar to assist the millimeter-wave radar in ranging; A radar information acquisition module 230, which is used to collect lidar information and millimeter-wave radar information. Both the lidar information and the millimeter-wave radar information include ranging information, and the ranging information is the distance information between the roadheader and the target measured by the lidar or the millimeter-wave radar; ​​​An extraction module 240, configured to extract target position information from a target library; A calculation module 250, configured to subtract the target position information from the ranging information of the lidar to construct first branch information, and subtract the target position information from the ranging information of the millimeter-wave radar to construct second branch information; An information fusion module 260, configured to fuse the collected inertial navigation information, first branch information, and second branch information by using an information fusion technology based on a factor graph to form fusion information; A feedback module 270, configured to return and transmit the fusion information to an inertial navigator; A correction module 280, configured to correct navigation information by the inertial navigator according to the fusion information.

[0075] As a further implementation manner, the inertial navigation information acquisition module includes a dual-channel navigation solution operator module, configured to perform dual-channel navigation solution according to gyro information and accelerometer information in the inertial navigation information; the dual-channel navigation solution refers to performing two-way inertial navigation solution simultaneously, including an inertial navigation loop 1 and an inertial navigation loop 2; the inertial navigation loop 1 is used for information fusion; the inertial navigation loop 2 is used for Kalman filtering solution to perform filtering estimation of the vertical misalignment angle and vertical gyro drift, and then feed back the estimated values of the vertical misalignment angle and vertical gyro drift into the inertial navigation loop 1 to correct the inertial navigation error.

[0076] As a further implementation manner, the system further includes an identification module, configured to perform target identification according to the lidar information before extracting the target position information from the target library, and determine whether the measured target belongs to a new target; if it is a new target, expand the target library and update the new target position information; if it is not a new target, start the extraction module.

[0077] As a further implementation manner, the information fusion module is specifically configured to fuse the inertial navigation information of the inertial navigation loop 1 in the dual-channel navigation solution with the first branch information and the second branch information by using an information fusion technology based on a factor graph to form fusion information.

[0078] As a further implementation manner, the information fusion module further includes an optimization sub-module, configured to optimize the observation information of the information fusion filter from conventional position information in the past to original ranging information; and after performing information fusion by using an information fusion technology based on a factor graph, further use a covariance intersection (CI) algorithm to improve the fusion accuracy; the CI fusion formula is shown in Equation (1): (1) where ω is a constant that minimizes the trace of the fused covariance matrix, and the fused error covariance matrix given by the CI algorithm is the least upper bound on any error correlation between sensors; Observation equations based on the original ranging information can be divided into two categories: (1) Using a target point to calibrate the position error of an inertial navigation system (INS). Assume that the latitude, altitude, and longitude of the target point Si are Si( , , ), the position information of the INS is ( , , ), and the distance measured by the sensor between the two is L i . Its error equation should satisfy: (2) (2) Using the position information of the INS to calibrate the position error of the target point. In this case, the error equation should satisfy: (3).

[0079] As a further implementation, the system further includes: an expansion module, configured to collect information of other sensors and measurement devices, and perform information fusion on the collected information of other sensors and measurement devices, inertial navigation information, first branch information, and second branch information by using an information fusion technology based on a factor graph to form fusion information.

[0080] For the device embodiment, since it basically corresponds to the method embodiment, it is described relatively simply. For the relevant parts, refer to the partial description of the method embodiment. The device embodiment described above is only illustrative. The units described as separate components may or may not be physically separated. The components shown as

[0081] units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units, such as distributed on a server and a client. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0082] At present, there is no single positioning device that can fully meet the performance requirements of coal mine underground operations. Therefore, this application provides a heterogeneous multi-sensor information fusion method. Considering the harsh working conditions of coal mining underground operations, such as a large amount of dust, curved roadways, and complex lighting conditions, the advantages and disadvantages of existing pose measurement schemes for mining equipment are comprehensively analyzed. A precise positioning system for mining equipment is proposed, which uses inertial navigation as the basis and lidar and millimeter-wave radar as the main auxiliary devices. The system adopts the multi-sensor fusion algorithm shown in Figure 3. This algorithm achieves the purpose of complementary advantages and mutual promotion through the deep fusion of various sensor information. Through key technology research such as multi-sensor information fusion schemes of lidar / millimeter-wave radar / inertial navigation and heterogeneous multi-sensor information fusion algorithms for precise positioning of coal mine underground mining equipment, a prototype of the precise positioning system for mining equipment is developed, and the multi-sensor information fusion algorithm proposed in this application is applied to the prototype. Through experimental verification, the following main index requirements are met: roadway forming error ≤ 15 cm; roadway X-axis positioning error does not exceed ±5 cm; roadway Y-axis positioning error does not exceed ±10 cm; roadway Z-axis positioning error does not exceed ±5 cm; attitude heading angle error of mining equipment (roadheader) does not exceed 0.1°; attitude roll angle error of mining equipment does not exceed 0.015°; attitude pitch angle error of mining equipment does not exceed 0.015°.

[0083] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A heterogeneous multi-sensor information fusion method for precise positioning of underground coal mining and excavation equipment, characterized in that This method installs a lidar on a roadheader and supplements it with multiple targets that advance along with the coal mine roadway; at the same time, a millimeter-wave radar is installed on the roadheader, and the millimeter-wave radar obtains position information by using the target information library constructed by the lidar; and an inertial navigator is also installed on the roadheader; the method specifically includes the following steps: S1: Collect inertial navigation information for navigation solution, and extract the pose information of the roadheader from it. The inertial navigation information includes gyro information and accelerometer information of the inertial navigator; S2: Input the pose information of the roadheader into the millimeter-wave radar to assist the millimeter-wave radar in ranging; S3: Collect lidar information and millimeter-wave radar information. Both the lidar information and the millimeter-wave radar information include ranging information, and the ranging information is the distance information between the roadheader and the target measured by the lidar or the millimeter-wave radar; S4: Extract the target position information from the target library; S5: Subtract the target position information from the ranging information of the lidar to construct the first branch information, and subtract the target position information from the ranging information of the millimeter-wave radar to construct the second branch information; S6: Use the information fusion technology based on the factor graph to fuse the collected inertial navigation information, the first branch information, and the second branch information to form fusion information; S7: Transmit the fusion information back to the inertial navigator; S8: The inertial navigator corrects the navigation information according to the fusion information.

2. The heterogeneous multi-sensor information fusion method for precise positioning of coal mine underground mining equipment according to claim 1, characterized in that, In S1, the navigation solution according to the gyro information and accelerometer information in the inertial navigation information is specifically a dual-channel navigation solution according to the gyro information and accelerometer information in the inertial navigation information; the dual-channel navigation solution means that two-way inertial navigation solutions are performed simultaneously, including an inertial navigation loop 1 and an inertial navigation loop 2; the inertial navigation loop 1 is used for information fusion; the inertial navigation loop 2 is used for Kalman filter solution to perform filtering estimation of the celestial misalignment angle and celestial gyro drift, and then feedback the estimated values of the celestial misalignment angle and celestial gyro drift to the inertial navigation loop 1 to correct the inertial navigation error.

3. A heterogeneous multi-sensor information fusion method for precise positioning of coal mine underground excavation equipment according to claim 1 or 2, characterized in that, Before extracting the target position information from the target library in S4, it also includes: Perform target recognition according to the lidar information to judge whether the measured target is a new target; if it is a new target, expand the target library and update the new target position information; if it is not a new target, directly execute step S4.

4. The heterogeneous multi-sensor information fusion method for precise positioning of coal mine underground mining equipment according to claim 2, wherein, S6 specifically includes: Use the information fusion technology based on the factor graph to fuse the inertial navigation information of the inertial navigation loop 1 in the dual-channel navigation solution with the first branch information and the second branch information to form fusion information.

5. A heterogeneous multi-sensor information fusion method for precise positioning of coal mine underground excavation equipment as described in claim 1 or 2 or 4, characterized in that, In S6 during information fusion, the observation information of the information fusion filter is optimized from the previous conventional position information to the original ranging information; and after using the information fusion technology based on the factor graph for information fusion, the covariance intersection algorithm is used to improve the fusion accuracy; the CI fusion formula is shown in Equation (1): （1） Among them, ω is a constant that minimizes the trace of the fused covariance matrix, and the fusion error covariance matrix given by the CI algorithm is the least upper bound for any error correlation between sensors; P denotes the error covariance matrix; the subscript a and b denote the information identifiers participating in the fusion, and the subscript c denotes the information identifier after fusion; The observation equations based on the original ranging information can be divided into two categories: (1) Using the calibration target point to calibrate the inertial navigation position error. Assume that the latitude, altitude, and longitude of the calibration target point Si are Si( , , ), and the latitude, altitude, and longitude of the inertial navigation (inertial navigation instrument) are( , , ). The distance measured by the sensor between the two is L i ; The symbol represents the error, that is L i represents L i 's error, and its error equation should satisfy: （2） (2) Use the inertial navigation position information to calibrate the target point position error, and at this time the error equation should satisfy: （3）。 6. A heterogeneous multi-sensor information fusion system for precise positioning of underground coal mining and excavation equipment, characterized in that, The system includes a lidar, a millimeter-wave radar, and an inertial navigator installed on a roadheader; multiple targets advancing with the coal mine roadway; a target library constructed by the lidar; the system further includes the following modules: An inertial navigation information acquisition module, configured to acquire inertial navigation information for navigation solution and extract the pose information of the roadheader therefrom, where the inertial navigation information includes gyro information and accelerometer information of the inertial navigator; A transmission module, configured to input the pose information of the roadheader into the millimeter-wave radar to assist the millimeter-wave radar in ranging; A radar information acquisition module, configured to acquire lidar information and millimeter-wave radar information, where both the lidar information and the millimeter-wave radar information include ranging information, and the ranging information is the distance information between the roadheader and the target measured by the lidar or the millimeter-wave radar; An extraction module, configured to extract target position information from the target library; A calculation module, configured to subtract the target position information from the lidar ranging information to construct a first branch of information, and subtract the target position information from the millimeter-wave radar ranging information to construct a second branch of information; An information fusion module, configured to fuse the acquired inertial navigation information, the first branch of information, and the second branch of information by using an information fusion technology based on a factor graph to form fusion information; A feedback module, configured to return the fusion information to the inertial navigator; A correction module, configured to correct the navigation information of the inertial navigator according to the fusion information.

7. The heterogeneous multi-sensor information fusion system for precise positioning of coal mine underground excavation equipment according to claim 6, wherein, The inertial navigation information acquisition module includes a dual-channel navigation solution sub-module, configured to perform dual-channel navigation solution according to the gyro information and accelerometer information in the inertial navigation information; the dual-channel navigation solution refers to performing two-channel inertial navigation solution simultaneously, including an inertial navigation loop 1 and an inertial navigation loop 2; the inertial navigation loop 1 is used for information fusion; the inertial navigation loop 2 is used for Kalman filter solution to perform filtering estimation of the vertical misalignment angle and the vertical gyro drift, and then feed the estimated values of the vertical misalignment angle and the vertical gyro drift back to the inertial navigation loop 1 to correct the inertial navigation error.

8. A heterogeneous multi-sensor information fusion system for precise positioning of coal mine underground excavation equipment according to claim 6 or 7, characterized in that The system further includes an identification module, configured to identify the target according to the lidar information before extracting the target position information from the target library, and determine whether the measured target belongs to a new target; if it is a new target, expand the target library and update the new target position information; if it is not a new target, start the extraction module.

9. A heterogeneous multi-sensor information fusion system for precise positioning of coal mine underground excavation equipment according to claim 7, characterized in that, The information fusion module is specifically configured to fuse the inertial navigation information of the inertial navigation loop 1 in the dual-channel navigation solution with the first branch of information and the second branch of information by using an information fusion technology based on a factor graph to form fusion information.

10. A heterogeneous multi-sensor information fusion system for precise positioning of coal mine underground excavation equipment according to claim 6 or 7 or 9, characterized in that, The information fusion module further includes an optimization sub-module, configured to optimize the observation information of the information fusion filter from the previous conventional position information to the original ranging information; and after fusing the information by using the information fusion technology based on the factor graph, further use the covariance intersection algorithm to improve the fusion accuracy; the CI fusion formula is shown in Equation (1): （1） where ω is a constant that minimizes the trace of the fused covariance matrix, and the fusion error covariance matrix given by the CI algorithm is the least upper bound of any error correlation between sensors; The observation equations based on the original ranging information can be divided into two categories: (1) Calibrating the position error of the inertial navigator using the calibration target point. Assume that the latitude, altitude, and longitude of the calibration target point Si are Si ( , , ), and the inertial navigation position information is ( , , ). The distance measured by the sensor between the two is L i . Its error equation should satisfy: （2） (2) Using the position information of the inertial navigator to calibrate the position error of the calibration target point. In this case, the error equation should satisfy: （3）。

Images