Visible light mixed fingerprint positioning method for underground mine environment

By constructing a hybrid fingerprint database in an underground mine, combining direct signal intensity and the arrival time of the reflection path, the BP neural network is used for optimization, which solves the problem of low positioning technology accuracy in underground mines and achieves high-precision visible-light hybrid fingerprint positioning.

CN118941620AActive Publication Date: 2024-11-12INNER MONGOLIA UNIV OF SCI & TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411028419.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-12
Estimated Expiration
2044-07-29

AI Technical Summary

Technical Problem

The existing positioning technology in underground mines is difficult to achieve high-precision positioning due to the weak GPS signal, short positioning distance of Bluetooth and ZigBee and large environmental impact, and the performance of fingerprint positioning based on direct signal strength in dark wall environments.

Method used

A hybrid fingerprint positioning method for visible light in underground mine environment is adopted. By collecting point cloud data in tunnel space, a complete point cloud model is constructed, combining direct signal intensity and arrival time of the first reflection path, a mixed feature fingerprint database is established, and the use of BP neural network is optimized to achieve high-precision positioning.

Benefits of technology

This method can effectively solve the problem of reduced positioning performance caused by the propagation of obstacle blocking light signals in underground mines, improve positioning accuracy and reliability, and achieve more accurate positioning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941620A_ABST
    Figure CN118941620A_ABST
Patent Text Reader

Abstract

The invention discloses a visible light mixed fingerprint positioning method for an underground mine environment, and relates to the technical field of underground mine positioning. According to the method, on the basis that the signal arrival time only depends on the length of a propagation path and is not influenced by wall surface materials and colors, the first arrival path has uniqueness, high-precision positioning of direct signal strength is taken as a reference, and the arrival time of a first reflection path is combined to construct a mixed feature fingerprint database; the positioning technical defect that the performance is reduced due to the fact that obstacles in an underground mine block light signal propagation is overcome.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of underground mine positioning, and in particular to a visible light hybrid fingerprint positioning method in an underground mine environment. Background Art

[0002] The exploration and development of mineral resources have made important contributions to promoting the national economy and social development. The mining production technology and management methods have changed from extensive to intensive, relying on scientific and technological progress, implementing development and protection in both directions, improving resource exploitation and utilization while doing a good job in safety management, especially in dangerous underground mines, building real-time and reliable communication base stations and systems, and strengthening the positioning and tracking of staff to ensure that workers do not go deep into dangerous areas and ensure their safety.

[0003] The existing communication system in underground mines combines wired and wireless methods. Wired communication is mainly based on carrier communication, and a large number of cable equipment will be laid underground, such as the use of lighting sources. Traditional radio frequency wireless communication will produce strong electromagnetic effects and is not suitable for use in closed underground mines. Wireless systems that rely on visible light communication have become the first choice to solve this problem. Among the existing positioning technologies, GPS is limited by the depth of the mine, the signal is weak, and the positioning effect is not ideal. Bluetooth and ZigBee positioning methods not only have a short positioning distance, but are also greatly affected by complex environments. Green and environmentally friendly visible light communication has no electromagnetic interference and meets wireless communication and positioning applications at the same time. It is an effective method to solve the positioning of underground personnel.

[0004] Visible light positioning technology can be divided into three types: proximity method, triangulation method and fingerprint positioning method. The proximity method depends on the number of transmitters. The transmitter sends out a light signal carrying location information. The positioning point receives the signal strength of different transmitters to infer which transmitter is closer, and then judge the coordinates of the positioning point. The triangulation method requires three or more transmitters and determines their location coordinates. Then the arrival time, signal strength, arrival angle, arrival time difference, etc. of the signals of different transmitters at the positioning point are measured. The objective function is established based on the distance relationship between the transmitter and the receiver to estimate the coordinates of the positioning point. The fingerprint positioning method is divided into two stages: offline database establishment and online measurement and matching. In the offline stage, fingerprint information (signal strength value, signal arrival time, arrival angle, etc.) of each position in the positioning plane needs to be collected to establish a database based on the corresponding relationship between position coordinates and fingerprint values. In the online stage, the fingerprint value of the unknown positioning point is measured and compared with the information in the offline database to finally estimate the coordinates of the positioning point.

[0005] Light sources in narrow underground mines are usually arranged in a collinear manner, with a lamp placed at the top of the tunnel at a long distance. Therefore, the proximity method and triangulation positioning method are not suitable for mine environments, and spatial segmentation single-lamp fingerprint positioning has become the preferred method. Among the common fingerprint options, the measurement method of received signal strength is simple and the equipment cost is low. It is widely used in different scenarios and has formed a complete set of mature positioning systems. In the transmission of visible light signals at fixed positions, the line-of-sight link corresponds to a unique path, while the non-line-of-sight link is a beam that reaches the receiver after being reflected by walls, the ground, etc. The propagation path is rich and the direction is uncertain, which easily forms a multipath effect. Therefore, most positioning methods only use direct signal strength to eliminate the uncertainty error caused by multipath reflection. However, the underground environment has a complex structure, and the uneven walls cause slight changes in the reflection angle and direction of the light. Obstacles that block light propagation cause the disappearance of direct signals in a certain area. The quality of the fingerprint positioning database based on direct signal strength is severely damaged, and the positioning effect is very unsatisfactory. At present, a team has studied the visible light positioning method combined with the reflected signal strength to assist in harsh environments. The results show that the positioning performance has been significantly improved. What is less than ideal is that the walls in the mine environment are mostly deep dark in color and have good light absorption properties, resulting in extremely low reflected light signal intensity and significantly reduced application performance. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a visible light hybrid fingerprint positioning method for underground mine environments. This method is based on the fact that the signal arrival time only depends on the length of the propagation path, is not affected by the wall material and color, and the first arrival path is unique. It takes the high-precision positioning of the direct signal intensity as the benchmark, and combines the arrival time of the first reflection path to construct a hybrid feature fingerprint database to solve the positioning technology defect of reduced performance caused by obstacles blocking the propagation of light signals in underground mines.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A visible light hybrid fingerprint positioning method for an underground mine environment, the method comprising:

[0009] Step 1: Collect initial point cloud data including walls and obstacles in the tunnel space to generate multiple initial point clouds.

[0010] Step 2: Apply the point cloud denoising algorithm to the initial point cloud data obtained in step 1 to filter out outliers and noise points; apply the point cloud registration algorithm to the denoised multiple point clouds, and regularly merge the multiple point clouds representing the same object based on the points with the same characteristics, so as to preliminarily obtain the overall point cloud model of the two walls and obstacles in the tunnel space.

[0010] Step 3: Input the coordinate data of the overall point cloud model of the missing points into the pre-trained PoinTr deep network to complete the point cloud, thereby constructing a more complete overall point cloud model.

[0011] Step 4: Match the overall point cloud model after point cloud completion with the rectangular coordinate system constructed by the actual scene.

[0012] Step 5: Based on the matched point cloud data in step 4, a visible light propagation model of the underground tunnel space is constructed. The visible light propagation model includes a channel response model, and a calculation model for the direct signal intensity and the arrival time of the first reflection path.

[0013] Step 6: Based on the mathematical model obtained in step 5, a hybrid fingerprint database is established.

[0014] Step 7: Normalize the hybrid fingerprint database obtained in step 6, and use the processed database to train the BP neural network optimized by the thinking evolution algorithm to build a prediction model; the BP neural network uses the fingerprint information in the database as training input and the coordinate information as training output, and generates a nonlinear expression between input and output under large-scale data iterative learning.

[0015] Step 8: After the prediction model is constructed, the transmitter signal in the tunnel space is received by the receiver for positioning.

[0016] Furthermore, the denoising algorithm in step 2 adopts any one of voxel filtering, Gaussian filtering, and statistical filtering.

[0017] Furthermore, the registration algorithm in step 2 adopts an iterative closest point algorithm or a deep network registration algorithm.

[0018] Furthermore, in step 3, the training method of the pre-trained PoinTr deep network includes:

[0019] Step 3.1: Collect real point cloud data similar to the underground mine environment.

[0020] Step 3.2: Perform step 2 on the point cloud data obtained in step 3.1.

[0021] Step 3.3: Based on the open3d library programming, the point cloud obtained in step 3.2 is cropped. After cropping, each point cloud is based on the downsampling algorithm, retaining only N points and saved as a .ply file.

[0022] Step 3.4, based on open3d library programming, extract the point cloud coordinates of the .ply file formed after cropping, form a point cloud coordinate vector group after normalization, and input the point cloud coordinate vector group into the PoinTr network for training; the point cloud coordinate vector group is divided into training data and verification data in a ratio of 4:1; use the verification data to cross-validate the completion models with different parameters to obtain the best completion model; input the overall point cloud model of the missing points obtained in step 2 into the best completion model to complete the point cloud completion.

[0023] Furthermore, in step 4, a three-dimensional space rigid body transformation algorithm is applied to match the wall and obstacle point clouds in the camera coordinate system with the rectangular coordinate system constructed by the actual scene.

[0024] Furthermore, in step 4, the obstacle point cloud model is plane rectified.

[0025] The beneficial effects of the present invention are as follows: the present invention provides a visible light hybrid fingerprint positioning method for an underground mine environment. The method is based on the fact that the signal arrival time only depends on the length of the propagation path, is not affected by the wall material and color, and the first arrival path is unique. Based on the high-precision positioning of the direct signal intensity, a hybrid feature fingerprint database is constructed in combination with the arrival time of the first reflection path to solve the positioning technology defect of reduced performance caused by obstacles blocking the propagation of light signals in underground mines. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of a specific scenario of the present application.

[0027] Figure 2 It is a schematic diagram of the specific position coordinates of the cart in this application.

[0028] Figure 3 This is a partial screenshot of the .ply file of an initial point cloud of a wall collected by scanning the alley space using the ZED2i binocular camera in this application.

[0029] Figure 4 This is a comparison chart of the initial point cloud data of a wall in this application before and after denoising.

[0030] Figure 5 This is a display diagram of the overall point cloud model constructed by the point cloud registration algorithm used in this application.

[0031] Figure 6 This is a comparison diagram of the point cloud model before and after point cloud completion of this application, where the left side is the point cloud model diagram before point cloud completion, and the right side is the point cloud model diagram after point cloud completion.

[0032] Figure 7 This is a schematic diagram showing the angle difference between the overall point cloud obstacle captured by the depth camera and the placement of the mine obstacle.

[0033] Figure 8 This is a comparison diagram of the obstacle position before and after the coordinate transformation of the present application.

[0034] Fig. 9 This is a top view of the array of 5 receivers of this application.

[0035] Fig.10 This is a positioning effect diagram of the hybrid fingerprint positioning method of this application.

[0036] Fig.11 It is the positioning error probability distribution diagram of the hybrid fingerprint positioning method of this application.

[0037] Fig.12 This is the positioning effect diagram of the single-feature fingerprint positioning method.

[0038] Fig.13 It is the positioning error probability distribution diagram of the single feature fingerprint positioning method.

[0039] Fig.14 It is a flowchart of the application method. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] Below, this application describes a visible light hybrid fingerprint positioning method for an underground mine environment provided by the present invention in conjunction with a specific scenario.

[0042] like Figure 1 The figure shows a schematic diagram of a specific scenario of the present application. In an underground tunnel space of length l (m), width w (m), and height h (m) (specifically 6m long, 4m wide, and 2.5m high), a three-dimensional rectangular coordinate system is established with a vertex at the bottom of the tunnel space as the coordinate origin. The x-axis direction of the rectangular coordinate system is the same as the length direction of the tunnel space, the y-axis direction of the rectangular coordinate system is the same as the width direction of the tunnel space, and the z-axis direction of the rectangular coordinate system is the same as the height direction of the tunnel space. After establishing the three-dimensional rectangular coordinate system, the position coordinates of each point in the tunnel space can be determined, and the spatial position coordinates of obstacles in the space can also be determined, laying the foundation for the subsequent establishment of a prediction model.

[0043] In addition, a transmitter (i.e., LED light) is set in the center of the top surface of the tunnel space, and its position coordinates can be determined according to the established rectangular coordinate system, which is recorded as (x t ,y t,h), in this specific scene, the position coordinates of the transmitter are (3m, 2m, 2.5m). Furthermore, a cart is placed in the space to simulate an obstacle. The length of the cart is 1.36 (m) (x-axis direction), the width is 0.68 (m) (y-axis direction), and the height is 0.64 (m) (z-axis direction).

[0044] The specific location coordinates of the cart are defined as follows: Figure 2 Shown is a schematic diagram of the specific position coordinate definition of the cart in the present application; in the present application, a rectangular parallelepiped surrounding the cart is formed according to the length, width and height of the cart, and the three-dimensional coordinates of the minimum points of the x-component, y-component and z-component of the rectangular parallelepiped are (2.5m, 2.2m, 0m), and the cart as a whole is perpendicular to the ground (xoy plane) and parallel to the xoz plane.

[0045] It should be noted that, in the present application, the selection of the coordinate origin and the directions of the x-axis, y-axis, and z-axis can be set according to actual conditions. This application only provides examples to illustrate the technical solution of the present application, and does not specifically limit the method of establishing a rectangular coordinate system.

[0046] Below, based on the above specific scenario, the specific content of the visible light hybrid fingerprint positioning method for an underground mine environment provided by the present invention is described in detail.

[0047] like Fig.14 As shown, the present invention provides a visible light hybrid fingerprint positioning method for an underground mine environment, comprising the following specific steps:

[0048] Step 1: Use a depth camera or laser device to scan the tunnel space multiple times and at multiple angles, collect the initial point cloud data of the tunnel space, and generate multiple pieces of initial point clouds for building the overall point cloud model. The initial point cloud data is saved as a .ply file in the form of the three-dimensional spatial coordinates of the points. In this application, the initial point cloud data of the tunnel space walls and obstacles are collected, and the initial point cloud data of multiple pieces of tunnel space walls and multiple pieces of obstacles are generated.

[0049] like Figure 3 The following is a partial screenshot of the .ply file of the initial point cloud of a wall collected by scanning the lane space using the ZED2i binocular camera in this application. Figure 3 As shown in , the ply file records the position coordinates of all points in the initial point cloud of the piece. The position coordinates are formed based on the camera coordinate system, which is different from the rectangular coordinate system artificially set in the specific scene. The two need to be unified in the subsequent steps of this application.

[0050] Step 2: Use the initial point cloud data obtained in step 1 to construct the overall point cloud model. This includes the following steps:

[0051] Step 2.1: Apply a point cloud denoising algorithm to the initial point cloud data obtained in step 1 to filter out outliers and noise points. The denoising algorithm is not limited to voxel filtering, Gaussian filtering, statistical filtering, etc.

[0052] like Figure 4 Shown is a comparison chart of the initial point cloud data of a wall in the present application before and after denoising, wherein the left side is the point cloud data chart before denoising, and the right side is the point cloud data chart after denoising.

[0053] Step 2.2: Apply the point cloud registration algorithm to the denoised point clouds, and regularly merge the points with the same characteristics to obtain the overall point cloud model of the two walls and obstacles in the tunnel space. The registration algorithm can be an iterative closest point algorithm, a deep network registration algorithm, etc.

[0054] like Figure 5 Shown is a display diagram of the overall point cloud model constructed by the point cloud registration algorithm used in this application.

[0055] Furthermore, due to the dim underground environment, point cloud collection is difficult and the amount of point cloud data is small, resulting in varying degrees of data missing in the constructed overall point cloud model; therefore, for the case of a small amount of point cloud data, this application provides a point cloud completion solution. The point cloud completion solution is described in step 3 below.

[0056] Step 3: Input the coordinate data of the overall point cloud model of the missing points into the pre-trained PoinTr deep network to complete the point cloud, thereby constructing a more complete overall point cloud model.

[0057] In this application, the method of pre-training the PoinTr deep network is as follows:

[0058] The point cloud model included in the public data set generally used in the PoinTr deep network training operation lacks the collection and mapping of the underground mine environment walls, background structures, etc., resulting in low accuracy of the completion model. Therefore, in this application, the PoinTr deep network is trained based on the point cloud data collected in the field, including the following steps:

[0059] Step 3.1. Use a depth camera or laser device to collect more real point cloud data similar to underground mine environments, such as underground simulated mine bases, underground parking lots, tunnels, etc., generate a large amount of point cloud data, and save it as a .ply file.

[0060] Step 3.2: Perform step 2 on the point cloud data obtained in step 3.1.

[0061] Step 3.3: Based on the open3d library programming, the point cloud obtained in step 3.2 is cropped. Each point cloud after cropping is based on the downsampling algorithm, retaining only N points (for example, 50,000 points) and saved as a .ply file. The purpose of the cropping operation is to increase the training data and crop a point cloud into more than two parts.

[0062] Step 3.4 is based on open3d library programming, and the point cloud coordinates of the .ply file formed after cropping are extracted. After normalization, a point cloud coordinate vector group is formed, and the point cloud coordinate vector group is input into the PoinTr network for training. Specifically, the point cloud coordinate vector group is divided into training data and verification data in a ratio of 4:1. Based on the powerful feature capture capability of the Transformer deep learning architecture, the training data is used to complete the learning of the completion model, and the verification data is used to cross-validate the completion models with different parameters to obtain the best completion model. The overall point cloud model of the missing points obtained in step 2 of this application is input into the best completion model to complete the point cloud completion.

[0063] like Figure 6 Shown is a comparison diagram of the point cloud model before and after point cloud completion of the present application, wherein the left side is the point cloud model diagram before point cloud completion, and the right side is the point cloud model diagram after point cloud completion.

[0064] The coordinates of the above point cloud model in this application are based on the camera coordinate system, while the rectangular coordinate system in the specific scene is artificially defined. Therefore, this application needs to match the overall point cloud model after point cloud completion with the rectangular coordinate system constructed by the actual scene, as described in step 4 below.

[0065] Step 4: Match the overall point cloud model after point cloud completion with the rectangular coordinate system constructed by the actual scene.

[0066] In the camera coordinate system, the positive direction of the x-axis is directly to the right, the positive direction of the y-axis is directly upward, and the positive direction of the z-axis is directly forward. The rectangular coordinate system constructed in the actual scene is as follows Figure 1 As shown, the positive direction of the x-axis is directly in front, the positive direction of the y-axis is directly to the right, and the positive direction of the z-axis is directly above. Apply the three-dimensional space rigid body transformation algorithm to match the wall and obstacle point cloud in the camera coordinate system with the rectangular coordinate system constructed in the actual scene, as shown in Formula 1:

[0067] In formula 1, (x c ,y c ,z c ) is the point cloud coordinate before matching, (x w ,y w ,z w ) represents the coordinates of the matched point cloud, [T x ,T y ,Tz ] T Represents the translation matrix. R c-w It is the rotation matrix, which is the rotation angle of the rectangular coordinate system constructed by matching the actual scene after rotating the x, y, and z axes in the camera coordinate system Calculated, as shown in Formula 2:

[0068] It should be noted that since the obstacle is a 360-degree stereo structure, the depth camera needs to constantly change the shooting angle around the obstacle when shooting the obstacle. Therefore, the camera perspectives of the multiple pieces of initial point cloud data of the obstacle generated in step 1 are different, resulting in a slight tilt of the overall obstacle point cloud after registration in step 2, that is, the overall normal vector of the obstacle is not strictly perpendicular to the ground and parallel to the xoz plane.

[0069] like Figure 7 The figure shows the angle difference between the overall point cloud obstacle captured by the depth camera and the position of the mine obstacle. The obstacle point cloud is matched with the rectangular coordinate system constructed by the actual scene through the rigid body transformation in three-dimensional space. Figure 7 The red point cloud after matching corresponds to the point cloud taken by the depth camera with a slight tilt, and the white point cloud corresponds to the plane and point coordinates of the mine obstacle. The two point clouds have obvious angle deviations. Therefore, this application also needs to perform additional plane correction on the obstacle point cloud model. The specific means is to fit the plane parameters of the obstacle point cloud in combination with the least squares method, and then obtain the overall normal vector of the obstacle, and use the Rodriguez method to correct the slightly tilted obstacle point cloud plane. During the plane correction process, the coordinates of the obstacle produce a slight deviation. After that, continue to adjust the position of the obstacle to match the actual obstacle coordinate position. Specifically, the obstacle coordinates adjusted by the plane correction are transformed into the actual obstacle coordinates through translation.

[0070] like Figure 8 Shown is a comparison diagram of the obstacle position coordinates before and after the transformation of the present application. The left side is a screenshot of the coordinate part before matching, and the right side is a screenshot of the coordinate part after matching.

[0071] Step 5: Based on the matched point cloud data in step 4, a visible light propagation model of the underground tunnel space is constructed. The visible light propagation model includes a channel response model, and a calculation model for the direct signal intensity and the arrival time of the first reflection path.

[0072] The channel response model is established as follows:

[0073] Step 5.1, characterize the influence of the complex concavity and convexity of the wall in the mine on the visible light channel. The parameter calculation process is to combine the plane projection method and the point-by-point insertion algorithm to perform Delaunay triangulation on the three-dimensional point cloud data of the wall in the point cloud model after matching in step 4. The scattered and disordered point cloud simulates the concave and convex structure of the real wall in the form of a triangular mesh. Each triangular mesh is regarded as a reflection face element, and the face element area is calculated from the three-dimensional coordinates of the vertex of the face element Coordinates of the center of gravity of the surface element and the surface normal vector

[0074] Step 5.2: Characterize the impact of the shadow formed by the obstacle under illumination on the visible light channel. The parameter calculation process is to use the point cloud coordinate data of the obstacle to calculate the shadow point set P of the LED light rays passing through each point of the obstacle falling on the receiving plane. s , combined with the iterative progressive convex hull algorithm, we get the point set P s The minimum convex polygon structure is regarded as the shadow area D generated when the LED light source illuminates the obstacle. r ,y r ,z r ) is contained in the convex polygon D, the light ray is blocked, and the shadow coefficient α is expressed as the following formula 3:

[0075] For a single reflection channel, the centroid of each reflection surface element of the wall is regarded as a point light source. Repeat the above steps for each reflection point light source to obtain the corresponding convex polygon D w , and further obtain the corresponding shadow coefficient α w .

[0076] Step 5.3: Combining the two processes of step 5.1 and step 5.2, and considering the characterization parameters of the two influencing factors of the wall and the obstacle, the direct channel response expression of the tunnel space can be determined as follows:

[0077] In formula 4, m is the Lambertian radiation mode number, A p is the physical receiving area of ​​the photoelectric receiver, d tr represents the distance between the transmitter and the receiver, φ tr represents the radiation angle of the transmitter, θ tr represents the incident angle of the direct channel receiver, G(θ tr ) is the product of the filter and concentrator gain at the receiver, c is the speed of light, δ(·) is the impulse function, T is the transmitter coordinate, and R is the receiver coordinate.

[0078] The expression of the first reflection channel response is as follows:

[0079] In formula 5, W represents the total number of reflection elements, ρ represents the wall reflection coefficient, and d tw and d wr Respectively represent the distance between the transmitter and the reflection surface element, and the distance between the reflection surface element and the receiver. tw and φ wr They represent the incident angle and radiation angle of the wth reflection element of the wall, θ wr is the receiver incidence angle of the primary reflection channel.

[0080] The calculation model for the direct signal strength and the arrival time of the first reflected path is established as follows:

[0081] Step 5.4: First, determine the signal strength mathematical model related to time through the channel response model. The specific method is as follows:

[0082] In practical applications, the visible light channel adopts the intensity modulation / direct reception method, and the transmitted waveform is the DC optical power signal x(t)=P of the LED t , P t is the emitted light power of the LED. The receiver detects the photocurrent signal as shown in the following formula 6:

[0083] In Formula 6, γ is the receiver responsivity, and N(t) represents the Gaussian noise generated in the channel.

[0084] After amplification, filtering, sampling and other operations, the photocurrent signal outputs a discrete signal in the time domain, which is an ordered sampling time signal X(t) = {X(t0), X(t1), ..., X(t n )}, the amplitude variable is the signal strength P at the sampling time r (t) = {P r (t0),P r (t1),...,P r (t n )}, the signal strength is simulated as the ratio of the photocurrent signal to the receiver responsivity γ.

[0085] Step 5.5: Determine the fingerprint feature according to the signal strength mathematical model in step 5.4, that is, the calculation model of the direct signal strength and the arrival time of the first reflection path. The specific method is as follows:

[0086] According to the propagation path characteristics of visible light signals, the direct path is unique and has the shortest length. Therefore, the receiver at each location always receives the direct signal first. According to the distribution characteristics of visible light intensity in underground mines, the contribution of direct light signals is much higher than that of reflected signals. Therefore, the minimum threshold of direct signals is set to p. los , then the direct signal strength P LOS For sequence P r The only variable P in (t) that meets the signal threshold LOS =P r (t i )≥p los , otherwise, P LOS =0.

[0087] The strength value of the first reflected path reaching the receiver is the sequence P r The first variable {P r (t i )<p los} i_min , at this time, the first reflection path arrives at time t f =t i_min .

[0088] Step 6: Establish a hybrid fingerprint database based on the mathematical model obtained in step 5. The specific method is as follows:

[0089] Step 6.1, determine the array form of the receiver. Using a single receiver can only collect two fingerprint values, including the direct signal strength value corresponding to the receiver coordinates and the arrival time value of the first reflection path. The positioning effect based on the two-dimensional fingerprint vector is poor. Therefore, the present invention uses 5 receivers, such as Fig. 9 The figure shows a top view of the array of five receivers of the present application. A receiver is placed at the center of the fingerprint point, and a receiver is placed at intervals of d (cm) in front, behind, left and right of the receiver, to collect a total of 10-dimensional fingerprint vectors.

[0090] Step 6.2: Divide the positioning plane grid in the tunnel space. Delineate a rectangular grid with intervals of lx(m) and ly(m) along the x-axis and y-axis respectively. A total of l(m)×w(m)×h(m) is obtained in the tunnel space. grids. The receiving array is placed with the center of each grid as the fingerprint point coordinate. Taking the i-th fingerprint point as an example, its real coordinate is (x i ,y i ,z i ), collect the direct signal strength values ​​of the five receivers in the array and the arrival time of the first reflected path, that is, C0{P LOS ,t f |(x i ,y i ,zi )},C1{P LOS ,t f |(x i -d,y i ,z i )},C2{P LOS ,t f |(x i +d,y i ,z i )},C3{P LOS ,t f |(x i ,y i -d,z i )},C4{P LOS ,t f |(x i ,y i +d,z i )}. Therefore, the information corresponding to the i-th fingerprint point in the database is as shown in the following formula 7: F i =[x i ,y i ,z i ,P LOSi0 ,P LOSi1 ,P LOSi2 ,P LOSi3 ,P LOSi4 ,t fi0 ,t fi1 ,t fi2 ,t fi3 ,t fi4 ] (Formula 7).

[0091] In formula 7, P LOSij =C j {P LOS}, t fij =C j {t f}, where j = 0, 1, ..., 4. LOS With t f The calculation model is shown in step 5.5.

[0092] Step 7: normalize the hybrid fingerprint database obtained in step 6, and use the processed database to train the BP neural network optimized by the thinking evolution algorithm to build a prediction model.

[0093] The above BP neural network has excellent nonlinear fitting ability and can transform the fingerprint information in the database As training input, coordinate information As training output, a nonlinear expression between input and output, namely the prediction model, is generated through iterative learning of a large amount of data.

[0094] The above-mentioned thinking evolution algorithm belongs to the evolutionary intelligent optimization algorithm, which simulates the "convergence" and "alienation" of human thinking characteristics, and searches for the optimal solution in the population space through the idea of ​​"variation inheritance". The purpose of applying the optimization algorithm is to improve the defects of random setting of relevant parameters in the BP neural network, the parameters include initial weights and thresholds, find the optimal values ​​of the parameters, and improve the prediction performance.

[0095] Once the prediction model is built, underground positioning can be achieved by simply carrying an array of five receivers on the underground personnel, such as on a safety helmet.

[0096] In order to prove the effectiveness of the method of the present invention, the present invention provides the following experiments for verification.

[0097] The 10-dimensional fingerprint vector collected at the prediction point is input into the mature neural network prediction model trained in step 7 for coordinate positioning, and the error values ​​of the predicted output coordinates and the real coordinates are calculated at the same time. The average positioning error value is calculated as follows: Formula 8, and the root mean square positioning error value is calculated as follows: Formula 9:

[0098] Among them, N represents the number of prediction points, (x ai ,y ai ,z ai ) represents the real coordinates, (x pi ,y pi ,z pi ) represents the predicted coordinates.

[0099] According to the above method, the present application scans a wall and obstacles in a real environment and builds Figure 1 The visible light positioning system model shown in the figure realizes the simulation positioning experiment and outputs the error results. The parameters and results in the specific implementation case are as follows:

[0100] Step 1: Use a depth camera or laser device to scan the tunnel space multiple times and at multiple angles to collect the initial point cloud data of the tunnel space, generate multiple initial point clouds of the tunnel space walls and multiple initial point clouds of obstacles, and use them to build the initial point cloud model. The initial point cloud data is saved as a .ply file in the form of the three-dimensional spatial coordinates of the points.

[0101] Step 2: Use the initial point cloud data obtained in step 1 to construct the overall point cloud model. This includes the following steps:

[0102] Step 2.1: Apply a point cloud denoising algorithm to the initial point cloud data obtained in step 1 to filter out outliers and noise points. The denoising algorithm is not limited to voxel filtering, Gaussian filtering, statistical filtering, etc.

[0103] Step 2.2: Apply the point cloud registration algorithm to the denoised point clouds, and regularly merge the points with the same characteristics to obtain the overall point cloud model of the two walls and obstacles in the tunnel space. The registration algorithm can be an iterative closest point algorithm, a deep network registration algorithm, etc.

[0104] Step 3: Input the coordinate data of the overall point cloud model of the missing points into the pre-trained PoinTr deep network to complete the point cloud, so as to construct a relatively complete overall point cloud model.

[0105] In this application, the method of pre-training the PoinTr deep network is as follows:

[0106] Since the point cloud model included in the public dataset lacks the collection and mapping of underground mine environment walls, background structures, etc. In this application, the PoinTr deep network is trained based on the point cloud data collected in the field, including the following steps:

[0107] Step 3.1. Use a depth camera or laser device to collect more real point cloud data similar to underground mine environments, such as underground simulated mine bases, underground parking lots, tunnels, etc., generate a large amount of point cloud data, and save it as a .ply file.

[0108] Step 3.2: Perform step 2 on the point cloud data obtained in step 3.1.

[0109] Step 3.3: Based on the open3d library programming, the point cloud obtained in step 3.2 is cropped. Each point cloud after cropping is based on the downsampling algorithm, retaining only 50,000 points and saved as a .ply file. The purpose of the cropping operation is to increase the training data and crop a point cloud into more than two parts.

[0110] Step 3.4 is based on open3d library programming, and the point cloud coordinates of the .ply file formed after cropping are extracted. After normalization, a point cloud coordinate vector group is formed, and the point cloud coordinate vector group is input into the PoinTr network for training. Specifically, the point cloud coordinate vector group is divided into training data and verification data in a ratio of 4:1. Based on the powerful feature capture capability of the Transformer deep learning architecture, the training data is used to complete the learning of the completion model, and the verification data is used to cross-validate the completion models with different parameters to obtain the best completion model. The overall point cloud model of the missing points obtained in step 2 of this application is input into the best completion model to complete the point cloud completion.

[0111] Step 4: Match the overall point cloud model after point cloud completion with the rectangular coordinate system constructed by the actual scene.

[0112] First, the three-dimensional rigid body transformation algorithm is used to match the wall and obstacle point cloud coordinates in the camera coordinate system with the rectangular coordinate system constructed by the actual scene. The matching process is shown in Formula 1. In this case, the positive direction of the x-axis of the camera coordinate system is directly to the right, the positive direction of the y-axis is directly upward, and the positive direction of the z-axis is directly forward. The rectangular coordinate system constructed by the actual scene is as follows: Figure 1 As shown in the figure, the positive direction of the x-axis is directly forward, the positive direction of the y-axis is directly to the right, and the positive direction of the z-axis is directly upward. Therefore, the x, y, and z axes in the camera coordinate system are rotated 90 degrees to match the rectangular coordinate system constructed by the actual scene.

[0113] It should be noted that since the obstacle is a 360-degree stereo structure, the depth camera needs to constantly change the shooting angle around the obstacle when shooting the obstacle. Therefore, the camera perspectives of the multiple pieces of initial point cloud data of the obstacle generated in step 1 are different, resulting in a slight tilt of the overall obstacle point cloud after registration in step 2, that is, the overall normal vector of the obstacle is not strictly perpendicular to the ground and parallel to the xoz plane.

[0114] like Figure 7 The figure shows an angle difference between the overall point cloud obstacle captured by the depth camera and the placement of the mine obstacle. Therefore, the present application also requires additional plane correction of the obstacle point cloud model. The specific means is to fit the plane parameters of the obstacle point cloud in combination with the least squares method, and then obtain the overall normal vector of the obstacle, and use the Rodriguez method to correct the slightly tilted obstacle point cloud plane. During the plane correction process, the coordinates of the obstacle produce a slight deviation. After that, continue to adjust the position of the obstacle to match the actual obstacle coordinate position. Specifically, the obstacle coordinates adjusted by the plane correction are transformed into the actual obstacle coordinates through translation.

[0115] Step 5: Based on the matched point cloud data in step 4, a visible light propagation model of the underground tunnel space is constructed. The visible light propagation model includes a channel response model, and a calculation model for the direct signal intensity and the arrival time of the first reflection path.

[0116] The channel response model is established as follows:

[0117] Step 5.1, characterize the influence of the complex concavity and convexity of the wall in the mine on the visible light channel. The parameter calculation process is to combine the plane projection method and the point-by-point insertion algorithm to perform Delaunay triangulation on the three-dimensional point cloud data of the wall in the point cloud model after matching in step 4. The scattered and disordered point cloud simulates the concave and convex structure of the real wall in the form of a triangular mesh. Each triangular mesh is regarded as a reflection face element, and the face element area is calculated from the three-dimensional coordinates of the vertex of the face element Coordinates of the center of gravity of the surface element and the surface normal vector

[0118] Step 5.2: Characterize the impact of the shadow formed by the obstacle under illumination on the visible light channel. The parameter calculation process is to use the point cloud coordinate data of the obstacle to calculate the shadow point set P of the LED light rays passing through each point of the obstacle falling on the receiving plane. s , combined with the iterative progressive convex hull algorithm, we get the point set P s The minimum convex polygon structure is regarded as the shadow area D generated when the LED light source illuminates the obstacle. r ,y r ,z r ) is contained in the convex polygon D, the light rays are blocked, and the shadow coefficient α is applied as shown in Formula 3.

[0119] For a single reflection channel, the centroid of each reflection surface element of the wall is regarded as a point light source. Repeat the above steps for each reflection point light source to obtain the corresponding convex polygon D w , and further obtain the corresponding shadow coefficient α w .

[0120] Step 5.3: Combining the two processes of step 5.1 and step 5.2, and considering the characterization parameters of the two influencing factors of the wall and the obstacle, the direct channel response expression of the tunnel space can be determined as formula 4.

[0121] In formula 4, m is 1, A p The value is 0.0001, G(θ tr ) is taken as 2.55, and the coordinates of the transmitter are (3m, 2m, 2.5m).

[0122] The expression of the first reflection channel response is formula 5.

[0123] In Formula 5, W is the total number of wall reflection elements, and ρ is 0.03.

[0124] The calculation model for the direct signal strength and the arrival time of the first reflected path is established as follows:

[0125] Step 5.4: First, determine the signal strength mathematical model related to time through the channel response model. The specific method is as follows:

[0126] In practical applications, the visible light channel adopts the intensity modulation / direct reception method, and the transmitted waveform is the DC optical power signal x(t)=P of the LED t , P t is the emitted optical power of the LED, which is set to 20 W in this case. The receiver detects the photocurrent signal, as shown in Equation 6.

[0127] In Formula 6, the value of γ is 0.53.

[0128] After amplification, filtering, sampling and other operations, the photocurrent signal outputs a time domain discrete signal. In this case, the time sampling rate is t = 0.05 (ns), and the time signal X(t) = {0, 0 + 0.05, 0 + 2 × 0.05, ..., 0 + n × 0.05} is obtained. The amplitude variable is the signal intensity P at the sampling time. r (t) = {P r (t0),P r (t1),...,P r (t n )}, the signal strength is simulated as the ratio of the photocurrent signal to the receiver responsivity γ.

[0129] Step 5.5: Determine the fingerprint feature according to the signal strength mathematical model in step 5.4, that is, the calculation model of the direct signal strength and the arrival time of the first reflection path. The specific method is as follows:

[0130] According to the propagation path characteristics of visible light signals, the direct path is unique and has the shortest length. Therefore, the receiver at each location always receives the direct signal first. According to the distribution characteristics of visible light intensity in underground mines, the contribution of direct light signals is much higher than that of reflected signals. Therefore, the minimum threshold of direct signals is set to p. los , then the direct signal strength P LOS For sequence P r The only variable P in (t) that meets the signal threshold LOS =P r (t i )≥p los , otherwise, P LOS =0.

[0131] The strength value of the first reflected path reaching the receiver is the sequence P r The first variable {P r (t i )<p los} i_min , at this time, the first reflection path arrives at time t f=t i_min .

[0132] Step 6: Establish a hybrid fingerprint database based on the mathematical model obtained in step 5. The specific method is as follows:

[0133] Step 6.1, determine the array form of the receiver. The present invention uses 5 receivers, such as Fig. 9 The figure shows a top view of the array of five receivers of the present application. A receiver is placed at the center of the fingerprint point, and a receiver is placed at intervals of 5 (cm) in front, behind, left and right of the receiver, to collect a total of 10-dimensional fingerprint vectors.

[0134] Step 6.2, divide the positioning plane grid in the tunnel space. In this case, the ground, that is, plane z=0, is used as the positioning plane. A rectangular grid is drawn along the x-axis and y-axis at intervals of 0.133 (m) and 0.089 (m), respectively, and a total of 2025 grids are obtained in the 6m×4m×2.5m tunnel space. The center of each grid is used as the fingerprint point coordinate to place the receiving array. Taking the i-th fingerprint point as an example, its real coordinate is (x i ,y i ,z i ), collect the direct signal strength values ​​of the five receivers in the array and the arrival time of the first reflected path, that is, C0{P LOS ,t f |(x i ,y i ,z i )},C1{P LOS ,t f |(x i -0.05,y i ,z i )},C2{P LOS ,t f |(x i +0.05,y i ,z i )},C3{P LOS ,t f |(x i ,y i -0.05,z i )},C4{P LOS ,t f |(x i ,y i +0.05,z i )}. Combined with the direct signal strength P in step 5.5 LOS and the arrival time t of the first reflected path f The calculation model obtains the 10-dimensional fingerprint vector of the i-th fingerprint point.

[0135] Repeat the above steps for each fingerprint point to obtain a mixed fingerprint database:

[0136] It should be noted that in the present application, the positioning plane can be selected at different heights according to actual conditions. This application only illustrates the technical solution of the present application. In actual application, the positioning plane is not limited to the ground.

[0137] Step 7: normalize the hybrid fingerprint database obtained in step 6, and use the processed database to train the BP neural network optimized by the thinking evolution algorithm to build a prediction model.

[0138] The above BP neural network is a single hidden layer structure, the number of inputs is 10, the number of outputs is 3, the number of hidden layer neurons is set to 20, the number of iterations is set to 1000, and the target error is set to 1e-6. As training input, coordinate information As training output, a nonlinear expression between input and output, namely the prediction model, is generated through iterative learning of a large amount of data.

[0139] The above-mentioned thinking evolution algorithm belongs to the evolutionary intelligent optimization algorithm, which simulates the "convergence" and "alienation" of human thinking characteristics, and searches for the optimal solution in the population space through the idea of ​​"variation inheritance". The purpose of applying the optimization algorithm is to improve the defects of random setting of relevant parameters in the above-mentioned BP neural network, the parameters include initial weights and thresholds, find the optimal values ​​of the parameters, and improve the prediction performance.

[0140] After the prediction model is built, you only need to place an array of 5 receivers on the prediction point that needs to be located, collect a 10-dimensional fingerprint vector, and then output the coordinates of the prediction point to achieve positioning.

[0141] In this case, 225 prediction points are selected, the real coordinates of each prediction point are recorded, and the error is calculated with the predicted output coordinates. The results show that the average positioning error of the hybrid fingerprint positioning method is 0.96 (cm), and the root mean square positioning error is 1.83 (cm).

[0142] like Fig.10 The following is a diagram showing the positioning effect of the hybrid fingerprint positioning method: Fig.11 Shown is the positioning error probability distribution diagram of the hybrid fingerprint positioning method.

[0143] In order to prove the low error advantage of the hybrid fingerprint positioning method described in this application, this case also conducted a verification experiment of the single-feature fingerprint positioning method. Based on the above steps 1 to 7, the single-feature fingerprint database uses 2025 fingerprint points with the same coordinates, but the fingerprint value uses the 5-dimensional direct signal intensity to obtain the single-feature fingerprint database The prediction model used in the single-feature fingerprint positioning method is to use the single-feature fingerprint database F s The training thought evolution algorithm optimizes the BP neural network, and the network parameters are the same as step 7. The prediction point coordinates in the verification experiment of the single-feature fingerprint positioning method are consistent with the verification experiment of the hybrid fingerprint positioning method. The results show that the average positioning error of the single-feature fingerprint positioning method is 2.15 (cm) and the root mean square positioning error is 9.14 (cm).

[0144] like Fig.12 The following is a diagram showing the positioning effect of the single-feature fingerprint positioning method: Fig.13 Shown is the positioning error probability distribution diagram of the single-feature fingerprint positioning method.

[0145] Combined with the positioning error data, the error of the single-feature fingerprint positioning method is significantly higher than that of the hybrid fingerprint positioning method. Fig.10 and Fig.12 , after applying the hybrid fingerprint positioning method, each predicted point corresponds to a unique coordinate, while the predicted coordinates of multiple predicted points in the single feature fingerprint positioning method are consistent. Fig.13 As shown in Figure 2, the maximum error of the single-feature fingerprint positioning method exceeds 70 (cm), and the positioning accuracy is low. Fig.11 It shows that the maximum positioning error of hybrid fingerprint positioning does not exceed 16 (cm), and the positioning accuracy is higher. The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A visible light hybrid fingerprint positioning method for underground mine environment, characterized in that: The method comprises: Step 1: Collect initial point cloud data including walls and obstacles in the tunnel space to generate multiple initial point clouds; Step 2: Apply the point cloud denoising algorithm to the initial point cloud data obtained in step 1 to filter out outliers and noise points; apply the point cloud registration algorithm to the multiple point clouds after denoising, and regularly merge the multiple point clouds representing the same object based on the points with the same characteristics, so as to preliminarily obtain the overall point cloud model of the two walls and obstacles in the lane space; Step 3: Input the coordinate data of the overall point cloud model of the missing points into the pre-trained PoinTr deep network to complete the point cloud, so as to construct a more complete overall point cloud model; Step 4: Match the overall point cloud model after point cloud completion with the rectangular coordinate system constructed by the actual scene; Step 5: Based on the matched point cloud data in step 4, a visible light propagation model of the underground tunnel space is constructed, where the visible light propagation model includes a channel response model, and a calculation model of the direct signal intensity and the arrival time of the first reflection path; Step 6: Establish a hybrid fingerprint database based on the mathematical model obtained in step 5; Step 7, normalize the hybrid fingerprint database obtained in step 6, and use the processed database to train the BP neural network optimized by the thinking evolution algorithm to build a prediction model; the BP neural network uses the fingerprint information in the database as training input and the coordinate information as training output, and generates a nonlinear expression between input and output under a large amount of data iterative learning; Step 8: After the prediction model is constructed, the transmitter signal in the tunnel space is received by the receiver for positioning.

2. The visible light hybrid fingerprint positioning method for underground mine environment according to claim 1 is characterized in that: In step 2, the denoising algorithm uses any one of voxel filtering, Gaussian filtering, and statistical filtering.

3. The visible light hybrid fingerprint positioning method for underground mine environment according to claim 1 is characterized in that: The registration algorithm in step 2 adopts an iterative closest point algorithm or a deep network registration algorithm.

4. The visible light hybrid fingerprint positioning method for underground mine environment according to claim 1, characterized in that: In step 3, the training method of the pre-trained PoinTr deep network includes: Step 3.1, collect real point cloud data similar to underground mine environment; Step 3.2, perform step 2 on the point cloud data obtained in step 3.1; Step 3.3: Based on the open3d library programming, the point cloud obtained in step 3.2 is cropped. Each point cloud after cropping is based on the downsampling algorithm, retaining only N points and saved as a .ply file. Step 3.4, based on open3d library programming, extract the point cloud coordinates of the .ply file formed after cropping, form a point cloud coordinate vector group after normalization, and input the point cloud coordinate vector group into the PoinTr network for training; the point cloud coordinate vector group is divided into training data and verification data in a ratio of 4:1; use the verification data to cross-validate the completion models with different parameters to obtain the best completion model; input the overall point cloud model of the missing points obtained in step 2 into the best completion model to complete the point cloud completion.

5. The visible light hybrid fingerprint positioning method for underground mine environment according to claim 1, characterized in that: In step 4, a three-dimensional rigid body transformation algorithm is applied to match the wall and obstacle point clouds in the camera coordinate system with the rectangular coordinate system constructed by the actual scene.

6. The visible light hybrid fingerprint positioning method for underground mine environment according to claim 5, characterized in that: In step 4, the obstacle point cloud model is plane rectified.

Citation Information

Patent Citations

  • Vehicle positioning method based on road fingerprint space incidence matrix

    CN113721254A

  • Underground space situation awareness system

    CN116645609A

  • Mine card pose detection method based on dual attention mechanism feature fusion

    CN117541645A