An underground mine environment visible light hybrid fingerprint positioning method

Abstract

The application discloses a visible light mixed fingerprint positioning method for an underground mine environment and relates to the underground positioning technical field; the method is based on the fact that the signal arrival time only depends on the length of a propagation path, is not affected by wall material and color, and has uniqueness of the first arrival path, takes high-precision direct signal strength positioning as a benchmark, and combines the arrival time of the first reflection path to construct a mixed characteristic fingerprint database, so that the positioning technical defects caused by the performance reduction of the light signal propagation blocked by obstacles in the underground mine are solved.

Description

Technical Field

[0001] This invention relates to the field of underground mine positioning technology, and in particular to a visible light hybrid fingerprint positioning method for underground mine environments. Background Technology

[0002] The exploration and development of mineral resources has made significant contributions to promoting national economic and social development. Mining production technology and management methods have shifted from extensive to intensive, relying on scientific and technological progress to implement development and protection in both directions. While improving resource extraction and utilization rates, safety management should be strengthened, especially in dangerous underground mines. Real-time and reliable communication base stations and systems should be established, and the positioning and tracking of personnel should be strengthened to ensure that workers do not enter dangerous areas and to ensure their safety.

[0003] Existing communication systems in underground mines combine wired and wireless methods. Wired communication primarily uses carrier wave communication, requiring extensive underground cabling for purposes such as lighting. Traditional radio frequency wireless communication generates strong electromagnetic interference, making it unsuitable for the confined environment of underground mines. Wireless systems relying on visible light communication have become the preferred solution. Among existing positioning technologies, GPS is limited by mine depth, resulting in weak signals and unsatisfactory positioning performance. Bluetooth and ZigBee positioning methods have short positioning distances and are greatly affected by complex environments. Visible light communication, being environmentally friendly and free from electromagnetic interference, simultaneously meets the requirements for wireless communication and positioning applications, making it an effective method for locating personnel underground.

[0004] Visible light positioning technology can be divided into three types: proximity method, triangulation method, and fingerprint positioning method. The proximity method relies on the number of transmitters. Each transmitter emits a light signal carrying location information, and the positioning point receives signal strengths from different transmitters. This allows the system to infer which transmitter is closer and thus determine the positioning point's coordinates. The triangulation method requires three or more transmitters, whose coordinates are determined. Then, the arrival time, signal strength, angle of arrival, and time difference of the signals from different transmitters at the positioning point are measured. A target function is established based on the distance relationship between the transmitters and receivers to estimate the positioning point's coordinates. The fingerprint positioning method consists of two stages: offline database establishment and online measurement and matching. The offline stage collects fingerprint information (signal strength value, signal arrival time, angle of arrival, etc.) at various locations in the positioning plane, establishing a database based on the correspondence between position coordinates and fingerprint values. The online stage measures the fingerprint value of unknown positioning points and compares it with the information in the offline database to ultimately estimate the positioning point's coordinates.

[0005] In narrow underground mines, lighting sources are typically arranged collinearly, with a single lamp placed at considerable intervals along the tunnel ceiling. Therefore, proximity methods and triangulation are unsuitable for mine environments, making spatially segmented single-lamp fingerprint positioning the preferred approach. Among common fingerprint selection methods, measuring received signal strength is simple and inexpensive, widely used in various scenarios, forming a mature positioning system. In the propagation of visible light signals at fixed locations, line-of-sight links correspond to a unique path, while non-line-of-sight links involve light beams reflecting off walls, ground, etc., resulting in diverse and directional propagation paths, easily leading to multipath effects. Therefore, most positioning methods only use direct signal strength to eliminate uncertainties caused by multipath reflections. However, the complex structure of underground environments, with uneven walls causing slight variations in the angle and direction of light reflection, and obstacles blocking light propagation causing the disappearance of direct signals in certain areas, severely damages the quality of fingerprint positioning databases based on direct signal strength, resulting in unsatisfactory positioning performance. Currently, some teams are researching visible light positioning methods in harsh environments that combine reflected signal strength with other methods, showing a significant improvement in positioning performance. Less than ideal, the walls in the mine environment are mostly dark in color, which has good light absorption properties, resulting in extremely low intensity of reflected light signals and significantly reduced application performance. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to provide a visible light hybrid fingerprint positioning method for underground mine environments. This method is based on the fact that the signal arrival time depends only on the length of the propagation path and is not affected by the wall material or color. Furthermore, the first arrival path is unique. Using high-precision positioning based on the intensity of the direct signal as a benchmark, a hybrid feature fingerprint database is constructed by combining the arrival time of the first reflected path. This method aims to solve the positioning technology defects caused by the obstruction of light signal propagation by obstacles in underground mines.

[0007] The technical solution adopted in this invention is as follows:

[0008] A visible light hybrid fingerprint localization method for underground mine environments, the method comprising:

[0009] Step 1: Collect initial point cloud data, including walls and obstacles, in the tunnel space, and generate multiple initial point clouds.

[0010] Step 2: Apply a point cloud denoising algorithm to the initial point cloud data obtained in Step 1 to filter out outliers and noise points; apply a point cloud registration algorithm to the denoised multi-point cloud to regularly merge the multi-point cloud representing the same object based on points with the same characteristics, and initially obtain the overall point cloud model of the two walls and obstacles of the alley space.

[0011] Step 3: Input the coordinate data of the overall point cloud model of the missing points into the pre-trained PoinTr deep network for point cloud completion, thereby constructing a more complete overall point cloud model.

[0012] Step 4: Match the completed point cloud model with the Cartesian coordinate system constructed from the actual scene.

[0013] Step 5: Based on the matched point cloud data in Step 4, construct a visible light propagation model for the underground roadway space. The visible light propagation model includes a channel response model, as well as calculation models for the direct signal strength and the arrival time of the first reflected path.

[0014] Step 6: Based on the mathematical model obtained in Step 5, establish a hybrid fingerprint database.

[0015] Step 7: Normalize the hybrid fingerprint database obtained in Step 6, and use the normalized database to train the BP neural network optimized by the thinking evolution algorithm to build a prediction model. The BP neural network uses the fingerprint information in the database as training input and the coordinate information as training output. Iterative learning with a large amount of data generates a nonlinear expression between the input and output.

[0016] Step 8: Once the prediction model is built, the receiver receives transmitter signals in the tunnel space for positioning.

[0017] Furthermore, in step 2, the denoising algorithm employs any one of voxel filtering, Gaussian filtering, or statistical filtering.

[0018] Furthermore, in step 2, the registration algorithm adopts either the iterative nearest point algorithm or the deep network registration algorithm.

[0019] Furthermore, in step 3, the training method for the pre-trained PoinTr deep network includes:

[0020] Step 3.1: Collect real point cloud data similar to an underground mine environment.

[0021] Step 3.2: Perform step 2 on the point cloud data obtained in step 3.1.

[0022] Step 3.3: Based on the Open3D library, perform a cropping operation on the point cloud obtained in Step 3.2. Each cropped point cloud is based on a downsampling algorithm, retaining only N points, and is saved as a .ply file.

[0023] Step 3.4: Based on Open3D library programming, extract the point cloud coordinates from the cropped .ply file, normalize them to form a point cloud coordinate vector group, and input the point cloud coordinate vector group into the PoinTr network for training; the point cloud coordinate vector group is divided into training data and validation data in a 4:1 ratio; after cross-validation of the completion models with different parameters using the validation data, the best completion model is obtained; input the overall point cloud model of the missing points obtained in Step 2 into the best completion model to complete the point cloud completion.

[0024] Furthermore, in step 4, a three-dimensional rigid body transformation algorithm is applied to match the point clouds of the walls and obstacles in the camera coordinate system with the Cartesian coordinate system constructed from the actual scene.

[0025] Furthermore, in step 4, the obstacle point cloud model is subjected to planar correction.

[0026] The beneficial effects of this invention are as follows: This invention provides a visible light hybrid fingerprint positioning method for underground mine environments. This method is based on the fact that the signal arrival time depends only on the length of the propagation path and is not affected by the wall material and color. Moreover, the first arrival path is unique. Based on the high-precision positioning of the direct signal intensity, a hybrid feature fingerprint database is constructed by combining the arrival time of the first reflected path. This solves the positioning technology defects caused by the obstruction of light signal propagation by obstacles in underground mines. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of a specific scenario in this application.

[0028] Figure 2 This is a schematic diagram illustrating the specific position coordinates of the trolley in this application.

[0029] Figure 3 This is a partial screenshot of the .ply file containing the initial point cloud of a wall surface collected by scanning the alleyway space using a ZED2i binocular camera, as described in this application.

[0030] Figure 4 This is a comparison image of the initial point cloud data of a wall surface before and after denoising in this application.

[0031] Figure 5 This is a diagram illustrating the overall point cloud model constructed using the point cloud registration algorithm in this application.

[0032] Figure 6 This is a comparison diagram of the point cloud model before and after point cloud completion in this application. The left side is the point cloud model before point cloud completion, and the right side is the point cloud model after point cloud completion.

[0033] Figure 7 This is a schematic diagram showing the angular difference between the overall point cloud of obstacles captured by a depth camera and the placement of obstacles in the mine.

[0034] Figure 8 This is a comparison diagram of the obstacle position coordinates before and after the transformation in this application.

[0035] Figure 9 This is a top view of the array of 5 receivers in this application.

[0036] Figure 10 This is a diagram illustrating the positioning effect of the hybrid fingerprint positioning method used in this application.

[0037] Figure 11 This is a probability distribution diagram of the positioning error of the hybrid fingerprint positioning method in this application.

[0038] Figure 12 This is a diagram showing the positioning effect of the single-feature fingerprint positioning method.

[0039] Figure 13 This is a probability distribution diagram of the positioning error in the single-feature fingerprint positioning method.

[0040] Figure 14 This is a flowchart of the method described in this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] Below, this application describes a visible light hybrid fingerprint positioning method for underground mine environments provided by the present invention, in conjunction with a specific scenario.

[0043] like Figure 1 The diagram illustrates a specific scenario of this application. In an underground tunnel space with a length of l (m), width of w (m), and height of h (m) (specifically 6m long, 4m wide, and 2.5m high), a three-dimensional rectangular coordinate system is established with a vertex at the bottom of the tunnel space as the origin. The x-axis of the rectangular coordinate system is aligned with the length of the tunnel space, the y-axis with the width, and the z-axis with the height. Establishing this three-dimensional rectangular coordinate system allows for the determination of the position coordinates of each point within the tunnel space, as well as the spatial coordinates of obstacles, laying the foundation for subsequent prediction model development.

[0044] In addition, a transmitter (i.e., an LED light) is installed in the center of the ceiling of the tunnel space, and its position coordinates can be determined according to the established rectangular coordinate system, denoted as (x... t ,y tIn this specific scenario, the transmitter's position coordinates are (3m, 2m, 2.5m). Furthermore, a small cart is placed in this space to simulate an obstacle. The cart has a length of 1.36m (x-axis direction), a width of 0.68m (y-axis direction), and a height of 0.64m (z-axis direction).

[0045] The specific coordinates of the trolley's location are defined as follows: (e.g.) Figure 2 The diagram shown is a schematic diagram of the specific position coordinates of the trolley in this application. In this application, a cuboid is formed around the trolley based on its length, width, and height. The three-dimensional coordinates of the minimum point of the x, y, and z components of the cuboid are (2.5m, 2.2m, 0m). The trolley is placed perpendicular to the ground (xoy plane) and parallel to the xoz plane.

[0046] It should be noted that in this application, the choice of the origin of the coordinate system and the directions of the x-axis, y-axis and z-axis can be set according to the actual situation. This application is only an example to illustrate the technical solution of this application and is not a specific limitation on the method of establishing a rectangular coordinate system.

[0047] The following describes in detail, based on the specific scenario described above, the specific content of the visible light hybrid fingerprint positioning method for underground mine environments provided by the present invention.

[0048] like Figure 14 As shown, the present invention provides a visible light hybrid fingerprint positioning method for underground mine environments, comprising the following specific steps:

[0049] Step 1: Use a depth camera or laser device to scan the tunnel space multiple times and from multiple angles to collect initial point cloud data of the tunnel space, generating multiple initial point cloud images for constructing the overall point cloud model. The initial point cloud data is saved as a .ply file in the form of three-dimensional spatial coordinates of points. In this application, the focus is on collecting initial point cloud data of the tunnel space walls and obstacles, generating multiple initial point cloud images of the tunnel space walls and multiple initial point cloud images of obstacles.

[0050] like Figure 3 The image shown is a partial screenshot of the .ply file containing the initial point cloud data of a wall surface collected using a ZED2i binocular camera to scan a laneway space, as described in this application. Figure 3 As shown, the ply file records the position coordinates of all points in the initial point cloud of the image. These position coordinates are formed based on the camera coordinate system, which is different from the Cartesian coordinate system set by humans in the specific scene. The two need to be unified in the subsequent steps of this application.

[0051] Step 2: Construct the overall point cloud model using the initial point cloud data obtained in Step 1. This includes the following steps:

[0052] Step 2.1: Apply a point cloud denoising algorithm to the initial point cloud data obtained in Step 1 to filter out outliers and noise points. The denoising algorithm is not limited to voxel filtering, Gaussian filtering, statistical filtering, etc.

[0053] like Figure 4 The image shown is a comparison of the initial point cloud data of a wall surface before and after denoising, with the left side showing the point cloud data before denoising and the right side showing the point cloud data after denoising.

[0054] Step 2.2: Apply a point cloud registration algorithm to the denoised multi-point cloud. This merges multiple point clouds representing the same object based on points with similar characteristics, thus initially obtaining a holistic point cloud model of the two walls and obstacles in the alleyway space. Registration algorithms can include iterative nearest-point algorithms, deep network registration algorithms, etc.

[0055] like Figure 5 The image shown is a diagram of the overall point cloud model constructed using the point cloud registration algorithm in this application.

[0056] Furthermore, due to the dim underground environment, point cloud acquisition is difficult and the amount of point cloud data is small, resulting in varying degrees of data loss in the constructed overall point cloud model. Therefore, for situations with limited point cloud data, this application provides a point cloud completion scheme. The point cloud completion scheme is described in step 3 below.

[0057] Step 3: Input the coordinate data of the overall point cloud model of the missing points into the pre-trained PoinTr deep network for point cloud completion, thereby constructing a more complete overall point cloud model.

[0058] In this application, the method for pre-training the PoinTr deep network is as follows:

[0059] The publicly available datasets typically used in training PoinTr deep networks often lack the acquisition and mapping of point cloud models depicting underground mine environments such as walls and background structures, resulting in low accuracy of the completed models. Therefore, this application employs a method of training the PoinTr deep network based on real-world point cloud data for completion, including the following steps:

[0060] Step 3.1: Use a depth camera or laser device to collect more real point cloud data of similar underground mine environments, such as underground simulated mine bases, underground parking lots, tunnels, etc., to generate a large amount of point cloud data and save it as a .ply file.

[0061] Step 3.2: Perform step 2 on the point cloud data obtained in step 3.1.

[0062] Step 3.3: Using the Open3D library, perform a cropping operation on the point cloud obtained in Step 3.2. Each cropped point cloud piece is processed using a downsampling algorithm, retaining only N points (e.g., 50,000 points) and saved as a .ply file. The purpose of the cropping operation is to increase the training data by cropping a point cloud into more than two parts.

[0063] Step 3.4 uses the Open3D library to programmatically extract the point cloud coordinates from the cropped .ply file. After normalization, a point cloud coordinate vector set is formed, which is then input into the PoinTr network for training. Specifically, the point cloud coordinate vector set is divided into training data and validation data in a 4:1 ratio. Leveraging the powerful feature capture capabilities of the Transformer deep learning architecture, the training data is used to learn the completion model. The validation data is used to cross-validate completion models with different parameters to obtain the optimal completion model. The overall point cloud model of the missing points obtained in Step 2 of this application is then input into the optimal completion model to complete the point cloud completion.

[0064] like Figure 6 The image shown is a comparison of the point cloud model before and after point cloud completion in this application. The left side is the point cloud model before point cloud completion, and the right side is the point cloud model after point cloud completion.

[0065] In this application, the coordinates of the point cloud model are based on the camera coordinate system, while the Cartesian coordinate system in the specific scene is defined by humans. Therefore, this application needs to match the overall point cloud model after point cloud completion with the Cartesian coordinate system constructed in the actual scene, as described in step 4 below.

[0066] Step 4: Match the completed point cloud model with the Cartesian coordinate system constructed from the actual scene.

[0067] In the camera coordinate system, the positive x-axis points to the right, the positive y-axis points to the top, and the positive z-axis points to the front. The actual scene's Cartesian coordinate system is constructed as follows: Figure 1 As shown, the positive x-axis points directly forward, the positive y-axis points directly to the right, and the positive z-axis points directly upward. A 3D rigid body transformation algorithm is applied to match the point clouds of walls and obstacles in the camera coordinate system with the Cartesian coordinate system constructed from the actual scene, as shown in Formula 1:

[0068]

[0069] In Formula 1 (x c ,y c ,z c (x) represents the point cloud coordinates before matching. w ,y w ,z w ) represents the coordinates of the matched point cloud, [T x ,Ty ,T z ] T R represents the translation matrix. c-w It is a rotation matrix, which is the rotation angle of a Cartesian coordinate system constructed based on the x, y, and z axes of the camera coordinate system and matched to the actual scene. The calculation yields the result shown in Formula 2:

[0070]

[0071] It is important to note that because obstacles are 360-degree three-dimensional structures, when using a depth camera to photograph them, the shooting angle needs to be constantly changed around the obstacle. Therefore, the camera perspectives of the multiple initial obstacle point cloud data generated in step 1 are different, resulting in a slight tilt in the overall obstacle point cloud after registration in step 2. That is, the overall normal vector of the obstacle is not strictly perpendicular to the ground and parallel to the xoz plane.

[0072] like Figure 7 The diagram shows an overall point cloud of obstacles captured by a depth camera, illustrating the angular difference between the placement of obstacles and their positions within the mine. The obstacle point cloud is matched with a Cartesian coordinate system constructed from the actual scene through rigid body transformation in 3D space. Figure 7 The red point cloud after matching corresponds to the point cloud captured by a depth camera with a slight tilt, while the white point cloud corresponds to the planar and point coordinate existence of the mine obstacle. The two point clouds have a significant angular deviation. Therefore, this application requires additional planar correction of the obstacle point cloud model. Specifically, the least squares method is used to fit the planar parameters of the obstacle point cloud, thereby obtaining the overall normal vector of the obstacle. Then, the slightly tilted obstacle point cloud planar correction is performed using the Rodriguez method, during which the obstacle coordinates exhibit a slight deviation. Afterward, the obstacle's position is further adjusted to match the actual obstacle coordinates. Specifically, the obstacle coordinates after planar correction are transformed into the actual obstacle coordinates through translation.

[0073] like Figure 8 The image shown is a comparison of the obstacle position coordinates before and after the transformation. The left side is a screenshot of the coordinates before the match, and the right side is a screenshot of the coordinates after the match.

[0074] Step 5: Based on the matched point cloud data in Step 4, construct a visible light propagation model for the underground roadway space. The visible light propagation model includes a channel response model, as well as calculation models for the direct signal strength and the arrival time of the first reflected path.

[0075] The channel response model is established as follows:

[0076] Step 5.1: Characterize the impact of complex uneven walls in the mine on the visible light channel. The parameter calculation process involves using Delaunay triangulation of the 3D point cloud data of the wall in the point cloud model after matching in Step 4, combining the planar projection method and the point-by-point insertion algorithm. The scattered and disordered point cloud, based on a triangular mesh, simulates the uneven structure of the real wall. Each triangular mesh is considered a reflective element, and the area of ​​the element is calculated from the 3D coordinates of its vertices. Centroid coordinates of surface element and surface element normal vector

[0077] Step 5.2: Characterize the impact of shadows cast by obstacles under illumination on the visible light channel. The parameter calculation process involves using the point cloud coordinate data of the obstacles to calculate the set of shadow points P that fall on the receiving plane when LED light rays passing through each point of the obstacle land on the receiving plane. s The point set P is obtained by combining the iterative asymptotic convex hull algorithm. s The smallest convex polygon structure is considered as the shadow region D produced when the LED light source illuminates the obstacle. When the coordinates are (x r ,y r ,z r The receiver is contained within a convex polygon D, and the light rays are blocked. The shadowing coefficient α is expressed as follows: (Formula 3)

[0078]

[0079] For a single reflection channel, the centroid of each reflecting surface element of the wall is considered as a point light source. By repeating the above steps for each reflecting point light source, the corresponding convex polygon D can be obtained. w Furthermore, the corresponding shading coefficient α is obtained. w .

[0080] Step 5.3: Combining steps 5.1 and 5.2, and considering the characterization parameters of both the wall surface and obstacles, the direct channel response expression for the tunnel space can be determined as follows: Formula 4:

[0081]

[0082] In Formula 4, m is the Lambert radiation mode number, and A p It is the physical receiving area of ​​the photoelectric receiver, d tr φ represents the distance between the transmitter and receiver. tr θ represents the radiation angle of the transmitter. tr G(θ) represents the incident angle of the receiver in the direct channel. tr ) is the product of the gains of the filter and the condenser on the receiver, c represents the speed of light, δ(·) represents the impulse function, T represents the transmitter coordinates, and R represents the receiver coordinates.

[0083] The expression for the single-reflection channel response is given by the following formula 5:

[0084]

[0085] In Formula 5, W represents the total number of reflective elements, ρ represents the wall reflectance coefficient, and d tw and d wr θ represents the distance between the transmitter and the reflector element, and between the reflector element and the receiver, respectively. tw and φ wr Let θ represent the incident angle and the radiation angle of the w-th reflecting surface element of the wall, respectively. wr It is the receiver incident angle of a first-order reflection channel.

[0086] The calculation model for the direct signal strength and the arrival time of the first reflected path is established as follows:

[0087] Step 5.4: First, determine the time-dependent mathematical model of signal strength using the channel response model. The specific method is as follows:

[0088] In practical applications, the visible light channel uses intensity modulation / direct reception, and the transmitted waveform is the DC optical power signal x(t) = P of the LED. t P t This is the emitted light power of the LED. The receiver detects the photocurrent signal, as shown in Formula 6 below:

[0089]

[0090] In Formula 6, γ is the receiver responsivity, and N(t) represents the Gaussian noise generated in the channel.

[0091] After amplification, filtering, and sampling, the photocurrent signal outputs a discrete-time signal, which is a temporally ordered sampling time signal X(t) = {X(t0), X(t1), ..., X(t2)}. n The amplitude variable is the signal strength P at the sampling time. r (t)={P r (t0),P r (t1),...,P r (t n The signal strength is simulated as the ratio of the photocurrent signal to the receiver responsivity γ.

[0092] Step 5.5: Determine the fingerprint features based on the signal strength mathematical model in Step 5.4, namely, the calculation model for the direct signal strength and the arrival time of the first reflected path. The specific method is as follows:

[0093] Based on the propagation path characteristics of visible light signals, the direct path is unique and the shortest; therefore, the receiver at each location always receives the direct signal first. According to the visible light intensity distribution characteristics in underground mines, the contribution of the direct light signal is much higher than that of the reflected signal. Therefore, the minimum threshold for the direct signal is set to p. los Then the direct signal strength P LOS For sequence P r The only variable P in (t) that satisfies the signal threshold LOS =P r (t i )≥p los Otherwise, P LOS =0.

[0094] The intensity value of the first reflected path reaching the receiver is sequence P. r The first variable {P} in (t) that does not meet the signal threshold r (t i )<p los} i_min At this time, the arrival time t of the first reflection path f =t i_min .

[0095] Step 6: Based on the mathematical model obtained in Step 5, establish a hybrid fingerprint database. The specific method is as follows:

[0096] Step 6.1: Determine the receiver array configuration. Using a single receiver can only acquire two fingerprint values: the direct signal strength corresponding to the receiver coordinates and the arrival time of the first reflected path. Therefore, the localization effect based on the two-dimensional fingerprint vector is poor. Thus, this invention uses five receivers, such as... Figure 9 The diagram shows a top view of the array of 5 receivers in this application. One receiver is placed at the center of the fingerprint point, and another receiver is placed at intervals d (cm) in front, behind, left, and right, to collect a total of 10-dimensional fingerprint vectors.

[0097] Step 6.2: Divide the positioning plane grid in the tunnel space. Draw a rectangular grid at intervals of lx (m) and ly (m) along the x-axis and y-axis respectively. Within the tunnel space of l (m) × w (m) × h (m), a total of... A grid is used. The receiving array is placed with the center of each grid as the fingerprint point coordinate. Taking the i-th fingerprint point as an example, its true coordinates are (x...). i ,y i ,z i The data acquisition array includes the direct signal strength values ​​of five receivers and the arrival time of the first reflected path, i.e., C0{P}. LOS ,t f |(x i ,y i ,zi )},C1{P LOS ,t f |(x i -d,y i ,z i )},C2{P LOS ,t f |(x i +d,y i ,z i )},C3{P LOS ,t f |(x i ,y i -d,z i )},C4{P LOS ,t f |(x i ,y i +d,z i Therefore, the information corresponding to the i-th fingerprint point in the database is as shown in Formula 7 below:

[0098] F i =[x i ,y i ,z i ,P LOSi0 ,P LOSi1 ,P LOSi2 ,P LOSi3 ,P LOSi4 ,t fi0 ,t fi1 ,t fi2 ,t fi3 ,t fi4 ] (Formula 7).

[0099] In Formula 7, P LOSij =C j {P LOS}, t fij =C j {t f}, where j = 0, 1, ..., 4. P LOS With t f The calculation model is shown in step 5.5.

[0100] Step 7: Normalize the hybrid fingerprint database obtained in Step 6, and use the normalized database to train the BP neural network optimized by the thinking evolution algorithm to build a prediction model.

[0101] The aforementioned BP neural network possesses excellent nonlinear fitting capabilities, effectively integrating fingerprint information from the database. Coordinate information is used as training input. As the training output, a nonlinear expression between the input and output is generated through iterative learning on a large amount of data, which is the prediction model.

[0102] The aforementioned thought evolution algorithm belongs to the category of evolutionary intelligent optimization algorithms. It simulates the "convergence" and "differentiation" characteristics of human thought, searching for the optimal solution in the population space through the idea of ​​"mutation inheritance." The purpose of applying the optimization algorithm is to improve the shortcomings of the randomly set parameters in the BP neural network. These parameters include initial weights and thresholds, and the algorithm seeks the optimal values ​​of these parameters to improve prediction performance.

[0103] Once the prediction model is built, all that is needed is to carry an array of five receivers on the personnel working underground, such as on a safety helmet, to achieve underground positioning.

[0104] To demonstrate the effectiveness of the method of the present invention, the present invention provides the following experiments for verification.

[0105] The 10-dimensional fingerprint vectors collected from the predicted points are input into the mature neural network prediction model trained in step 7 for coordinate localization. Simultaneously, the error values ​​of the predicted output coordinates and the true coordinates are calculated. The formula for calculating the average localization error is as follows: Formula 8; the formula for calculating the root mean square localization error is as follows: Formula 9.

[0106]

[0107]

[0108] Where N represents the number of prediction points, (x ai ,y ai ,z ai (x) represents the true coordinates, (x) pi ,y pi ,z pi ) represents the predicted coordinates.

[0109] Based on the above method, this application scans a section of a real environment's wall and obstacles, and constructs a structure as described above. Figure 1 The visible light positioning system model shown is used to conduct a simulated positioning experiment and output error results. The parameters and results in the specific implementation case are as follows:

[0110] Step 1: Use a depth camera or laser device to scan the tunnel space multiple times and from multiple angles to collect initial point cloud data of the tunnel space. This will generate initial point clouds of multiple tunnel wall surfaces and multiple initial point clouds of obstacles, which will be used to construct the initial point cloud model. The initial point cloud data is saved as a .ply file in the form of three-dimensional spatial coordinates of points.

[0111] Step 2: Construct the overall point cloud model using the initial point cloud data obtained in Step 1. This includes the following steps:

[0112] Step 2.1: Apply a point cloud denoising algorithm to the initial point cloud data obtained in Step 1 to filter out outliers and noise points. The denoising algorithm is not limited to voxel filtering, Gaussian filtering, statistical filtering, etc.

[0113] Step 2.2: Apply a point cloud registration algorithm to the denoised multi-point cloud. This merges multiple point clouds representing the same object based on points with similar characteristics, thus initially obtaining a holistic point cloud model of the two walls and obstacles in the alleyway space. Registration algorithms can include iterative nearest-point algorithms, deep network registration algorithms, etc.

[0114] Step 3: Input the coordinate data of the overall point cloud model of the missing points into the pre-trained PoinTr deep network for point cloud completion, thereby constructing a more complete overall point cloud model.

[0115] In this application, the method for pre-training the PoinTr deep network is as follows:

[0116] Because publicly available datasets lack the acquisition and mapping of underground mine environment features such as walls and background structures, the PoinTr deep network in this application is trained using point cloud data collected in the field. The training process includes the following steps:

[0117] Step 3.1: Use a depth camera or laser device to collect more real point cloud data of similar underground mine environments, such as underground simulated mine bases, underground parking lots, tunnels, etc., to generate a large amount of point cloud data and save it as a .ply file.

[0118] Step 3.2: Perform step 2 on the point cloud data obtained in step 3.1.

[0119] Step 3.3: Using the Open3D library, perform a cropping operation on the point cloud obtained in Step 3.2. Each cropped point cloud piece is processed using a downsampling algorithm, retaining only 50,000 points, and saved as a .ply file. The purpose of the cropping operation is to increase the training data by cropping a point cloud into more than two parts.

[0120] Step 3.4 uses the Open3D library to programmatically extract the point cloud coordinates from the cropped .ply file. After normalization, a point cloud coordinate vector set is formed, which is then input into the PoinTr network for training. Specifically, the point cloud coordinate vector set is divided into training data and validation data in a 4:1 ratio. Leveraging the powerful feature capture capabilities of the Transformer deep learning architecture, the training data is used to learn the completion model. The validation data is used to cross-validate completion models with different parameters to obtain the optimal completion model. The overall point cloud model of the missing points obtained in Step 2 of this application is then input into the optimal completion model to complete the point cloud completion.

[0121] Step 4: Match the completed point cloud model with the Cartesian coordinate system constructed from the actual scene.

[0122] First, a 3D rigid body transformation algorithm is applied to match the point cloud coordinates of the walls and obstacles in the camera coordinate system with the Cartesian coordinate system constructed from the actual scene. The matching process is shown in Formula 1. In this case, the positive x-axis of the camera coordinate system is positive to the right, the positive y-axis is positive up, and the positive z-axis is positive forward. The Cartesian coordinate system constructed from the actual scene is as follows: Figure 1 As shown, the positive x-axis points directly forward, the positive y-axis points directly to the right, and the positive z-axis points directly upward. Therefore, the x, y, and z axes in the camera coordinate system are rotated by 90 degrees to match the Cartesian coordinate system constructed from the actual scene.

[0123] It is important to note that because obstacles are 360-degree three-dimensional structures, when using a depth camera to photograph them, the shooting angle needs to be constantly changed around the obstacle. Therefore, the camera perspectives of the multiple initial obstacle point cloud data generated in step 1 are different, resulting in a slight tilt in the overall obstacle point cloud after registration in step 2. That is, the overall normal vector of the obstacle is not strictly perpendicular to the ground and parallel to the xoz plane.

[0124] like Figure 7 The diagram shows an angular difference between the overall point cloud of obstacles captured by a depth camera and the placement of obstacles in the mine. Therefore, this application requires additional planar correction of the obstacle point cloud model. Specifically, the planar parameters of the obstacle point cloud are fitted using the least squares method to obtain the overall normal vector of the obstacle. Then, the slightly tilted obstacle point cloud planar model is corrected using the Rodriguez method, resulting in a slight deviation in the obstacle coordinates during the planar correction process. Afterward, the obstacle positions are adjusted to match the actual obstacle coordinates. Specifically, the obstacle coordinates after planar correction are transformed into the actual obstacle coordinates through translation.

[0125] Step 5: Based on the matched point cloud data in Step 4, construct a visible light propagation model for the underground roadway space. The visible light propagation model includes a channel response model, as well as calculation models for the direct signal strength and the arrival time of the first reflected path.

[0126] The channel response model is established as follows:

[0127] Step 5.1: Characterize the impact of complex uneven walls in the mine on the visible light channel. The parameter calculation process involves using Delaunay triangulation of the 3D point cloud data of the wall in the point cloud model after matching in Step 4, combining the planar projection method and the point-by-point insertion algorithm. The scattered and disordered point cloud, based on a triangular mesh, simulates the uneven structure of the real wall. Each triangular mesh is considered a reflective element, and the area of ​​the element is calculated from the 3D coordinates of its vertices. Centroid coordinates of surface element and surface element normal vector

[0128] Step 5.2: Characterize the impact of shadows cast by obstacles under illumination on the visible light channel. The parameter calculation process involves using the point cloud coordinate data of the obstacles to calculate the set of shadow points P that fall on the receiving plane when LED light rays passing through each point of the obstacle land on the receiving plane. s The point set P is obtained by combining the iterative asymptotic convex hull algorithm. s The smallest convex polygon structure is considered as the shadow region D produced when the LED light source illuminates the obstacle. When the coordinates are (x r ,y r ,z r The receiver is contained within a convex polygon D, and the light rays are blocked. The shadow coefficient α is expressed as Equation 3.

[0129] For a single reflection channel, the centroid of each reflecting surface element of the wall is considered as a point light source. By repeating the above steps for each reflecting point light source, the corresponding convex polygon D can be obtained. w Furthermore, the corresponding shading coefficient α is obtained. w .

[0130] Step 5.3: Combining the two processes of Step 5.1 and Step 5.2, and considering the characterization parameters of the two influencing factors, namely the wall and the obstacle, the direct channel response expression of the alley space can be determined as Formula 4.

[0131] In Formula 4, m takes the value 1, A p The value is 0.0001, G(θ) tr The value is 2.55, and the transmitter coordinates are (3m, 2m, 2.5m).

[0132] The expression for the single-reflection channel response is Equation 5.

[0133] In Formula 5, W is the total number of wall reflective elements, and ρ is 0.03.

[0134] The calculation model for the direct signal strength and the arrival time of the first reflected path is established as follows:

[0135] Step 5.4: First, determine the time-dependent mathematical model of signal strength using the channel response model. The specific method is as follows:

[0136] In practical applications, the visible light channel uses intensity modulation / direct reception, and the transmitted waveform is the DC optical power signal x(t) = P of the LED. t P t This is the emitted light power of the LED, which is set to 20W in this case. The receiver detects the photocurrent signal, as shown in Formula 6.

[0137] In Formula 6, γ takes the value of 0.53.

[0138] After amplification, filtering, and sampling, the photocurrent signal is output as a discrete-time signal. In this case, the sampling rate is t = 0.05 ns, resulting in the time signal X(t) = {0, 0 + 0.05, 0 + 2 × 0.05, ..., 0 + n × 0.05}, where the amplitude variable is the signal strength P at the sampling time. r (t)={P r (t0),P r (t1),...,P r (t n The signal strength is simulated as the ratio of the photocurrent signal to the receiver responsivity γ.

[0139] Step 5.5: Determine the fingerprint features based on the signal strength mathematical model in Step 5.4, namely, the calculation model for the direct signal strength and the arrival time of the first reflected path. The specific method is as follows:

[0140] Based on the propagation path characteristics of visible light signals, the direct path is unique and the shortest; therefore, the receiver at each location always receives the direct signal first. According to the visible light intensity distribution characteristics in underground mines, the contribution of the direct light signal is much higher than that of the reflected signal. Therefore, the minimum threshold for the direct signal is set to p. los Then the direct signal strength P LOS For sequence P r The only variable P in (t) that satisfies the signal threshold LOS =P r (t i )≥p los Otherwise, P LOS =0.

[0141] The intensity value of the first reflected path reaching the receiver is sequence P. r The first variable {P} in (t) that does not meet the signal threshold r (t i )<p los} i_min At this time, the arrival time t of the first reflection path f=t i_min .

[0142] Step 6: Based on the mathematical model obtained in Step 5, establish a hybrid fingerprint database. The specific method is as follows:

[0143] Step 6.1: Determine the array configuration of the receivers. This invention uses 5 receivers, such as... Figure 9 The diagram shows a top view of the array of five receivers in this application. One receiver is placed at the center of the fingerprint point, and another receiver is placed 5 cm apart in front of, behind and to the left and right, to collect a total of 10-dimensional fingerprint vectors.

[0144] Step 6.2: Divide the positioning plane into grids within the tunnel space. In this case, the ground, i.e., plane z = 0, is used as the positioning plane. A rectangular grid is drawn at intervals of 0.133 (m) and 0.089 (m) along the x-axis and y-axis respectively, resulting in 2025 grids within the 6m × 4m × 2.5m tunnel space. The receiving array is placed with the center of each grid as the fingerprint point coordinate. Taking the i-th fingerprint point as an example, its true coordinates are (x... i ,y i ,z i The data acquisition array includes the direct signal strength values ​​of five receivers and the arrival time of the first reflected path, i.e., C0{P}. LOS ,t f |(x i ,y i ,z i )},C1{P LOS ,t f |(x i -0.05,y i ,z i )},C2{P LOS ,t f |(x i +0.05,y i ,z i )},C3{P LOS ,t f |(x i ,y i -0.05,z i )},C4{P LOS ,t f |(x i ,y i +0.05,z i Combined with the direct signal strength P in step 5.5 LOS and the arrival time t of the first reflection path f The computational model yields the 10-dimensional fingerprint vector of the i-th fingerprint point.

[0145] Repeat the above steps for each fingerprint point to obtain a hybrid fingerprint database:

[0146]

[0147] It should be noted that in this application, the positioning plane can be selected at different heights depending on the actual situation. This application is only an example to illustrate the technical solution of this application. In actual application, the positioning plane is not limited to the ground.

[0148] Step 7: Normalize the hybrid fingerprint database obtained in Step 6, and use the normalized database to train the BP neural network optimized by the thinking evolution algorithm to build a prediction model.

[0149] The aforementioned BP neural network has a single hidden layer structure, with 10 inputs, 3 outputs, 20 hidden layer neurons, 1000 iterations, and a target error of 1e-6. The fingerprint information from the database is then used. Coordinate information is used as training input. As the training output, a nonlinear expression between the input and output is generated through iterative learning on a large amount of data, which is the prediction model.

[0150] The aforementioned thought evolution algorithm belongs to the category of evolutionary intelligent optimization algorithms. It simulates the "convergence" and "differentiation" characteristics of human thought, searching for the optimal solution in the population space through the idea of ​​"mutation inheritance." The purpose of applying the optimization algorithm is to improve the shortcomings of the randomly set parameters in the aforementioned BP neural network. These parameters include initial weights and thresholds, and the algorithm seeks the optimal values ​​of these parameters to improve prediction performance.

[0151] Once the prediction model is built, simply place an array of 5 receivers on the prediction point that needs to be located, collect a 10-dimensional fingerprint vector, and the coordinates of the prediction point can be output to achieve localization.

[0152] In this case study, 225 prediction points were selected, and the true coordinates of each prediction point were recorded. The error between these true coordinates and the predicted output coordinates was calculated. The results show that the average positioning error of the hybrid fingerprint positioning method is 0.96 cm, and the root mean square positioning error is 1.83 cm.

[0153] like Figure 10 The image shown is a localization result diagram of the hybrid fingerprint localization method, as follows: Figure 11 The figure shown is a probability distribution diagram of the positioning error of the hybrid fingerprint positioning method.

[0154] To demonstrate the low-error advantage of the hybrid fingerprint localization method described in this application, an additional verification experiment using a single-feature fingerprint localization method was conducted. Based on steps 1-7 above, where the single-feature fingerprint database uses 2025 fingerprint points with the same coordinates, but the fingerprint values ​​use 5-dimensional direct signal intensity, a single-feature fingerprint database is obtained. The prediction model used in the single-feature fingerprint localization method is based on the single-feature fingerprint database F. s The backpropagation neural network optimized by the training evolutionary algorithm has the same network parameters as in step 7. In the verification experiment of the single-feature fingerprint localization method, the predicted point coordinates are consistent with those in the verification experiment of the hybrid fingerprint localization method. The results show that the average localization error of the single-feature fingerprint localization method is 2.15 (cm), and the root mean square localization error is 9.14 (cm).

[0155] like Figure 12 The image shown is a localization result diagram of the single-feature fingerprint localization method, as follows: Figure 13 The figure shown is a probability distribution of the positioning error of the single feature fingerprint positioning method.

[0156] Based on the positioning error data, the single-feature fingerprint positioning method has a significantly higher error than the hybrid fingerprint positioning method. (Comparison) Figure 10 and Figure 12 After applying the hybrid fingerprint localization method, each predicted point corresponds to a unique coordinate, whereas in the single-feature fingerprint localization method, the predicted coordinates of multiple predicted points are consistent. Furthermore, according to... Figure 13 As shown, the maximum error of the single-feature fingerprint localization method exceeds 70 cm, indicating low localization accuracy. According to... Figure 11 The results show that the maximum positioning error of hybrid fingerprint positioning is no more than 16 cm, which is more accurate.

[0157] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A visible light hybrid fingerprint positioning method for underground mine environments, characterized in that, The method includes: Step 1: Collect initial point cloud data, including walls and obstacles, in the alleyway space, and generate multiple initial point clouds; Step 2: Apply a point cloud denoising algorithm to the initial point cloud data obtained in Step 1 to filter out outliers and noise points; apply a point cloud registration algorithm to the denoised point cloud to fuse multiple point clouds representing the same object together in a regular manner based on points with the same characteristics, and initially obtain the overall point cloud model of the two walls and obstacles of the alleyway space. Step 3: Input the coordinate data of the overall point cloud model of the missing points into the pre-trained PoinTr deep network for point cloud completion, thereby constructing a more complete overall point cloud model; Step 4: Match the completed point cloud model with the Cartesian coordinate system constructed from the actual scene. Step 5: Based on the matched point cloud data in Step 4, construct a visible light propagation model for the underground roadway space. The visible light propagation model includes a channel response model, as well as calculation models for the direct signal strength and the arrival time of the first reflected path. Step 6: Based on the mathematical model obtained in Step 5, establish a hybrid fingerprint database; Step 7: Normalize the hybrid fingerprint database obtained in Step 6, and use the normalized database to train the BP neural network optimized by the thinking evolution algorithm to build a prediction model. The BP neural network uses the fingerprint information in the database as training input and the coordinate information as training output. Iterative learning with a large amount of data generates a nonlinear expression between the input and output. Step 8: Once the prediction model is built, the receiver receives transmitter signals in the tunnel space for positioning.

2. The visible light hybrid fingerprint positioning method for underground mine environments according to claim 1, characterized in that, In step 2, the denoising algorithm uses any one of voxel filtering, Gaussian filtering, or statistical filtering.

3. The visible light hybrid fingerprint positioning method for underground mine environments according to claim 1, characterized in that, In step 2, the registration algorithm uses either the iterative nearest point algorithm or the deep network registration algorithm.

4. The visible light hybrid fingerprint positioning method for underground mine environments according to claim 1, characterized in that, In step 3, the training methods for the pre-trained PoinTr deep network include: Step 3.1: Collect real point cloud data similar to an underground mine environment; Step 3.2: Perform step 2 on the point cloud data obtained in step 3.1; Step 3.3: Based on the Open3D library, perform a cropping operation on the point cloud obtained in Step 3.

2. Each cropped point cloud is based on a downsampling algorithm, retaining only N points, and is saved as a .ply file. Step 3.4: Based on Open3D library programming, extract the point cloud coordinates from the cropped .ply file, normalize them to form a point cloud coordinate vector group, and input the point cloud coordinate vector group into the PoinTr network for training; the point cloud coordinate vector group is divided into training data and validation data in a 4:1 ratio; after cross-validation of the completion models with different parameters using the validation data, the best completion model is obtained; input the overall point cloud model of the missing points obtained in Step 2 into the best completion model to complete the point cloud completion.

5. The visible light hybrid fingerprint positioning method for underground mine environments according to claim 1, characterized in that, In step 4, a three-dimensional rigid body transformation algorithm is applied to match the point clouds of walls and obstacles in the camera coordinate system with the Cartesian coordinate system constructed from the actual scene.

6. The visible light hybrid fingerprint positioning method for underground mine environments according to claim 5, characterized in that, In step 4, the obstacle point cloud model is corrected in planar shape.

Images