A helmet display method and device for rock mass structural plane recognition

Abstract

The present invention discloses a helmet display method and device for rock mass structural plane recognition, which relates to the field of image recognition and analysis. The method includes: acquiring a target rock mass image, positioning and pose data, and an external environmental light brightness signal; establishing a refined point cloud data fusion model of the target rock mass according to the target rock mass image and the positioning and pose data; identifying and marking the distribution information of the structural planes of the target rock mass according to the refined point cloud data fusion model; generating brightness adjustment information according to the external environmental light brightness signal; and adjusting the display brightness of the distribution information of the structural planes of the target rock mass according to the brightness adjustment information. The present invention can realize the rapid, high-precision, and real-time recognition and analysis of rock mass structural planes, and synchronously display the analysis results of rock mass structural planes on the actual rock mass.

Description

Technical Field

[0001] The present invention relates to the field of image recognition and analysis, and particularly to a helmet display method and device for rock mass structural plane recognition. Background Art

[0002] The recognition of rock mass structural planes is of great significance for analyzing rock mass structural characteristics, rock mass deformation characteristics, etc. During field exploration, if the rock mass structural planes can be quickly recognized and the recognition and analysis results can be presented in real time on the actual rock mass, it can provide great convenience for the staff to conduct field exploration and timely judge the situation of the rock mass.

[0003] Most of the existing rock mass structural plane recognition methods directly obtain the information of rock mass structural planes by using contact measurement, or use non-contact measurement such as using unmanned aerial vehicles, three-dimensional laser scanning, etc. to obtain rock mass images, and import these data into a computer for processing, and finally obtain the required rock mass structural information. The patent with the publication number CN114705682A discloses a rock mass structure intelligent vision detection and recognition imaging device. This device installs an intelligent vision detector on a five-axis robotic arm, and conducts 360° all-round detection of the rock mass structure through the intelligent vision detector. The detection results are transmitted to the recognition imaging system for recognition imaging, and multi-dimensional and high-precision recognition imaging of the rock mass structure can be realized. However, this device cannot synchronously display the analyzed structural plane information in front of the eyes in real time through virtual reality. That is to say, this device cannot enable the human eye to see both the real environment picture and the situation of the analyzed structural plane information synchronously covered on the rock mass. Summary of the Invention

[0004] The purpose of the present invention is to provide a helmet display method and device for rock mass structural plane recognition, so as to realize the rapid, high-precision and real-time recognition and analysis of rock mass structural planes, and synchronously display the analysis results of rock mass structural planes on the actual rock mass.

[0005] To achieve the above purpose, the present invention provides the following solutions:

[0006] A helmet display method for rock mass structural plane recognition, comprising:

[0007] Obtaining a target rock mass image, positioning and attitude data, and an external environmental light brightness signal; the target rock mass image includes a binocular image and a point cloud image; the positioning and attitude data includes spatial orientation data and positioning data;

[0008] Establishing a refined point cloud data fusion model of the target rock mass according to the target rock mass image and the positioning and attitude data;

[0009] Identify and mark the distribution information of the structural planes of the target rock mass according to the refined point cloud data fusion model; the distribution information of the structural planes of the target rock mass includes all the rock mass structural plane groups of the target rock mass and the list of dominant attitudes of each group;

[0010] Generate brightness adjustment information according to the external environmental light brightness signal;

[0011] Adjust the display brightness of the distribution information of the structural planes of the target rock mass according to the brightness adjustment information.

[0012] Optionally, the establishment of the refined point cloud data fusion model of the target rock mass according to the target rock mass image and the positioning and attitude data specifically includes:

[0013] Establish a real-time three-dimensional point cloud model of the target rock mass based on the binocular image and the point cloud image using SLAM technology;

[0014] Perform data fusion on the point cloud image and the real-time three-dimensional point cloud model based on the Iterative Closest Point (ICP) algorithm to establish a refined point cloud data fusion model of the target rock mass.

[0015] Optionally, the identification and marking of the distribution information of the structural planes of the target rock mass according to the refined point cloud data fusion model specifically includes:

[0016] Calculate the surface normal vector of the point cloud based on the nearest point neighborhood search algorithm and the iterative weighted average fitting algorithm;

[0017] Calculate the attitude of the structural planes of the target rock mass according to the surface normal vector of the point cloud; the attitude of the structural planes includes the dip direction and dip angle of the structural planes of the target rock mass;

[0018] Identify all the rock mass structural plane groups of the target rock mass and the list of dominant attitudes of each group according to the attitude of the structural planes of the target rock mass;

[0019] Mark different rock mass structural plane groups of the target rock mass with different colors.

[0020] Optionally, the calculation of the attitude of the structural planes of the target rock mass according to the surface normal vector of the point cloud specifically includes:

[0021] According to the three components l, m, and n of the surface normal vector of the point cloud, according to the formulas and calculate the plunge direction θ and plunge angle δ respectively; where Q is the angle conversion value;

[0022] According to the plunge direction θ, use the formula to calculate the dip direction DD of the structural planes of the target rock mass;

[0023] According to the plunge angle δ, the dip angle DA of the structural plane of the target rock mass is calculated using the formula DA = 90° - δ.

[0024] A helmet display device for rock mass structural plane identification, comprising: a helmet main body and an external back device;

[0025] An information acquisition module and a display module are provided on the helmet main body; the external back device includes a processing module and a power supply; the power supply is respectively connected to the information acquisition module, the display module, and the processing module for power supply; the information acquisition module and the display module are respectively connected to the processing module;

[0026] The information acquisition module is used to acquire target rock mass images, positioning and attitude data, and external environmental light brightness signals and send them to the processing module; the target rock mass images include binocular images and point cloud images; the positioning and attitude data include spatial orientation data and positioning data;

[0027] The processing module includes an image processing module, a rock mass structural plane identification module, and a signal processing module; the image processing module is used to establish a refined point cloud data fusion model of the target rock mass according to the target rock mass images and the positioning and attitude data; the rock mass structural plane identification module is used to identify and mark the structural plane distribution information of the target rock mass according to the refined point cloud data fusion model; the structural plane distribution information of the target rock mass includes all rock mass structural plane groups of the target rock mass and a list of dominant occurrences of each group; the signal processing module is used to generate brightness adjustment information according to the external environmental light brightness signal;

[0028] The display module is used to adjust the display brightness of the structural plane distribution information of the target rock mass according to the brightness adjustment information and display it.

[0029] Optionally, the information acquisition module includes: a binocular camera, a lidar, an inertial navigation system, a GPS system, and an ambient light sensor;

[0030] The binocular camera is located on both sides of the helmet main body and is used to acquire binocular images of the target rock mass; the lidar is located on the top of the helmet main body and is used to acquire point cloud images of the target rock mass; the inertial navigation system and the GPS system are located on the same side of the helmet main body and are respectively connected to the binocular camera and the lidar; the inertial navigation system is used to acquire spatial orientation data, and the GPS system is used to acquire positioning data; the ambient light sensor is located on one side of the helmet main body and is used to acquire external environmental light brightness signals.

[0031] Optionally, the display module includes: a micro display, a display screen, and an optical system;

[0032] The microdisplay and the display screen are hung on the front edge of the helmet body; the microdisplay is respectively connected to the rock mass structural plane recognition module and the signal processing module, and is used to display the distribution information of the structural planes of the target rock mass through the display screen, and adjust the display brightness of the distribution information of the structural planes of the target rock mass according to the brightness adjustment information; the optical system is arranged at the front end of the helmet body and is respectively connected to the microdisplay and the display screen, and is used to magnify and project the distribution information of the structural planes of the target rock mass onto the display screen.

[0033] Optionally, the image processing module specifically includes:

[0034] A real-time three-dimensional point cloud model establishment unit, configured to establish a real-time three-dimensional point cloud model of the target rock mass based on the binocular image and the point cloud image by using the SLAM technology;

[0035] A refined point cloud data fusion model establishment unit, configured to perform data fusion on the point cloud image and the real-time three-dimensional point cloud model based on the iterative closest point ICP algorithm to establish a refined point cloud data fusion model of the target rock mass.

[0036] Optionally, the rock mass structural plane recognition module specifically includes:

[0037] A point cloud surface normal vector calculation unit, configured to calculate the point cloud surface normal vector based on the nearest point neighborhood search algorithm and the iterative weighted mean fitting algorithm;

[0038] A structural plane attitude calculation unit, configured to calculate the attitude of the structural plane of the target rock mass according to the point cloud surface normal vector; the attitude of the structural plane includes the dip direction and dip angle of the structural plane of the target rock mass;

[0039] A rock mass structural plane group recognition unit, configured to recognize all rock mass structural plane groups of the target rock mass and the dominant attitude list of each group according to the attitude of the structural plane of the target rock mass;

[0040] A rock mass structural plane group marking unit, configured to mark different rock mass structural plane groups of the target rock mass with different colors.

[0041] Optionally, the structural plane attitude calculation unit specifically includes:

[0042] A plunge direction and plunge angle calculation sub-unit, configured to calculate the plunge direction θ and plunge angle δ according to the three components l, m, and n of the point cloud surface normal vector according to the formulas and respectively; where Q is to obtain the angle conversion value;

[0043] A structural plane dip direction calculation sub-unit, configured to calculate according to the plunge direction θ by using the formula Calculate the dip direction DD of the structural plane of the target rock mass;

[0044] The structural plane dip angle calculation subunit is configured to calculate the dip angle DA of the structural plane of the target rock mass according to the plunge angle δ using the formula DA = 90° - δ.

[0045] According to the specific embodiments provided by the present invention, the following technical effects are disclosed:

[0046] The present invention provides a helmet display method and device for identifying the structural plane of a rock mass. The method includes: acquiring an image of the target rock mass, positioning and pose data, and the ambient light luminance signal; establishing a refined point cloud data fusion model of the target rock mass according to the image of the target rock mass and the positioning and pose data; identifying and marking the distribution information of the structural plane of the target rock mass according to the refined point cloud data fusion model; generating brightness adjustment information according to the ambient light luminance signal; and adjusting the display brightness of the distribution information of the structural plane of the target rock mass according to the brightness adjustment information. The method of the present invention can achieve rapid, high-precision real-time identification and analysis of the structural plane of the rock mass, and using the helmet as a carrier, synchronously display the analysis results of the structural plane of the rock mass on the actual rock mass, realizing that when the human eye in the field of vision covers the target rock mass, the distribution information of the structural plane of the rock mass can be observed on the target rock mass, greatly improving the field exploration efficiency. Description of the Drawings

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0048] Figure 1 It is a flowchart of the helmet display method for identifying the structural plane of a rock mass according to the present invention;

[0049] Figure 2 It is a structural schematic diagram of the helmet display device for identifying the structural plane of a rock mass according to the present invention;

[0050] Figure 3 It is a working flowchart of the helmet display device for identifying the structural plane of a rock mass according to the present invention.

[0051] Symbol Explanation:

[0052] 1 - helmet main body, 2 - external back device, 3 - binocular camera, 4 - lidar, 5 - inertial navigation system, 6 - GPS system, 7 - ambient light sensor, 8 - micro display, 9 - optical system, 10 - display screen, 11 - processing module, 12 - power supply. Detailed Embodiments

[0053] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0054] The purpose of the present invention is to provide a helmet display method and device for identifying rock mass structural planes, so as to achieve rapid, high-precision, and real-time identification and analysis of rock mass structural planes, and synchronously display the analysis results of rock mass structural planes on the actual rock mass.

[0055] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] Figure 1 is a flowchart of the helmet display method for identifying rock mass structural planes according to the present invention. As Figure 1 shown, a helmet display method for identifying rock mass structural planes includes:

[0057] Step 101: Obtain a target rock mass image, positioning and attitude data, and an external environmental light brightness signal; the target rock mass image includes a binocular image and a point cloud image; the positioning and attitude data includes spatial orientation data and positioning data.

[0058] The target rock mass image, positioning and attitude data, and external environmental light brightness signal are collected based on a helmet display device for identifying rock mass structural planes. As Figure 2 shown, the helmet display device mainly includes a helmet main body 1 and an external back device 2. The helmet main body 1 includes: an information collection module (including a binocular camera 3, a lidar 4, an inertial navigation system 5, a GPS system 6, and an ambient light sensor 7) and a display module (including a microdisplay 8, an optical system (optical path box) 9, and a display screen 10). The external back device includes: a processing module 11 (including a signal processing module, an image processing module, and a rock mass structural plane identification module) and a power supply 12.

[0059] The helmet main body 1 and the external back device 2 are connected by wires; the binocular cameras 3 are located on both sides of the helmet main body 1, and the lidar 4 is located on the top of the helmet main body 1. Both change their orientations as the human head moves; the inertial navigation system 5 and the GPS system 6 are located on one side of the helmet main body 1 and are respectively connected to the binocular cameras 3 and the lidar 4 by wires; the ambient light sensor 7 is located on the other side of the helmet main body 1 to sense the brightness of the external ambient light; the entire information acquisition module is connected to the processing module of the external back device 2 through a line, and the processing module is then connected to the display module by wire; the microdisplay 8 and the display screen 10 are hung on the front edge of the helmet main body 1, and the optical path box 9 is located at the front end of the helmet main body 1; the power supply 12 powers the entire device.

[0060] Step 102: Establish a refined point cloud data fusion model of the target rock mass according to the target rock mass image and the positioning and attitude data.

[0061] During use, the binocular cameras 3 and the lidar 4 move with the helmet main body 1 following the person's line of sight, and conduct omnidirectional shooting and scanning of the target rock mass. The binocular image sequence collected by the binocular cameras 3 and the point cloud image collected by the lidar 4 are fused, and data deviation caused by environmental factors can be corrected and supplemented, and finally an accurate rock mass point cloud data image is obtained.

[0062] The inertial navigation system 5 is respectively connected to the binocular cameras 3 and the lidar 4 to sense the spatial orientation. The GPS system 6 obtains positioning data and provides spatial position coordinates for the binocular cameras 3 and the lidar 4; the obtained point cloud data and positioning and attitude data are transmitted to the processing module, and the binocular images obtained by the binocular cameras 3 and the point cloud images obtained by the lidar are fused in the image processing module to obtain a fine point cloud model of the target rock mass, so that the structural plane information (structural plane distribution, scale, etc.) of the rock mass can be identified and marked through the rock mass structural plane automatic recognition module.

[0063] Therefore, the specific content of step 102 includes: establishing a real-time three-dimensional point cloud model of the target rock mass based on the SLAM technology according to the binocular image and the point cloud image; performing data fusion on the point cloud image and the real-time three-dimensional point cloud model based on the iterative closest point ICP algorithm to establish a refined point cloud data fusion model of the target rock mass.

[0064] Step 103: Identify and mark the structural plane distribution information of the target rock mass according to the refined point cloud data fusion model.

[0065] The structural plane distribution information of the rock mass can be identified and marked through the rock mass structural plane automatic recognition module. The specific content of step 103 to identify and mark the structural plane distribution information of the target rock mass according to the refined point cloud data fusion model includes:

[0066] Step 3.1: Calculate the surface normal vector of the point cloud based on the K-Nearest Neighbor (KNN) search algorithm and the iterative weighted least squares fitting algorithm. The specific formula is as follows:

[0067] The refined point cloud data fusion model includes a three-dimensional point cloud model set P. The coordinate values of each point in the set are x, y, and z, and the number of points in the point cloud is N, that is, P = {p1, p2,..., p i ,..., p N}. For the given three-dimensional point cloud model set P = {p1, p2,..., p i ,..., p N}, N k (p i ) represents the k-neighborhood subset of point p i controlled by the Euclidean distance of the KNN search algorithm. For any point p i , assign weights to the KNN neighborhood point set N k (p i ) with Gaussian weights and residual factors. Obtain the best-fitting plane Pl(d t , n t ) through the iterative weighted least squares fitting algorithm (the number of iterations is t), abbreviated as Pl:

[0068]

[0069] where p j belongs to N k (p i ); t is the number of iterations; d is the distance from the neighborhood point p i to the plane Pl; d t represents the distance from the neighborhood point p i to the plane Pl at the t-th iteration.

[0070] represents the fitting residual factor of the single point p j at the t-th iteration;

[0071] represents the Gaussian weight function of the distance from point p i to point p j ;

[0072] represents the fitting residual of point p j at the t-th iteration;

[0073] The parameters σ d and σ r represent the distance bandwidth and the fitting residual bandwidth, respectively.

[0074] n tIt is the estimated normal vector of the fitting plane Pl in the t-th iteration, and its value is the eigenvector corresponding to the minimum eigenvalue of the covariance matrix when solving in the t-th iteration of Equation (1).

[0075] n is the estimated normal vector of the final fitting plane Pl after the iteration, that is, the surface normal vector of the point cloud.

[0076] Step 3.2: Calculate the occurrence of the structural plane of the target rock mass according to the surface normal vector of the point cloud; the occurrence of the structural plane includes the dip direction and dip angle of the structural plane of the target rock mass, specifically including:

[0077] Based on the surface normal vector n of the point cloud solved above, establish the conversion relationship between the occurrence on the surface of the structural plane of the target rock mass, that is, the dip direction and dip angle of the structural plane. The calculation formulas for the plunge direction θ and plunge angle δ are as follows:

[0078]

[0079]

[0080] Where l, m, and n are the three components of the surface normal vector n of the point cloud.

[0081] Q in the formula is to obtain the angle conversion value, and the value formula is:

[0082] When l ≥ 0 and m ≥ 0, Q = 0°;

[0083] When l ≥ 0 and m < 0, Q = 360°;

[0084] When l < 0, Q = 180°.

[0085] The conversion formulas between the dip direction DD and dip angle DA of the structural plane and the plunge direction θ and plunge angle δ are:

[0086]

[0087] DA = 90° - δ (5)

[0088] Step 3.3: Identify all the rock mass structural plane groups of the target rock mass and the list of dominant occurrences of each group according to the occurrence of the structural plane of the target rock mass, specifically including:

[0089] Plot all the obtained structural plane occurrences on the Schmidt equal-area projection net to form a stereographic projection pole figure. Based on the kernel density estimation KDE algorithm, perform density clustering on all the occurrence poles in the above projection net to obtain several density peaks. The number of density peaks is the number JS of the dominant groups of the structural plane (i.e., the rock mass structural plane group), and the corresponding occurrence is the dominant occurrence JO of the dominant group.

[0090] In the preferential grouping of structural planes, a maximum angle threshold MaxAngle is set for each group to screen the poles of the structural plane attitudes in each group. The specific operation is as follows: The difference between the attitudes of all poles and the preferential attitude JO is less than the maximum angle threshold MaxAngle, and these are the poles belonging to the group; those greater than the maximum angle threshold MaxAngle are non - belonging poles, or called noise points. After solving in turn and cycling, the set of poles belonging to each preferential grouping of structural planes can be obtained, which serves as the list of preferential attitudes of the rock mass structural plane groups. The point cloud set corresponding to the set of poles belonging to each group is the point cloud subset of that group.

[0091] Step 3.4: Mark different rock mass structural plane groups of the target rock mass with different colors, specifically including:

[0092] Assign different user - defined RGB colors to all point cloud subsets of different groups.

[0093] Step 104: Generate brightness adjustment information according to the external ambient light brightness signal.

[0094] Step 105: Adjust the display brightness of the structural plane distribution information of the target rock mass according to the brightness adjustment information.

[0095] Transmit all the above - identified RGB color information of the rock mass structural plane groups to the micro - display 8 of the display module as the image source. At the same time, after the ambient light sensor 7 collects the external ambient light brightness signal and transmits it to the signal processing module, corresponding brightness adjustment information is sent according to the brightness adjustment curve, and the brightness signal is transmitted to the image source of the display module through the brightness control instruction; the image source is processed by the optical system 9 to generate an enlarged image projected onto the display screen 10. What enters the human eye is the point cloud data image of the rock mass at that place, where the point cloud data belonging to different rock mass structural plane groups is displayed in different user - defined colors, and the list of preferential attitudes of each group is displayed in the lower right corner of the screen. That is, when a field worker's eyes sweep over a certain rock mass, all the information such as the preferential grouping, distribution area and scale, and preferential attitudes of the structural planes of the rock mass will be subsequently displayed on the screen in front of the eyes, providing great convenience for field exploration of rock mass structures.

[0096] Figure 2 This is the structural schematic diagram of the helmet display device for rock mass structural plane recognition according to the present invention. Figure 3 This is the working flow chart of the helmet display device for rock mass structural plane recognition according to the present invention. As Figure 2 and Figure 3As shown in the figure, a helmet display device for rock mass discontinuity identification includes: a helmet main body 1 and an external back device 2, which are connected by wires. An information acquisition module and a display module are provided on the helmet main body 1, and the external back device 2 includes a processing module 11 and a power supply 12. Among them, the power supply 12 is respectively connected to the information acquisition module, the display module, and the processing module 11 for power supply, and the information acquisition module and the display module are respectively connected to the processing module 11.

[0097] The information acquisition module is used to collect target rock mass images, positioning and attitude data, and external environmental light brightness signals and send them to the processing module 11. Among them, the target rock mass images include binocular images and point cloud images, and the positioning and attitude data include spatial orientation data and positioning data.

[0098] As a preferred embodiment, the information acquisition module includes a binocular camera 3, a lidar 4, an inertial navigation system 5, a GPS system 6, and an ambient light sensor 7.

[0099] The binocular cameras 3 are located on both sides of the helmet main body 1 and are used to collect binocular images of the target rock mass; the lidar 4 is located on the top of the helmet main body 1 and is used to collect point cloud images of the target rock mass, and both change their orientations as the human head moves.

[0100] When the binocular cameras 3 and the lidar 4 move with the helmet main body 1 following the human line of sight, the target rock mass is photographed and scanned in all directions, and the binocular images collected by the binocular cameras 3 and the point cloud images collected by the lidar 4 are data - fused, which can correct and supplement the data deviation caused by environmental factors, and finally obtain an accurate point cloud data image of the target rock mass.

[0101] The inertial navigation system 5 and the GPS system 6 are located on the same side of the helmet main body 1 and are respectively connected to the binocular cameras 3 and the lidar 4 by wires. The inertial navigation system 5 is used to sense and collect spatial orientation data, and the GPS system 6 is used to collect positioning data to provide spatial position coordinates for the binocular cameras 3 and the lidar 4.

[0102] The ambient light sensor 7 is located on one side of the helmet main body 1 and is used to sense the external environmental light brightness and collect the external environmental light brightness signal.

[0103] As a preferred embodiment, the processing module 11 includes an image processing module, a rock mass discontinuity identification module, and a signal processing module.

[0104] The image processing module is used to establish a refined point cloud data fusion model of the target rock mass according to the target rock mass images and the positioning and attitude data.

[0105] The collected point cloud data and positioning and attitude data are transmitted to the processing module 11. The binocular images collected by the binocular camera 3 and the point cloud images collected by the lidar 4 are subjected to data fusion in the image processing module to obtain a refined point cloud data fusion model of the target rock mass. The image processing module specifically includes:

[0106] A real-time three-dimensional point cloud model establishment unit, configured to establish a real-time three-dimensional point cloud model of the target rock mass based on the binocular image and the point cloud image by using the SLAM technology;

[0107] A refined point cloud data fusion model establishment unit, configured to perform data fusion on the point cloud image and the real-time three-dimensional point cloud model based on the Iterative Closest Point (ICP) algorithm to establish a refined point cloud data fusion model of the target rock mass.

[0108] The rock mass structural plane recognition module is configured to recognize and mark the distribution information of the structural planes of the target rock mass according to the refined point cloud data fusion model. The distribution information of the structural planes of the target rock mass includes all the rock mass structural plane groups of the target rock mass and the list of dominant attitudes of each group. The rock mass structural plane recognition module specifically includes:

[0109] A point cloud surface normal vector calculation unit, configured to calculate the point cloud surface normal vector based on the K-Nearest Neighbor (KNN) search algorithm and the iterative weighted mean fitting algorithm, specifically including:

[0110] For a given set P of refined point cloud data fusion models (each point coordinate value in the set is x, y, and z, and the number of point clouds is N), that is, P = {p1, p2,..., p i ,..., p N}}. N k (p i ) represents the k-neighborhood subset of point p i controlled by the Euclidean distance based on the KNN search algorithm. For any point p i , the KNN neighboring point set N k (p i ) is weighted by assigning Gaussian weights and residual factors. The best fitting plane Pl is obtained through the iterative weighted mean fitting algorithm (the number of iterations is t):

[0111]

[0112] where p j belongs to N k (p i ); t is the number of iterations;

[0113] d is the distance from the neighboring point p i to the plane Pl; d t represents the neighboring point p at the t-th iterationi Distance to the plane Pl;

[0114] Denote the fitting residual factor of the single point p at the t-th iteration j ;

[0115] Denote the Gaussian weight function of the distance from the point p i to the point p j ;

[0116] Denote the fitting residual of the point p at the t-th iteration j ;

[0117] The parameter σ d and σ r represent the distance bandwidth and the fitting residual bandwidth respectively.

[0118] n t is the estimated normal vector of the fitting plane Pl at the t-th iteration, and its value is the eigenvector corresponding to the minimum eigenvalue of the covariance matrix when solving the t-th iteration of Equation (1).

[0119] n is the estimated normal vector of the final fitting plane Pl after the iteration, that is, the surface normal vector of the point cloud.

[0120] The structural plane attitude calculation unit is used to calculate the structural plane attitude of the target rock mass according to the surface normal vector of the point cloud; the structural plane attitude includes the structural plane dip direction and the structural plane dip angle of the target rock mass, specifically including:

[0121] Based on the surface normal vector n of the point cloud solved above, establish the conversion relationship between the attitude of the surface of the structural plane of the target rock mass, that is, the structural plane dip direction and the structural plane dip angle. The calculation formulas of the plunge direction θ and the plunge angle δ:

[0122]

[0123]

[0124] where l, m, and n are the three components of the surface normal vector n of the point cloud;

[0125] Q in the formula is to obtain the angle conversion value, and the value formula is:

[0126] When l ≥ 0 and m ≥ 0, Q = 0°;

[0127] When l ≥ 0 and m < 0, Q = 360°;

[0128] When l < 0, Q = 180°.

[0129] The conversion formulas between the dip direction DD and dip angle DA of the structural plane and the plunge direction θ and plunge angle δ are as follows:

[0130]

[0131] DA = 90° - δ (5)

[0132] The rock mass structural plane group identification unit is used to identify all the rock mass structural plane groups of the target rock mass and the list of dominant occurrences of each group according to the occurrence of the structural plane of the target rock mass, specifically including:

[0133] Plot all the obtained occurrences on the Schmidt equal-area projection net to form a stereographic projection pole figure. Based on the kernel density estimation (KDE) algorithm, perform density clustering on all the occurrence poles in the above projection net to obtain several density peaks. The number of density peaks is the number of groups JS of the dominant structural plane grouping, and the corresponding occurrence is the dominant occurrence JO of this dominant group.

[0134] The rock mass structural plane group marking unit is used to mark different rock mass structural plane groups of the target rock mass with different colors, specifically including:

[0135] Set a maximum angle threshold MaxAngle for each group in the dominant structural plane grouping to screen the occurrence poles of the structural plane of each group. The specific operation is as follows: The angle difference between the occurrence of all poles and the dominant occurrence JO, those less than the maximum angle threshold MaxAngle are the poles belonging to this group, and those greater than the maximum angle threshold MaxAngle are non-belonging poles, or called noise points. After solving in turn and cycling, the set of poles belonging to each dominant structural plane group can be obtained. The point cloud set corresponding to the set of poles belonging to each group is the point cloud subset of this group. Assign different user-defined RGB colors to the point cloud subsets of all different groups.

[0136] The signal processing module is used to generate brightness adjustment information according to the external ambient light brightness signal.

[0137] As a preferred embodiment, the display module includes: a microdisplay 8, a display screen 10, and an optical system 9.

[0138] The microdisplay 8 and the display screen 10 are hung on the front edge of the helmet main body 1, and the microdisplay 8 is respectively connected to the rock mass structural plane identification module and the signal processing module, and is used to display the structural plane distribution information of the target rock mass through the display screen 10, and adjust the display brightness of the structural plane distribution information of the target rock mass according to the brightness adjustment information.

[0139] The optical system 9 is arranged at the front end of the helmet main body 1, and is respectively connected to the microdisplay 8 and the display screen 10, and is used to magnify and project the structural plane distribution information of the target rock mass onto the display screen 10.

[0140] Transmit all the RGB color information of the identified rock mass structural plane groups above to the microdisplay 8 of the display module as the image source. At the same time, after the ambient light sensor 7 collects the external ambient light brightness signal and transmits it to the information processing module, corresponding brightness adjustment information is sent according to the brightness adjustment curve, and the brightness signal is transmitted to the image source of the display module through the brightness control instruction; the image source is processed by the optical system 9 to generate an enlarged image projected onto the display screen 10. What enters the human eye is the point cloud data image of the rock mass at that place, and the point cloud data belonging to different rock mass structural plane groups is displayed in different user-defined colors. And the dominant attitude list of each group is displayed in the lower right corner of the screen, that is, when the human eye of the field staff sweeps across a certain rock mass, it can quickly, highly accurately and real-time identify and analyze the information of the rock mass structural plane, and synchronously display the analysis results of the rock mass structural plane on the actual rock mass, so that the information such as the dominant grouping, distribution area and scale, and dominant attitude of all the structural planes of the rock mass are displayed on the screen in front of the staff, providing great convenience for the field exploration of the rock mass structure.

[0141] The above is only the preferred embodiment of the present invention and does not have any restrictive effect on the present invention. Any equivalent changes and modifications made in any form based on the essence and principle of the technical solution of the present invention all belong to the content of the technical solution that has not departed from the present invention and still fall within the protection scope of the present invention.

[0142] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0143] Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A helmet display method for rock mass structural plane recognition, characterized in that, The helmet display method includes: Obtaining a target rock mass image, positioning and attitude data, and an external environmental light brightness signal; the target rock mass image includes a binocular image and a point cloud image; the positioning and attitude data includes spatial orientation data and positioning data; Establishing a refined point cloud data fusion model of the target rock mass according to the target rock mass image and the positioning and attitude data; Identify and mark the distribution information of the structural planes of the target rock mass according to the refined point cloud data fusion model; the distribution information of the structural planes of the target rock mass includes all the rock mass structural plane groups of the target rock mass and the list of dominant attitudes of each group; calculate the surface normal vector of the point cloud based on the nearest point neighborhood search algorithm and the iterative weighted least squares fitting algorithm, and calculate the attitude of the structural plane of the target rock mass according to the surface normal vector of the point cloud; the attitude of the structural plane includes the dip direction and dip angle of the structural plane of the target rock mass; identify all the rock mass structural plane groups of the target rock mass and the list of dominant attitudes of each group according to the attitude of the structural plane of the target rock mass; mark different rock mass structural plane groups of the target rock mass with different colors; use the formula to obtain the best fitting plane Pl(d t , n t ); where t represents the number of iterations, d represents the distance from the neighborhood point p i to the plane Pl, d t represents the distance from the neighborhood point p i to the plane Pl at the t-th iteration, n represents the estimated normal vector of the final fitting plane Pl after the iteration, n t is the estimated normal vector of the fitting plane Pl at the t-th iteration, p j represents any single point in the 3D point cloud model set, k represents the neighborhood subset, represents the fitting residual factor of the single point p j at the t-th iteration, w d (p j ) represents the height of the distance from the point p i to the point p j , represents the fitting residual of the point p j at the t-th iteration; Generating brightness adjustment information according to the external environmental light brightness signal; Adjusting the display brightness of the structural plane distribution information of the target rock mass according to the brightness adjustment information.

2. The helmet display method for rock mass structural plane recognition according to claim 1, characterized in that, The step of establishing a refined point cloud data fusion model of the target rock mass according to the target rock mass image and the positioning and attitude data specifically includes: Based on the binocular image and the point cloud image, establishing a real-time three-dimensional point cloud model of the target rock mass based on the SLAM technology; Performing data fusion on the point cloud image and the real-time three-dimensional point cloud model based on the Iterative Closest Point (ICP) algorithm to establish a refined point cloud data fusion model of the target rock mass.

3. The helmet display method for rock mass structural plane recognition according to claim 1, characterized in that The step of calculating the attitude of the structural plane of the target rock mass according to the normal vector of the point cloud surface specifically includes: According to the three components l, m, and n of the surface normal vector of the point cloud, according to the formula and the plunge direction θ and plunge angle δ are calculated respectively; where Q is to obtain the angle conversion value; According to the plunge direction θ, use the formula to calculate the dip direction DD of the structural plane of the target rock mass; According to the plunge angle δ, calculating the dip angle DA of the structural plane of the target rock mass using the formula DA = 90° - δ.

4. A helmet display device for rock mass structural plane recognition, which is applied to the helmet display method for rock mass structural plane recognition according to any one of claims 1-3, and is characterized in that, The helmet display device includes: a helmet main body and an external back device; An information acquisition module and a display module are arranged on the helmet main body; the external back device includes a processing module and a power supply; the power supply is respectively connected to the information acquisition module, the display module, and the processing module for power supply; the information acquisition module and the display module are respectively connected to the processing module; The information acquisition module is used to acquire a target rock mass image, positioning and attitude data, and an external environmental light brightness signal and send them to the processing module; the target rock mass image includes a binocular image and a point cloud image; the positioning and attitude data includes spatial orientation data and positioning data; The processing module includes an image processing module, a rock mass structural plane recognition module, and a signal processing module; the image processing module is used to establish a refined point cloud data fusion model of the target rock mass according to the target rock mass image and the positioning and attitude data; the rock mass structural plane recognition module is used to identify and mark the structural plane distribution information of the target rock mass according to the refined point cloud data fusion model; the structural plane distribution information of the target rock mass includes all rock mass structural plane groups of the target rock mass and a list of the dominant attitudes of each group; the signal processing module is used to generate brightness adjustment information according to the external environmental light brightness signal; The display module is used to adjust and display the display brightness of the structural plane distribution information of the target rock mass according to the brightness adjustment information.

5. The helmet display device for rock mass structural plane recognition according to claim 4, characterized in that, The information acquisition module includes: a binocular camera, a lidar, an inertial navigation system, a GPS system, and an ambient light sensor; The binocular camera is located on both sides of the helmet body and is used to collect binocular images of the target rock mass; the lidar is located on the top of the helmet body and is used to collect point cloud images of the target rock mass; the inertial navigation system and the GPS system are located on the same side of the helmet body and are respectively connected to the binocular camera and the lidar; the inertial navigation system is used to collect spatial orientation data, and the GPS system is used to collect positioning data; the ambient light sensor is located on one side of the helmet body and is used to collect the external ambient light brightness signal.

6. The helmet display device for rock mass structural plane identification according to claim 4, characterized in that, The display module includes: a micro display, a display screen and an optical system; The micro display and the display screen are hung on the front edge of the helmet body; the micro display is respectively connected to the rock mass structural plane recognition module and the signal processing module, and is used to display the distribution information of the structural planes of the target rock mass through the display screen, and adjust the display brightness of the distribution information of the structural planes of the target rock mass according to the brightness adjustment information; the optical system is arranged at the front end of the helmet body and is respectively connected to the micro display and the display screen, and is used to magnify and project the distribution information of the structural planes of the target rock mass onto the display screen.

7. The helmet display device for rock mass structural plane recognition according to claim 4, characterized in that, The image processing module specifically includes: A real-time three-dimensional point cloud model building unit, which is used to build a real-time three-dimensional point cloud model of the target rock mass based on the SLAM technology according to the binocular image and the point cloud image; A refined point cloud data fusion model building unit, which is used to perform data fusion on the point cloud image and the real-time three-dimensional point cloud model based on the Iterative Closest Point (ICP) algorithm to build a refined point cloud data fusion model of the target rock mass.

8. The helmet display device for rock mass structural plane recognition according to claim 4, characterized in that, The rock mass structural plane recognition module specifically includes: A point cloud surface normal vector calculation unit, which is used to calculate the point cloud surface normal vector based on the nearest point neighborhood search algorithm and the iterative weighted mean fitting algorithm; A structural plane attitude calculation unit, which is used to calculate the attitude of the structural planes of the target rock mass according to the point cloud surface normal vector; the structural plane attitude includes the dip direction and dip angle of the structural planes of the target rock mass; A rock mass structural plane group recognition unit, which is used to recognize all the rock mass structural plane groups of the target rock mass and the list of dominant attitudes of each group according to the attitude of the structural planes of the target rock mass; A rock mass structural plane group marking unit, which is used to mark different rock mass structural plane groups of the target rock mass with different colors.

9. The helmet display device for rock mass structural plane identification according to claim 8, characterized in that, The structural plane attitude calculation unit specifically includes: The plunge direction and plunge angle calculation sub-unit is used to calculate the plunge direction θ and plunge angle δ respectively according to the three components l, m, and n of the surface normal vector of the point cloud, according to the formula and where Q is to obtain the angle conversion value; The structural plane dip calculation subunit is used to calculate the structural plane dip DD of the target rock mass according to the plunge θ using the formula ; A structural plane dip angle calculation sub-unit, which is used to calculate the structural plane dip angle DA of the target rock mass according to the plunge angle δ using the formula DA = 90° - δ.

Images