Multi-mode well mining road surface identification method and vehicle hydro-pneumatic suspension active control method

Abstract

The invention discloses a multi-mode well mining road surface identification method and a vehicle hydro-pneumatic suspension active control method. The identification method comprises the following steps: acquiring a road surface image of a target well mine in real time to determine illumination intensity, dynamically determining the weight of a camera and a laser radar according to the illumination intensity, calculating the signal-to-noise ratio of the camera according to the road surface image, and determining the optical perception credibility by combining the weight of the camera; determining a laser radar confidence index according to environmental point cloud data collected by the laser radar, and determining environmental point cloud reliability in combination with a laser radar weight; determining a vehicle response weight according to the weights of the camera and the laser radar, determining a matching degree between the vehicle response and the road surface features through the real-time acceleration of vehicle driving, and obtaining a vibration feature matching gain degree between the vehicle vibration response and the road surface features; and according to the optical sensing credibility, the environmental point cloud reliability and the vibration feature matching gain degree, obtaining a multi-dimensional sensing confidence degree, and determining the road surface grade of the target well mine. The method can realize accurate identification of the mine road surface of the well.

Description

Technical Field

[0001] The present disclosure relates to the technical field of underground mining road surface recognition, and in particular to a multi-modal underground mining road surface recognition method and a vehicle oil-gas suspension active control method. Background Art

[0002] Underground mine tunnels are an extremely complex environment, characterized by rugged surfaces, narrow spaces, high dust concentrations, and poor lighting conditions. These harsh conditions cause severe vibrations in auxiliary transport vehicles, seriously threatening transport safety and shortening equipment life. Existing technologies, particularly for road surface recognition, suffer from insufficient accuracy and poor robustness, making them difficult to adapt to the complex and changing environment of underground mines.

[0003] Currently, road surface recognition in underground mines primarily relies on single technologies, such as cameras, lidar, or vehicle response methods. However, these methods have significant limitations under varying lighting conditions. For example, camera image quality degrades in dim lighting, lidar is susceptible to dust interference, and vehicle response methods are slow to respond to sudden changes in road conditions. Furthermore, while deep learning-based visual recognition methods perform well in stable lighting conditions, they are less robust under the complex lighting and dust interference found in underground mines. Dust increases image noise and blur, reducing the accuracy of neural network recognition.

[0004] While existing multi-sensor fusion methods can improve recognition accuracy by combining data from multiple sensors, such as cameras and lidar, they mostly rely on fixed fusion models and lack dynamic adaptability to environmental factors. Under varying light intensities in underground mines, sensor weights cannot be adjusted in real time as sensor performance changes, affecting fusion results. Furthermore, while the method of identifying road surface types by emitting sound waves of a specific frequency using chassis-mounted vibration exciters is not restricted by visual conditions, has a fast detection response time, and offers low equipment cost, it requires the construction of a sophisticated underground acoustic feature library, is susceptible to interference from mechanical noise from the equipment, and has limited detection range. Summary of the Invention

[0005] In view of this, the embodiments of the present disclosure provide a multimodal underground mining road surface recognition method and a vehicle oil-gas suspension active control method, which can solve the problems of low underground mining road surface recognition accuracy and poor recognition reliability in the existing technology.

[0006] In a first aspect, an embodiment of the present disclosure provides a multimodal underground mining road surface recognition method, comprising:

[0007] Real-time acquisition of road surface images of target underground mines and determination of light intensity based on the road surface images;

[0008] Dynamically determining a camera weight based on the light intensity, and determining a camera signal-to-noise ratio based on the road surface image;

[0009] Determining optical perception credibility according to the camera signal-to-noise ratio and the camera weight;

[0010] Dynamically determining a lidar weight based on the light intensity;

[0011] Determine the LiDAR confidence index based on the environmental point cloud data of the target underground mine collected by the LiDAR;

[0012] Determining the reliability of the environment point cloud according to the laser radar confidence index and the laser radar weight;

[0013] Determining a vehicle response weight according to the camera weight and the lidar weight;

[0014] Determine the degree of matching between vehicle response and road surface characteristics based on the vehicle's real-time acceleration while traveling in a target underground mine.

[0015] determining a vibration characteristic matching gain between the vehicle vibration response and the road surface characteristic according to the vehicle response weight and the matching degree between the vehicle response and the road surface characteristic;

[0016] Obtaining a multidimensional perception confidence according to the optical perception credibility, the environmental point cloud reliability, and the vibration feature matching gain;

[0017] The road surface grade of the target underground mine is determined according to the multi-dimensional perception confidence.

[0018] In a second aspect, the embodiments of the present disclosure further provide a method for actively controlling a vehicle oil-gas suspension, including:

[0019] Dynamically optimizing a sliding mode controller in an active oil-pneumatic suspension system corresponding to an underground mining vehicle based on the target underground mining road surface grade obtained using the multi-modal underground mining road surface identification method;

[0020] The stiffness and damping characteristics of the vehicle's oil-gas suspension are regulated in real time based on the optimized sliding mode controller.

[0021] In a third aspect, the embodiments of the present disclosure further provide a computer device that adopts the following technical solution:

[0022] The computer device comprises:

[0023] at least one processor; and,

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the multimodal underground mining road surface recognition methods or vehicle oil-gas suspension active control methods described above.

[0026] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium storing computer instructions for enabling a computer to execute any of the above-mentioned multimodal underground mining road surface identification methods or vehicle oil-gas suspension active control methods.

[0027] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the above methods when executed by a processor.

[0028] The multimodal underground mining pavement recognition method disclosed in the present application comprehensively considers multimodal information such as camera, lidar and vehicle response through adaptive illumination weight distribution, fully utilizes the advantages of different sensors under different illumination conditions, reduces the limitations of a single sensor or information source, and thus improves the accuracy of pavement grade recognition; it can adapt to different illumination conditions and dynamically adjust the weights of each sensor, so that the recognition method can maintain good performance in various illumination environments, thereby enhancing the robustness of the method; it comprehensively considers information from multiple dimensions such as optical perception, environmental point cloud information and vehicle vibration response, and evaluates pavement conditions from different angles, making pavement grade recognition more comprehensive and obtaining accurate pavement grade information. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] Figure 1 A schematic flow chart of a multimodal underground mining road surface recognition method provided in an embodiment of the present disclosure.

[0031] Figure 2 A schematic flow chart of a method for determining light intensity provided in an embodiment of the present disclosure.

[0032] Figure 3 A flowchart of a method for dynamically acquiring camera weights provided in an embodiment of the present disclosure.

[0033] Figure 4 A flowchart of a method for dynamically acquiring lidar weights provided in an embodiment of the present disclosure.

[0034] Figure 5 A schematic flow chart of a vehicle oil-gas suspension active control method provided in an embodiment of the present disclosure.

[0035] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0037] Reference Figure 1 In a first aspect, the present application discloses a multimodal underground mining road surface recognition method, comprising:

[0038] S100 , collecting a road surface image of a target underground mine in real time, and determining light intensity based on the road surface image.

[0039] Specifically, a high-definition camera installed at the front end of an underground mining vehicle is used to capture road images. The light intensity determined based on the road image is the real-time light intensity of the road to be detected. This light intensity is an important factor affecting image quality and subsequent recognition accuracy. By determining the light intensity in real time, it can provide a basis for subsequent dynamic adjustment of camera weights and other operations, enabling the recognition method to adapt to different lighting conditions and improve the robustness of recognition.

[0040] S200, dynamically determining camera weights according to light intensity.

[0041] The image quality captured by the camera under different light intensities is different. Dynamically determining the camera weight according to the actual collected light intensity can ensure the accuracy of the overall recognition results.

[0042] S300, determining a camera signal-to-noise ratio based on the road surface image;

[0043] The optical perception credibility is determined based on the camera signal-to-noise ratio and camera weight.

[0044] Among them, the optical perception credibility is A: A=ω α ·S c ,ω α is the camera weight, S c is the camera signal-to-noise ratio.

[0045] The camera signal-to-noise ratio reflects the quality of the images captured by the camera. Calculating the optical perception credibility in combination with the camera weight can comprehensively consider the impact of lighting conditions and image quality on optical perception, more accurately assess the reliability of the road surface information provided by the camera, and provide a more valuable basis for subsequent road surface grade determination.

[0046] S400, dynamically determining the laser radar weight according to the light intensity.

[0047] Although lidar is less sensitive to light than cameras, strong light can still affect its performance. Dynamically adjusting lidar weights based on light intensity allows for the optimal use of lidar information under varying lighting conditions, improving overall recognition reliability.

[0048] S500, determining a laser radar confidence index based on environmental point cloud data of the target underground mine collected by the laser radar;

[0049] The reliability of the environmental point cloud is determined based on the lidar confidence index and lidar weight.

[0050] Among them, the reliability of the environmental point cloud is B: B = ω β ·ρ l ,ω β is the lidar weight, ρ l is the lidar confidence index.

[0051] The point cloud data collected by LiDAR can provide three-dimensional environmental information, but the reliability of the data is also affected by many factors. By determining the LiDAR confidence index and combining the LiDAR weight to calculate the reliability of the environmental point cloud, the reliability of the environmental information provided by the LiDAR can be accurately evaluated, providing more comprehensive information for road surface grade determination.

[0052] S600: Determine a vehicle response weight based on the camera weight and the lidar weight.

[0053] Among them, the vehicle response weight is ω γ ,ω γ =1-ω α -ω β ,ω α is the camera weight, ω β is the lidar weight.

[0054] Vehicle response information is another important aspect of road surface grade recognition. By determining vehicle response weights based on camera and lidar weights, we can rationally assign weights to vehicle response information based on a comprehensive consideration of optical and lidar perception information, making road surface grade recognition more comprehensive and accurate.

[0055] S700, determining a degree of matching between a vehicle response and road surface characteristics based on real-time acceleration of the vehicle while traveling in the target underground mine;

[0056] A vibration characteristic matching gain between the vehicle vibration response and the road surface characteristic is determined according to the vehicle response weight and the matching degree between the vehicle response and the road surface characteristic.

[0057] Among them, the vibration feature matching gain is C: C = ω γ M,ωγ is the vehicle response weight, and M is the matching degree between the vehicle response and the road surface characteristics.

[0058] As a vehicle travels on different road surfaces, its acceleration changes accordingly. By determining the degree of match between the vehicle response and road surface characteristics and combining this with the vehicle response weight to calculate the vibration signature matching gain, we can fully utilize the vehicle's real-time response information, providing more direct evidence for road surface grade identification and improving recognition accuracy.

[0059] S800: derive multi-dimensional perception confidence based on optical perception credibility, environmental point cloud reliability, and vibration feature matching gain.

[0060] Among them, the multi-dimensional perception confidence is F: F = A + B + C.

[0061] By comprehensively considering information from multiple dimensions, including optical perception, lidar perception, and vehicle response, we can fully utilize the advantages of different sensors and information sources, reduce the errors that may be caused by a single information source, and improve the overall perception confidence of road conditions, thereby more accurately determining the road grade.

[0062] S900: Determine the road surface grade of the target underground mine based on the multi-dimensional perception confidence level.

[0063] Determining the road surface grade through multi-dimensional perception confidence can comprehensively utilize a variety of perception information to make the determination of the road surface grade more objective, accurate and reliable, and provide a more valuable reference for subsequent vehicle control and operation management.

[0064] The multimodal underground mining pavement recognition method disclosed in the present application comprehensively considers multimodal information such as camera, lidar and vehicle response through adaptive illumination weight distribution, fully utilizes the advantages of different sensors under different illumination conditions, reduces the limitations of a single sensor or information source, and thus improves the accuracy of pavement grade recognition; it can adapt to different illumination conditions and dynamically adjust the weights of each sensor, so that the recognition method can maintain good performance in various illumination environments, thereby enhancing the robustness of the method; it comprehensively considers information from multiple dimensions such as optical perception, environmental point cloud information and vehicle vibration response, and evaluates pavement conditions from different angles, making pavement grade recognition more comprehensive and obtaining accurate pavement grade information.

[0065] The multimodal underground mining road surface recognition method disclosed in the present application collects road surface images of the target underground mining site in real time to determine the light intensity, dynamically determines the camera and lidar weights based on this, calculates the camera signal-to-noise ratio based on the road surface image, and determines the optical perception credibility in combination with the camera weights; determines the lidar confidence index based on the environmental point cloud data collected by the lidar, and determines the environmental point cloud reliability in combination with the lidar weights; determines the vehicle response weight based on the camera and lidar weights, determines the matching degree between the vehicle response and the road surface characteristics through the real-time acceleration of the vehicle, and obtains the vibration feature matching gain of the vehicle vibration response and the road surface characteristics; obtains multidimensional perception confidence based on the optical perception credibility, environmental point cloud reliability, and vibration feature matching gain, and uses this to determine the road surface grade of the target underground mining site, which can obtain comprehensive and accurate road surface recognition information. This method integrates information from multiple aspects such as optical perception, environmental point cloud perception, and vehicle vibration response, and evaluates the road surface from different angles. It reflects the true condition of the road surface more comprehensively than a single sensor or a simple multi-sensor fusion method, and improves the comprehensiveness and reliability of road surface recognition.

[0066] Existing single road surface recognition technology has obvious limitations under different lighting conditions. For example, the image quality of the camera decreases in dim light. However, this method collects road surface images of the target underground mine in real time to determine the light intensity, and dynamically determines the weights of the camera and lidar based on this. This means that when the light changes, the role ratio of the camera and lidar can be automatically adjusted, so that the advantages of each sensor can be fully utilized under different light intensities, improving the accuracy and robustness of the overall recognition, and effectively adapting to the complex and changeable lighting environment of underground mines.

[0067] In the existing technology, lidar is easily affected by dust interference, and the deep learning-based visual recognition method has poor robustness under dust interference. This method comprehensively considers the characteristics of cameras and lidars. On the one hand, it reduces the excessive impact of dust on a single sensor through dynamic weight distribution. On the other hand, it calculates the camera signal-to-noise ratio based on the road image, determines the optical perception credibility based on the camera weight, and determines the lidar confidence index and environmental point cloud reliability based on the environmental point cloud data collected by the lidar. This comprehensive evaluation method can more accurately judge the reliability of each sensor data in a dusty environment, thereby improving recognition accuracy.

[0068] Most existing multi-sensor fusion methods use fixed fusion models, which lack dynamic adaptability to environmental factors. This method dynamically adjusts the weights of cameras and lidar based on environmental factors such as light intensity, and then determines the vehicle response weight. This dynamic fusion approach can adapt to changes in the underground mining environment in real time. When sensor performance changes due to environmental factors (such as light and dust), the role of each sensor can be adjusted in a timely manner to ensure the accuracy of the fusion results, avoiding the limitations of fixed fusion models in the face of environmental changes.

[0069] This method not only integrates information from cameras and lidars, but also introduces vehicle response information. The matching degree between vehicle response and road surface characteristics is determined through the real-time acceleration of the vehicle, and the vibration feature matching gain is obtained. The multi-dimensional perception confidence is obtained by combining the optical perception credibility, the environmental point cloud reliability and the vibration feature matching gain. It makes full use of multiple sensors and vehicle response information to identify the road surface from multiple dimensions, greatly improving the comprehensiveness and accuracy of recognition.

[0070] The harsh environment of underground mine tunnels in existing technologies causes severe vibration in auxiliary transport vehicles, posing a serious threat to transport safety. This method more accurately identifies road surface grade. Based on this accurate road surface information, the driver or autonomous driving system can promptly adjust driving strategies, such as reducing speed and adjusting the route, thereby effectively reducing vibration and accident risks caused by poor road conditions and ensuring transport safety. Accurate road surface identification also helps to rationally plan equipment maintenance. By promptly understanding road conditions, appropriate measures can be taken in advance for sections of road that may cause excessive wear and tear on equipment, preventing excessive vibration caused by long-term driving on poor roads, thereby extending equipment life and reducing equipment maintenance costs. By integrating multiple sensors and a dynamic fusion approach, this method is more robust in the complex and changing environments of underground mines. Dynamic adjustment and multi-dimensional information fusion maintain high recognition accuracy, regardless of lighting changes, dust interference, or sudden changes in road conditions, reducing recognition errors caused by environmental factors.

[0071] Reference Figure 2 The method of S100, "real-time acquisition of road surface images of a target underground mine and determination of illumination intensity based on the road surface images," i.e., a method for determining illumination intensity, specifically includes:

[0072] S110, a real-time road surface image is collected by a camera installed on an underground mining vehicle, where the road surface image is an RGB image.

[0073] Specifically, an industrial camera suitable for underground mining environments can be selected and mounted in a suitable location on the vehicle, such as under the front of the vehicle, to ensure a clear view of the road surface. Camera parameter settings, such as exposure time and aperture size, can be initially adjusted based on the typical lighting conditions in the underground mining environment. For example, in darker underground mining environments, the exposure time and aperture size can be appropriately increased. The camera continuously captures RGB images of the road surface at a fixed frame rate (e.g., 15 frames per second). These images contain color information in three channels: red (R), green (G), and blue (B). RGB images contain rich color and texture information, fully reflecting the characteristics of the road surface. Different road conditions, such as dry, wet, or flooded, will appear in different colors and textures in the RGB images, providing more clues for subsequent road surface analysis and identification. Furthermore, the real-time image acquisition by the vehicle-mounted camera provides timely access to the latest road conditions, supporting real-time decision-making.

[0074] S120: Convert the real-time road surface image into a grayscale image.

[0075] Grayscale images contain only a single channel of information, requiring less data than RGB images, making them simpler and more efficient to process. Color information isn't essential when determining light intensity, while grayscale values ​​directly reflect the image's brightness. Therefore, converting RGB images to grayscale reduces unnecessary computation and improves processing speed. Furthermore, grayscale images eliminate color interference and focus on changes in image brightness, making the determination of light intensity more accurate.

[0076] S130: Determine the light intensity according to the grayscale image.

[0077] Wherein, the light intensity is L: I(x i ,y i ) is the coordinate of the grayscale image (x i ,y i ) is the pixel grayscale value corresponding to the pixel point, and N is the total number of pixels selected from the grayscale image.

[0078] Among them, the method for obtaining the total number of pixels includes: performing Gaussian filtering denoising on the original RGB image, that is, using Gaussian filtering to remove noise interference in the image, using the YOLO-MonoDepth fusion model to segment the road area, that is, performing road segmentation through the YOLO-MonoDepth network, outputting a binary mask, and counting the total number of pixels with a median value of 1 in the mask, which is the total number of pixels N selected from the grayscale image.

[0079] The light intensity values ​​obtained by this method can intuitively reflect the current road lighting conditions, providing a clear basis for subsequent operations such as camera weight assignment. Furthermore, this method is simple and easy to implement, requiring no complex equipment or algorithms, and is highly practical. This method is suitable for complex lighting environments, such as underground mining operations. It can accurately determine light intensity based on real-time road surface images, providing a foundation for subsequent operations such as adaptive light weight assignment. This allows the entire road surface recognition system to better adapt to varying lighting conditions, improving recognition accuracy and reliability.

[0080] Reference Figure 3 , the method of “dynamically determining camera weights according to light intensity” in S200, i.e., the method of dynamically obtaining camera weights, includes:

[0081] S210: Determine a preset medium light threshold of a target underground mine.

[0082] Among them, the preset medium light threshold is L mid In this embodiment, the preset medium light threshold is preferably 100, which is a typical value of the "transition lighting zone" of a mine (the junction between the transport tunnel and the working face).

[0083] The preset medium light threshold provides a reference standard for subsequent camera weighting. In complex lighting environments like underground mines, light intensity varies significantly across different areas and at different times. By determining an appropriate medium light threshold, we can quantify light intensity relative to the threshold, facilitating the dynamic adjustment of camera weights based on the relationship between light intensity and the threshold.

[0084] S220: Dynamically determine a first slope factor according to the light intensity.

[0085] Among them, the first slope factor is k1;

[0086] The first slope factor reflects the sensitivity of camera weight adjustments to changes in light intensity. Different light intensity ranges have different impacts on camera performance. By dynamically determining the first slope factor, we can more flexibly adjust camera weights based on changes in light intensity.

[0087] S230 , dynamically determining a camera weight according to a first preset function, a preset medium illumination threshold, illumination intensity, and a first slope factor.

[0088] Among them, the camera weight is ω α : k1 is the first slope factor, which is used to adjust the rate of weight change, and its value range is between 0.05-0.1; L mid is the preset medium light threshold, L is the light intensity. When the light intensity L is much greater than L midWhen the camera weight ω α Approaches 1; when L is much smaller than L mid When, ω α Approaching 0.

[0089] Through this method, under different lighting conditions, weights can be dynamically allocated according to the actual performance of the camera, thereby improving the role of the camera in the entire road surface recognition system and thus improving the accuracy of road surface recognition.

[0090] The method of “determining the camera signal-to-noise ratio based on the road surface image” in S300 , that is, the method of obtaining the camera signal-to-noise ratio, includes:

[0091] S311, performing noise reduction processing on the road surface image, and using a YOLO-MonoDepth network to perform road segmentation on the noise-reduced road surface image to obtain a grayscale image;

[0092] The grayscale mean of the road surface area and the grayscale mean of the non-road surface area in the image are obtained according to the grayscale image.

[0093] Specifically, the denoised RGB image is converted into a weighted grayscale image perceived by the human eye, and the road surface mask is used to extract the road surface pixel set P r , then P r Take the arithmetic mean, that is, the grayscale mean μ of the road surface area r .

[0094] Cut off the top 1 / 4 area of ​​the image, retain the pixels with a mask of 0 in the area, and obtain the set P b , for P b Take the arithmetic mean, that is, the grayscale average μ of the non-road area on the image b .

[0095] Noise reduction processing can reduce noise interference in the image and improve the accuracy of subsequent road segmentation, because noise may cause incorrect boundary identification during road segmentation; using the YOLO-MonoDepth network for road segmentation can fully utilize its advantages in target detection and depth estimation, more accurately divide the road area, and calculate the grayscale mean of the road area and the grayscale mean of the non-road area, which provides basic data for the subsequent calculation of image noise standard deviation and grayscale contrast.

[0096] S312: Determine the image noise standard deviation based on the grayscale average value.

[0097] Among them, the standard deviation of image noise is σ n : μ b is the grayscale average value, I(x i ,y i ) is the coordinate of the grayscale image (x i,y i ) is the pixel grayscale value corresponding to the pixel point, and N is the total number of pixels selected from the grayscale image.

[0098] The image noise standard deviation reflects the intensity of the noise in the image. By calculating this value, the noise level in the image can be quantified, providing a basis for subsequent consideration of the impact of noise on the camera's signal-to-noise ratio.

[0099] S313, obtaining the road surface background grayscale contrast according to the grayscale mean and the grayscale average value.

[0100] Among them, the grayscale contrast of the road surface background is Δμ: Δμ=μ r -μ b , μ r is the grayscale mean, μ b is the average grayscale value.

[0101] The road surface background grayscale contrast in this step is a quantitative indicator of the grayscale difference between the road surface area and the upper non-road surface area, and essentially reflects the uniformity of light reflection of the road surface area relative to the background area.

[0102] Grayscale contrast is an important indicator for measuring the distinction between the road surface and the background. High contrast means that the road surface is easier to identify in the image. For cameras, high-contrast images can improve imaging quality and the accuracy of information extraction.

[0103] S314: Determine the dust interference intensity based on the road surface background grayscale contrast.

[0104] Among them, the dust interference intensity is D d : Δμ is the grayscale contrast of the road surface and background, C d It is the dust interference factor.

[0105] Furthermore, the method for obtaining the dust interference factor includes: 1) scanning the surrounding environment in real time by installing a laser radar on the top of the underground mining vehicle to obtain point cloud data; 2) preprocessing the point cloud data; 3) fitting ground points using the RANSAC algorithm based on the preprocessed point cloud data; 4) determining the dust interference factor based on the fitted ground points and the preprocessed point cloud data. The dust interference factor is C d : N v is the total number of ground points to be fitted, N t is the total number of preprocessed point cloud data.

[0106] Determining the intensity of dust interference can help us consider the impact of dust environments on camera imaging. In the presence of dust, the grayscale contrast will be reduced. By establishing this mapping relationship, the degree of dust interference on image quality can be quantified.

[0107] S315: Determine a dust compensation coefficient based on the image noise standard deviation and the dust interference intensity.

[0108] Among them, the dust compensation coefficient is η: σ n is the image noise standard deviation, D d is the dust interference intensity.

[0109] The dust compensation coefficient takes into account the combined effects of noise and dust. In a dusty environment, noise and dust will interact with each other and affect the camera's imaging quality. By introducing the dust compensation coefficient, this combined effect can be quantified and compensated.

[0110] S316 , determining a noise and dust comprehensive interference metric based on the image noise standard deviation, the dust compensation coefficient, and the dust interference intensity.

[0111] Among them, the noise and dust comprehensive interference measurement is P: P = σ n +η·D d , σ n is the image noise standard deviation, η is the dust compensation coefficient, D d is the dust interference intensity.

[0112] In this step, the logic of noise and dust comprehensive interference measurement is inherent noise (σ n ) and dust-induced noise (η·D d ) is used for subsequent camera signal-to-noise ratio correction and sensor weight allocation.

[0113] The noise and dust comprehensive interference metric comprehensively considers the impact of noise and dust on camera imaging. It can more comprehensively reflect the degree of interference faced by the camera in a complex environment (with noise and dust).

[0114] S317, determining the camera signal-to-noise ratio based on the road surface background grayscale contrast and the noise and dust comprehensive interference measurement.

[0115] Among them, the camera signal-to-noise ratio is S c :

[0116] The camera signal-to-noise ratio calculated in this way comprehensively considers the contrast between the road surface and the background as well as the interference of noise and dust. It can more accurately evaluate the imaging performance of the camera in actual complex environments, providing a more valuable reference indicator for camera selection and use.

[0117] The camera signal-to-noise ratio (SNR) calculation method disclosed in S311-S317 comprehensively considers multiple factors, including image noise, dust interference, and road-background contrast. This method provides a more accurate calculation of the camera SNR, more realistic than traditional methods that consider only a single factor. It can also accurately evaluate camera performance under varying environmental conditions (such as varying noise levels and dust concentrations), providing a reliable basis for evaluating camera applications in complex environments. The calculated SNR can be used to support decisions regarding camera installation location, parameter settings, and maintenance schedules, helping to improve overall system performance and reliability.

[0118] Reference Figure 4 , the method of S400 "dynamically determining the laser radar weight according to the light intensity", that is, the method of dynamically obtaining the laser radar weight, includes:

[0119] S410 , determining a lower threshold value of effective illumination of the target underground mine according to a current tunnel depth corresponding to when collecting a road surface image of the target underground mine.

[0120] Among them, the lower limit threshold of effective illumination is L min :L min =35+15tanh(0.02D t ), D t is the current tunnel depth. Further, L min The value range is between 30-50.

[0121] This step takes into account the impact of underground mine roadway depth on illumination. Lighting conditions vary significantly in roadways of different depths. By determining the effective illumination lower limit threshold based on the current roadway depth, we can more accurately reflect actual illumination conditions, providing more precise baseline data for subsequent steps. This approach is highly targeted, as each underground mine has different geological structures, mining conditions, and other factors, resulting in unique illumination distributions. By collecting and analyzing actual data from the target underground mine, we determine an effective illumination lower limit threshold that better reflects the mine's actual conditions, improving the applicability of the entire solution.

[0122] S420: Dynamically determine a second slope factor according to the light intensity and the lower effective light threshold.

[0123] The second slope factor is k2:

[0124] C d is the dust interference factor, L is the light intensity, L min is the lower threshold of effective illumination.

[0125] Light intensity is constantly changing. By dynamically determining the second slope factor based on the current light intensity and the effective lower limit threshold of light, subsequent calculations can be more adaptable to real-time changes in light, improving the flexibility and accuracy of the solution. Utilizing the slope factor allows for more flexible control of subsequent calculations, making reasonable adjustments based on different lighting conditions and avoiding the limitations of fixed parameters.

[0126] S430: Determine the suppression item of the light on the laser radar according to the effective light lower limit threshold, the light intensity, and the first slope factor.

[0127] Among them, the suppression term of light on lidar is y1:

[0128] In this step, the inhibitory effect of light on the performance of the lidar is taken into account. When the light intensity is strong, it may interfere with the lidar measurement. By determining the inhibitory term of light on the lidar, this interference can be reasonably compensated and adjusted in the subsequent calculation of the lidar weight, thereby improving the accuracy of the lidar measurement; combined with the effective light lower limit threshold and the first slope factor, the degree of light inhibition on the lidar can be accurately quantified according to different lighting conditions and the pre-set slope factor, making the entire scheme more scientific and reasonable.

[0129] S440: Determine an activation item of the illumination for the laser radar based on the effective illumination lower limit threshold, the illumination intensity, and the second slope factor.

[0130] Among them, the activation term of light on the lidar is y2:

[0131] In this step, the activation effect of light on the lidar is taken into account. Under certain lighting conditions, appropriate lighting can improve the performance of the lidar. By determining the activation item of light on the lidar, the positive effect of light can be fully utilized and the weight calculation of the lidar can be optimized. It also combines the effective light lower limit threshold and the second slope factor to dynamically adjust the size of the activation item according to the change of light intensity, so that the lidar can obtain reasonable weight distribution in different lighting environments.

[0132] S450: Determine a laser radar weight according to an activation item of the laser radar by illumination and an activation item of the laser radar by illumination.

[0133] Among them, the lidar weight is ω β :ω β =y1·y2.

[0134] In this embodiment summary, (ie y1) is the suppression term of light on the lidar. When the light intensity L exceeds the intermediate threshold L min, the exponential term increases, the entire factor approaches 0, and the lidar weight is suppressed. Because the lidar is susceptible to dust scattering under strong light, the point cloud quality decreases, so the weight is reduced. (ie y2) is the activation item of the light to the lidar. When the light intensity L is higher than the effective lower limit L min , the exponential term is adjusted, the factor approaches 1, and the lidar weight is activated. While lidar is not affected by low light, low light levels in mines can be accompanied by high dust levels. Lidar performance is stable in moderate light levels, so activation is necessary. The multiplication of two sigmoid functions requires that the lidar weight is significantly increased only when both function values ​​are close to 1 in neither strong nor weak light conditions.

[0135] The method disclosed in S410-S450 fully considers the impact of light intensity on the lidar by dynamically adjusting the lidar weight, can optimize the performance of the lidar under different lighting conditions, reduce light interference, and improve the accuracy and reliability of measurement; this scheme can dynamically adjust the lidar weight according to the real-time changes in light in the underground mine, so that the system can better adapt to the complex and changeable underground mine environment, and improve the stability and adaptability of the entire monitoring system; reasonable lidar weights provide a more accurate basis for subsequent data processing, and can make algorithms such as data fusion and target detection more efficient and accurate, thereby improving the intelligence level of the entire underground mine monitoring system; by optimizing the use of lidar, problems such as misjudgment and data inaccuracy caused by light interference are reduced, and the additional costs caused by data errors are reduced.

[0136] The method of "determining a lidar confidence index based on environmental point cloud data of a target underground mine collected by the lidar" in S500, i.e., a method for obtaining the lidar confidence index, includes:

[0137] S511 uses a lidar installed on the top of an underground mining vehicle to scan the surrounding environment in real time and obtain point cloud data.

[0138] Specifically, a lidar device can be installed in a suitable location on the roof of an underground mining vehicle. This location should ensure a wide scanning field of view and minimize obstruction by vehicle components. For example, a location slightly forward of the center of the vehicle's roof should be selected. Once activated, the lidar performs a 360-degree scan of the surrounding environment at a certain scanning frequency (e.g., 10 times per second). During each scan, a laser beam is emitted and reflected by surrounding objects. The distance between each reflection point and the lidar is calculated based on the time difference between transmission and reception and the speed of light. Combined with the laser beam's emission angle, the coordinates of the reflection point in three-dimensional space are determined, thus generating point cloud data. This point cloud data records the spatial positional information of surfaces such as tunnel walls, equipment, and obstacles within the underground mine. Real-time acquisition of point cloud data enables the system to promptly monitor dynamic changes in the underground mining environment, such as the appearance of new obstacles or changes in tunnel structure. Mounting the device on the roof of the vehicle can expand the scanning range, covering a larger area around the vehicle, providing more comprehensive data support for subsequent environmental analysis and decision-making.

[0139] S512, preprocessing the point cloud data;

[0140] The RANSAC algorithm is used to fit the ground points based on the preprocessed point cloud data.

[0141] Among them, preprocessing can include operations such as removing outliers and filtering point clouds.

[0142] Specifically, since the laser radar may be interfered with by noise during the scanning process, such as electronic noise, stray light, etc., resulting in the presence of some isolated noise points in the point cloud data, a statistical filtering method can be used to calculate the distance statistics between each point and the points in its neighborhood, and the points whose distance exceeds a certain threshold are regarded as noise points and removed. For example, the average distance from each point to the points in its neighborhood is calculated. If the average distance of a point is greater than a certain multiple (such as 2 times) of the average distance of all points, the point is removed. Point cloud data is usually very dense and contains a large number of data points, which increases the computational complexity of subsequent processing. Voxel grid filtering can be used for downsampling, dividing the point cloud space into small voxel grids. For each voxel grid, only one representative point is retained, thereby reducing the amount of data. For example, the point cloud space is divided into voxel grids with a side length of 0.1 meters, and the points in each grid are replaced by their centroid points.

[0143] The RANSAC algorithm is used to fit ground points. It is an iterative algorithm used to fit mathematical models from data containing noise and outliers. In this step, we need to fit the ground plane inside the underground mine. First, three points are randomly selected from the preprocessed point cloud data. These three points can define a plane. Then, the distance from all points to the plane is calculated. Points with a distance less than a certain threshold (such as 0.1 meters) are considered inliers and are considered to belong to the ground. Next, the number of inliers is counted. If the number of inliers exceeds a certain proportion (such as 80%), it is considered that a good ground plane model has been found. Otherwise, the above process is repeated until a suitable model is found or the maximum number of iterations is reached.

[0144] In this step, denoising improves the quality of the point cloud data and reduces the impact of noise on subsequent analysis. Downsampling reduces computational complexity, improves algorithm efficiency, and saves computing resources and time. The RANSAC algorithm is used to fit ground points, accurately separating them from the complex point cloud data and providing an important foundation for determining the lidar confidence index. The ground is a relatively stable feature in underground mines, and fitting ground points allows for better analysis of lidar performance in different environments.

[0145] S513: Determine a lidar confidence index based on the preprocessed point cloud data and the fitted ground points.

[0146] Among them, the lidar confidence index is ρ l : N v is the total number of ground points to be fitted, N t is the total number of preprocessed point cloud data, σ z is the standard deviation of the height of the fitted ground points, Z a is the average height of the fitted ground points.

[0147] Further, Refers to the proportion of effective ground points, reflecting the completeness of the lidar's spatial coverage of the ground; is the relative height fluctuation ratio, which is used to eliminate the influence of absolute height on fluctuation; This refers to the relative suppression coefficient of ground point height fluctuations, reflecting the vertical quality of the LiDAR point cloud. Specifically, the flatter the ground, the lower the vertical noise in the point cloud and the higher the confidence level. The LiDAR confidence index can be used to comprehensively quantify the quality of LiDAR point clouds.

[0148] By comprehensively evaluating the confidence of the lidar through multiple indicators, we can have a more comprehensive and accurate understanding of the working status and performance of the lidar in the underground mining environment. The confidence index can provide an important basis for subsequent decision-making.

[0149] The method disclosed in S511-S513 processes and analyzes the data collected by the laser radar to determine the confidence index of the laser radar. It can promptly discover problems that may occur in the operation of the laser radar and take corresponding measures to adjust and repair it, thereby improving the reliability and stability of the entire system; accurate laser radar confidence indicators can help the system better understand the environmental information in the underground mine, judge the complexity and uncertainty of the environment, and provide more reliable support for vehicle navigation, obstacle avoidance and other decisions; by real-time monitoring of the laser radar confidence, unnecessary accidents and losses caused by laser radar failure or poor performance can be avoided, and the cost of repairing and replacing equipment can be reduced; at the same time, the rational use of point cloud data, through preprocessing and downsampling and other methods, reduces the computing cost and improves the operation efficiency of the system.

[0150] The method of "determining the degree of matching between the vehicle response and the road surface characteristics based on the real-time acceleration of the vehicle traveling in the target underground mine" in S700, i.e., the method of obtaining the degree of matching between the vehicle response and the road surface characteristics, includes:

[0151] S710 , collecting real-time acceleration corresponding to a preset driving period of the vehicle in a target underground mine through an acceleration sensor installed on the vehicle chassis.

[0152] Specifically, a high-precision, high-reliability acceleration sensor is installed at a suitable position on the vehicle chassis (i.e., the chassis of an underground mining vehicle). This position can more accurately reflect the overall acceleration changes of the vehicle. Usually, a key position near the center of gravity of the chassis or a key position that can directly sense the transmission of road excitation is selected to ensure that the sensor is firmly connected to the chassis to avoid measurement errors caused by looseness. The sampling frequency of the acceleration sensor can be set to 200Hz, which means that the sensor will output an instantaneous value of the vehicle acceleration every 5ms. The higher sampling frequency can capture rapid changes in vehicle acceleration to adapt to local mutations that may occur on underground mining roads.

[0153] A sliding window of 100ms is selected, containing 20 acceleration samples. The sliding window performs local processing of acceleration data in the temporal dimension to analyze acceleration characteristics over a short period of time. As the vehicle travels, the window slides forward, each time by one sampling interval (i.e., 5ms), continuously collecting new data samples.

[0154] Within each 100ms sliding window, the 20 instantaneous acceleration values ​​within the window are arithmetic averaged to obtain the segment-level acceleration mean corresponding to the window. This segment-level acceleration mean represents the average acceleration response of the vehicle within that 100ms period. Following the above sliding window and segment-level acceleration calculation method, segment-level acceleration data is continuously collected within a preset period. The preset period can be set according to actual needs. For example, if it is set to 10 seconds, the sliding window will be continuously updated within these 10 seconds to calculate a series of segment-level acceleration means. This data will be used for subsequent analysis of vehicle driving status and road surface characteristics.

[0155] Because the actual road conditions in underground mining operations are complex and may include sudden potholes, bumps, or changes in slope, a higher sampling frequency (200Hz) can capture these instantaneous acceleration changes promptly, while a 100ms sliding window allows for comprehensive analysis of these changes within a short period of time. Calculating the segment-level acceleration mean smooths out some transient noise interference while highlighting the acceleration variation characteristics caused by localized road surface changes. This enables the system to more accurately perceive road surface changes and provides reliable data support for subsequent vehicle control and decision-making. The sliding window and segment-level acceleration mean calculation methods reduce the amount of data while retaining important acceleration characteristic information. Compared to processing a large number of instantaneous acceleration values, processing the segment-level acceleration mean reduces data processing complexity and improves computational efficiency. Furthermore, the segment-level acceleration mean reflects the overall acceleration trend of the vehicle over a short period of time, making it more suitable for analyzing the relationship between vehicle response and road surface characteristics, thereby improving the validity and practicality of the data.

[0156] S720: Perform Fourier transform on the real-time acceleration to extract the real-time main vibration frequency of the vehicle.

[0157] Specifically, a 100ms window (containing 20 acceleration samples) with 200Hz sampling is selected; the window acceleration mean a is calculated, and DFT is performed on the preprocessed sequence to obtain the frequency domain amplitude spectrum, and the frequency f corresponding to the maximum amplitude is selected. v As the main vibration frequency.

[0158] Time-domain acceleration data contains complex vibration information, making it difficult to directly analyze the main characteristics of vehicle vibration. After converting it to the frequency domain through Fourier transform, the distribution of different frequency components can be clearly displayed, facilitating the extraction of the main frequency characteristics of vehicle vibration. Extracting the real-time main frequency of the vehicle's vibration can grasp the main characteristics of the vehicle vibration and filter out some minor interference information, making subsequent analysis more focused and effective, and improving the accuracy and efficiency of the analysis.

[0159] S730: Determine the matching degree between the vehicle response and the road surface characteristics based on the real-time main vibration frequency of the vehicle and the preset characteristic frequency.

[0160] Among them, the matching degree is M: f v is the real-time main vibration frequency of the vehicle, f r is the preset characteristic frequency.

[0161] By comparing the vehicle's real-time dominant vibration frequency with a preset characteristic frequency, the matching relationship between the vehicle's response and road surface characteristics can be quantified and expressed as a specific matching degree, making the results more intuitive and clear. Accurate matching information provides an important reference for vehicle driving control and scheduling. For example, if the matching degree indicates that the vehicle is driving on a bumpy road, the vehicle can automatically adjust the driving speed and suspension system parameters to improve driving comfort and safety.

[0162] In this embodiment, f r It is the vibration main frequency reference value of the typical characteristic road surface of the target mine, which can be obtained through offline scene calibration. Specifically, typical characteristic roads such as "washboard road in transport lane, crushed zone in fully mechanized mining face, and hardened road surface at transfer point" in the mine are selected. In each scene, acceleration sensor data is collected at a sampling rate of 200Hz, covering vehicle speeds of 10 to 40km / h. Fourier transform is performed on the acceleration sequence of each scene, and the mean value of the vibration main frequency is calculated. The mean value is taken as the f of the scene. r .

[0163] For S900 "Determining the pavement grade of the target underground mine based on multi-dimensional perception confidence", it specifically includes: when 0≤F<0.125, the pavement grade of the target underground mine is determined to be A; when 0.125≤F<0.25, the pavement grade of the target underground mine is determined to be B; when 0.25≤F<0.375, the pavement grade of the target underground mine is determined to be C; when 0.375≤F<0.5, the pavement grade of the target underground mine is determined to be D; when 0.5≤F<0.625, the pavement grade of the target underground mine is determined to be E; when 0.625≤F<0.75, the pavement grade of the target underground mine is determined to be F; when 0.75≤F<0.875, the pavement grade of the target underground mine is determined to be G; when 0.875≤F≤1, the pavement grade of the target underground mine is determined to be H.

[0164] For the road roughness classification standards corresponding to AH grades, please refer to ISO 8608:2016 standard.

[0165] Reference Figure 5 The second aspect of the present application discloses a vehicle oil-gas suspension active control method, comprising:

[0166] S10, dynamically optimizing a sliding mode controller in an active oil-pneumatic suspension system corresponding to an underground mining vehicle using an HOA algorithm based on the target underground mining road surface grade obtained using the multimodal underground mining road surface recognition method;

[0167] S20, based on the optimized sliding mode controller, controls the stiffness and damping characteristics of the vehicle's oil-gas suspension in real time.

[0168] Specifically, when the road surface grade is A or B, it indicates that the road surface is good and the vehicle can travel smoothly. The sprung load in the sliding mode controller can be optimized, that is, a higher weight can be given to the sprung acceleration.

[0169] Furthermore, when the road surface grade is A or B, a larger weight (eg, 0.6) may be assigned to the vehicle body acceleration, while the weights of the suspension dynamic deflection and the tire dynamic load may be relatively smaller (eg, 0.2 and 0.2).

[0170] When the road surface grade is C, D, or E, it indicates that the road surface is average, and the sprung load in the sliding mode controller can be optimized. Specifically, the weight of the sprung acceleration can be reduced to 80%-85% of the original value.

[0171] Furthermore, when the road surface grade is C, D, or E, while ensuring a certain level of comfort, it is necessary to take into account the durability of the suspension system. The weight of the sprung acceleration can be appropriately reduced while the weight of the tire dynamic load is increased. The increased weight of the tire dynamic load can be 45%-55% of the original weight of the tire dynamic load, as long as the sum of all weights is 1.

[0172] When the road surface grade is F, G or H, it means that the road surface is poor and priority should be given to ensuring the safety of the suspension system and the ground contact performance of the tires to improve the overall performance.

[0173] Furthermore, when the road surface grade is F, G, or H, suspension system protection and tire-road contact stability become crucial to avoid suspension damage and tire detachment. Therefore, the weighting of suspension dynamic deflection and tire dynamic load can be significantly increased. Specifically, the original suspension dynamic deflection weighting can be increased by 45%-55% to obtain the increased suspension dynamic deflection; the original tire dynamic load weighting can be increased by 70%-80% to obtain the increased tire dynamic load weighting.

[0174] In this embodiment, this dynamic weight allocation method ensures that the controller can prioritize and optimize the most critical indicators under different road conditions, thereby improving the overall performance of the vehicle.

[0175] In the sliding mode controller of this embodiment, the error vector of the active oil-pneumatic suspension system is defined as e by constructing the vehicle body mass displacement integral error, the vehicle body mass displacement error, and the vehicle body mass velocity error: Then the derivative of the error is Among them, x s is the vertical displacement of the sprung mass, x sr is the reference displacement of the vehicle body. The sliding surface s is defined as the weighted sum of errors: s = ce = [c1 c2 c3][e1 e2 e3] T =c1e1+c2e2+c3e3, where c1, c2, and c3 are all sliding surface parameters.

[0176] Furthermore, by combining the HOA algorithm and the sliding mode controller, the controller parameters (c1, c2, and c3) are further optimized. After parameter optimization, the ideal active force can be obtained. The specific process is analyzed as follows: During the active suspension system design phase, the proper weighting of sprung acceleration, suspension dynamic deflection, and tire dynamic load is key to ensuring a balanced ride quality, stability, and safety in various driving environments. In this embodiment, road conditions are divided into four categories: Category A and B roads are smooth and have minimal vibration and impact, requiring suspension adjustment to provide optimal ride quality. Category C, D, and E roads are moderate, with bumps and unevenness, requiring improved suspension grip to ensure driving safety. Category F, G, and H roads are poor, with significant vibration and impact, and potential safety hazards. These roads require comprehensive suspension adjustment to meet the requirements of ride quality, safety, and stability.

[0177] During the optimization process, the HOA algorithm optimizes the sliding surface parameters c1, c2, and c3 of the sliding mode controller. Based on three evaluation indicators: body acceleration, suspension dynamic deflection, and tire dynamic load, a multi-objective optimization function is constructed. The weighted summation method is used to comprehensively optimize the above performance indicators, where the weight coefficient of each indicator is reasonably allocated according to the vehicle dynamic characteristics.

[0178] Among them, the optimized model is: a and a0 are the sprung accelerations of the active and passive suspensions, respectively, f d and f d0 are the suspension dynamic deflections of active and passive suspensions, F t and F t0 are the tire dynamic loads for active and passive suspension, respectively. a 、ω d 、ω t are the weights of sprung acceleration, suspension dynamic deflection, and tire dynamic load, respectively.

[0179] In a third aspect, the present application discloses a multimodal underground mining road surface recognition system for executing the multimodal underground mining road surface recognition method disclosed in the first aspect of the present application, the system comprising:

[0180] The light intensity dynamic acquisition module is used to collect the road surface image of the target mine in real time and determine the light intensity based on the road surface image;

[0181] The camera signal-to-noise ratio acquisition module is used to dynamically determine the camera weight according to the light intensity and determine the camera signal-to-noise ratio based on the road image;

[0182] An optical perception credibility acquisition module is used to determine the optical perception credibility based on the camera signal-to-noise ratio and camera weight;

[0183] The LiDAR weight acquisition module is used to dynamically determine the LiDAR weight according to the light intensity;

[0184] A laser radar confidence index acquisition module is used to determine the laser radar confidence index based on the environmental point cloud data of the target underground mine collected by the laser radar;

[0185] The environmental point cloud reliability acquisition module is used to determine the reliability of the environmental point cloud based on the lidar confidence index and lidar weight;

[0186] The vehicle response weight acquisition module is used to determine the vehicle response weight based on the camera weight and lidar weight;

[0187] A matching degree acquisition module is used to determine the matching degree between the vehicle response and the road surface characteristics based on the real-time acceleration of the vehicle traveling in the target underground mine;

[0188] A vibration feature matching gain acquisition module is used to determine the vibration feature matching gain between the vehicle vibration response and the road surface characteristics based on the vehicle response weight and the matching degree between the vehicle response and the road surface characteristics;

[0189] A multi-dimensional perception confidence acquisition module is used to obtain multi-dimensional perception confidence based on optical perception credibility, environmental point cloud reliability, and vibration feature matching gain;

[0190] The road surface recognition module is used to determine the road surface grade of the target underground mine based on the multi-dimensional perception confidence.

[0191] The computer device according to an embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache). The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc.

[0192] The processor can be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and can control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is used to execute the computer-readable instructions stored in the memory, causing the computer device to execute all or part of the steps of the multimodal underground mining road surface recognition method or the vehicle oil-pneumatic suspension active control method described in the various embodiments of the present disclosure.

[0193] Those skilled in the art should understand that in order to solve the technical problem of how to obtain a good user experience, this embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the scope of protection of this disclosure.

[0194] like Figure 6 The present invention provides a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Figure 6 The computer device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0195] like Figure 6 As shown, the computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). Various programs and data required for the operation of the computer device are also stored in the RAM. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0196] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes and hard disks; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 6A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0197] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the multimodal underground mining road surface recognition method or the vehicle oil-gas suspension active control method of the embodiment of the present disclosure are executed.

[0198] For detailed description of this embodiment, please refer to the corresponding description in the aforementioned embodiments, which will not be repeated here.

[0199] According to an embodiment of the present disclosure, a computer-readable storage medium stores non-transitory computer-readable instructions. When executed by a processor, the non-transitory computer-readable instructions execute all or part of the steps of the multi-modal underground mining road surface recognition method or the vehicle oil-pneumatic suspension active control method described in each embodiment of the present disclosure.

[0200] The above-mentioned computer-readable storage media include, but are not limited to, optical storage media (e.g., CD-ROMs and DVDs), magneto-optical storage media (e.g., MOs), magnetic storage media (e.g., magnetic tapes or mobile hard disks), media with built-in rewritable non-volatile memory (e.g., memory cards), and media with built-in ROM (e.g., ROM cartridges).

[0201] For detailed description of this embodiment, please refer to the corresponding description in the aforementioned embodiments, which will not be repeated here.

[0202] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A multimodal underground mining road surface recognition method, characterized in that: include: Real-time acquisition of road surface images of target underground mines and determination of light intensity based on the road surface images; Dynamically determining a camera weight based on the light intensity, and determining a camera signal-to-noise ratio based on the road surface image; Determining optical perception credibility according to the camera signal-to-noise ratio and the camera weight; Dynamically determining a lidar weight based on the light intensity; Determine the LiDAR confidence index based on the environmental point cloud data of the target underground mine collected by the LiDAR; Determining the reliability of the environment point cloud according to the laser radar confidence index and the laser radar weight; Determining a vehicle response weight according to the camera weight and the lidar weight; Determine the degree of matching between vehicle response and road surface characteristics based on the vehicle's real-time acceleration while traveling in a target underground mine. determining a vibration characteristic matching gain between the vehicle vibration response and the road surface characteristic according to the vehicle response weight and the matching degree between the vehicle response and the road surface characteristic; Obtaining a multidimensional perception confidence according to the optical perception credibility, the environmental point cloud reliability, and the vibration feature matching gain; The road surface grade of the target underground mine is determined according to the multi-dimensional perception confidence.

2. The multimodal underground mining road surface recognition method according to claim 1, characterized in that: The real-time acquisition of a road surface image of a target underground mine and the determination of light intensity based on the road surface image include: A camera installed on an underground mining vehicle collects real-time road surface images, wherein the road surface images are RGB images; Converting the real-time road surface image into a grayscale image; determining light intensity based on the grayscale image; The light intensity is L: I(x i ,y i ) is the coordinate of the grayscale image (x i ,y i ) is the pixel grayscale value corresponding to the pixel point, and N is the total number of pixels selected from the grayscale image.

3. The multimodal underground mining road surface recognition method according to claim 2, characterized in that: The dynamically determining the camera weight according to the light intensity includes: Determine the preset medium light threshold for the target underground mine; dynamically determining a first slope factor according to the light intensity; The first slope factor is k1: Dynamically determining a camera weight according to a first preset function, the preset medium light threshold, the light intensity, and the first slope factor; The camera weight is ω α : Among them, L mid is the preset medium light threshold.

4. The multimodal underground mining road surface recognition method according to claim 3, characterized in that: Determining the camera signal-to-noise ratio according to the road surface image includes: Performing noise reduction processing on the road surface image, and performing road surface segmentation on the road surface image after noise reduction processing using a YOLO-MonoDepth network to obtain a grayscale image; Obtaining a grayscale mean value of the road surface area and a grayscale mean value of a non-road surface area on the image according to the grayscale image; Determining an image noise standard deviation based on the grayscale average value; The image noise standard deviation is σ n : Among them, μ b is the grayscale average value; Obtaining a road surface background grayscale contrast according to the grayscale mean and the grayscale average value; The road surface background grayscale contrast is Δμ: Δμ=μ r -μ b , where μ r is the grayscale mean; Determining dust interference intensity based on the road surface background grayscale contrast; The dust interference intensity is D d : C d is the dust interference factor; determining a dust compensation coefficient according to the image noise standard deviation and the dust interference intensity; The dust compensation coefficient is η: Determining a noise and dust comprehensive interference metric based on the image noise standard deviation, the dust compensation coefficient, and the dust interference intensity; The noise and dust comprehensive interference measurement is P: P = σ n +η·D d ; Determining a camera signal-to-noise ratio based on the road surface background grayscale contrast and the noise and dust comprehensive interference metric; The camera signal-to-noise ratio is S c :

5. The multimodal underground mining road surface recognition method according to claim 4, characterized in that: The dynamically determining the laser radar weight according to the light intensity includes: Determine the effective illumination lower limit threshold of the target underground mine according to the current tunnel depth corresponding to the road surface image collected at the target underground mine; The effective illumination lower limit threshold is L min :L min =35+15tanh(0.02D t ), D t is the current tunnel depth; Dynamically determining a second slope factor according to the light intensity and the effective light lower limit threshold; The second slope factor is k2: Determining an illumination suppression item for the laser radar according to the effective illumination lower limit threshold, the illumination intensity, and the first slope factor; The suppression term of the illumination on the lidar is y1: Determining an activation item of illumination for the laser radar according to the effective illumination lower limit threshold, the illumination intensity, and the second slope factor; The activation term of the light on the lidar is y2: Determining a laser radar weight according to an activation item of the laser radar by the illumination and an activation item of the laser radar by the illumination; The laser radar weight is ω β :ω β =y1·y2.

6. The multimodal underground mining road surface recognition method according to claim 5, characterized in that: Determining the laser radar confidence index based on the environmental point cloud data of the target underground mine collected by the laser radar includes: The laser radar installed on the top of the underground mining vehicle scans the surrounding environment in real time to obtain point cloud data; Preprocessing the point cloud data; Fitting ground points using the RANSAC algorithm based on the preprocessed point cloud data; Determining a lidar confidence index based on the preprocessed point cloud data and the fitted ground points; The lidar confidence index is ρ l : Among them, N v is the total number of ground points to be fitted, N t is the total number of the point cloud data after preprocessing, σ z is the standard deviation of the height of the fitted ground points, Z a is the average height of the fitted ground points.

7. The multimodal underground mining road surface recognition method according to claim 5, characterized in that: The determining of the vehicle response weight according to the camera weight and the lidar weight includes: The vehicle response weight is ω γ ,ω γ =1-ω α -ω β .

8. The multimodal underground mining road surface recognition method according to claim 7, characterized in that: The determining of the matching degree between the vehicle response and the road surface characteristics based on the real-time acceleration of the vehicle traveling in the target underground mine includes: The real-time acceleration corresponding to the preset period of the vehicle's travel in the target mine is collected by an acceleration sensor installed on the vehicle chassis; Performing Fourier transform on the real-time acceleration to extract the real-time main vibration frequency of the vehicle; Determining the degree of matching between the vehicle response and the road surface characteristics based on the real-time main vibration frequency of the vehicle and the preset characteristic frequency; The matching degree is M: Among them, f v is the real-time vibration main frequency of the vehicle, f r is the preset characteristic frequency.

9. The multimodal underground mining road surface recognition method according to claim 1, characterized in that: The multi-dimensional perception confidence is F: F = A + B + C; A=ω α ·S c ; B=ω β ·r l ; C=ω γ ·M; Among them, ω α is the camera weight, S c is the camera signal-to-noise ratio, ω β is the laser radar weight, ρ l is the lidar confidence index, ω γ is the vehicle response weight, and M is the matching degree between the vehicle response and the road surface characteristics.

10. A vehicle oil-gas suspension active control method, characterized in that: include: Dynamically optimize the sliding mode controller in the active oil-pneumatic suspension system corresponding to the underground mining vehicle according to the target underground mining road surface grade obtained by using the multimodal underground mining road surface identification method according to any one of claims 1 to 9; The stiffness and damping characteristics of the vehicle's oil-gas suspension are regulated in real time based on the optimized sliding mode controller.

Images