Obstacle avoidance method and system for coal mining equipment based on machine vision

Abstract

The present invention relates to the field of mining equipment obstacle avoidance based on image analysis, specifically a method and system for coal mining equipment obstacle avoidance based on machine vision. The method obtains the frequency domain characteristic parameters of the current electromagnetic noise of underground electromechanical equipment during operation and simultaneously collects the current original image frame output by the visual sensor. After dynamic denoising to obtain the current denoised image frame, the obstacle contour feature point set is extracted through multi-scale edge detection, and an obstacle position confidence map is obtained based on spatiotemporal consistency analysis. The point cloud coordinates of the image data loss area determined to be caused by electromagnetic interference are obtained and mapped to the image coordinate system to obtain a three-dimensional geometric completion image. A dynamic occupancy grid map of the underground environment is constructed based on the three-dimensional geometric completion image and the obstacle position confidence map, and the obstacle avoidance path of the mining equipment is planned based on the dynamic occupancy grid map. This method improves the obstacle avoidance capability of the mining equipment and ensures the safe and efficient operation of coal mining operations.

Description

Technical Field

[0001] The present invention relates to the field of mining equipment obstacle avoidance based on image analysis, and in particular to a method and system for coal mining equipment obstacle avoidance based on machine vision. Background Art

[0002] Coal mining operations present a complex and ever-changing underground environment, with numerous uncertainties, such as irregularly distributed rock protrusions, obstacles caused by collapses, and unexpected situations that may arise during the mining process. Reliable obstacle avoidance is crucial to ensuring the safe and efficient operation of mining equipment and preventing collisions between equipment and obstacles. Using visual sensors to acquire real-time image information of the underground environment, and analyzing and processing this image data to identify obstacles and plan avoidance paths, offers the advantages of intuitive and rich information acquisition.

[0003] However, in actual applications, the operation of underground electromechanical equipment generates strong electromagnetic interference, which can seriously affect the performance of visual sensors, resulting in a large amount of noise in the collected image data, or even image data loss. Specifically, electromagnetic interference can cause noise such as snowflakes and streaks to appear in the image, reducing image quality and affecting the accurate recognition of obstacles. Image data loss also results in the loss of information in local areas, making it difficult to obtain the complete outline of obstacles, and thus making it impossible to accurately determine the location and shape of obstacles.

[0004] Therefore, there is an urgent need for a coal mining equipment obstacle avoidance method based on machine vision to plan a safe and reliable obstacle avoidance path, significantly improve the obstacle avoidance capability of coal mining equipment in complex underground environments, and ensure the safe and efficient progress of mining operations. Summary of the Invention

[0005] The purpose of the present invention is to provide a coal mining equipment obstacle avoidance method and system based on machine vision to solve the problems of large image noise and data loss caused by underground electromagnetic interference during the operation of underground coal mining equipment, as well as the problem that dynamic interference noise affects the accuracy of obstacle position judgment.

[0006] To achieve the above objectives, the present invention provides, on the one hand, a method for avoiding obstacles in coal mining equipment based on machine vision, the method comprising:

[0007] S1. Real-time acquisition of electromagnetic interference intensity distribution data generated by underground electromechanical equipment during operation, and synchronous acquisition of the current original image frame output by the visual sensor; extraction of frequency domain characteristic parameters of the current electromagnetic noise by spectrum analysis of the electromagnetic interference intensity distribution data.

[0008] S2. Dynamically denoising the current original image frame based on the coupling relationship between the frequency domain characteristic parameters and the operating frequency band of the visual sensor to obtain a current denoised image frame, extracting an obstacle contour feature point set from the current denoised image frame through multi-scale edge detection, and obtaining an obstacle position confidence map based on the spatiotemporal consistency analysis of the obstacle contour feature point set of the previous denoised image frame adjacent in time sequence.

[0009] S3. When it is detected that the pixel grayscale variance of a preset number of consecutive rows in the current denoised image frame is lower than a preset noise threshold, it is determined to be an image data loss area caused by electromagnetic interference, and the point cloud coordinates of the image data loss area are obtained; the point cloud coordinates are mapped to the image coordinate system to obtain a three-dimensional geometric completion image.

[0010] S4. Construct a dynamic occupancy grid map of the underground environment based on the three-dimensional geometric completion image and the obstacle position confidence map, and plan an obstacle avoidance path for the mining equipment based on the dynamic occupancy grid map.

[0011] Furthermore, the method of performing dynamic denoising on the current original image frame according to the coupling relationship between the frequency domain characteristic parameters and the working frequency band of the visual sensor to obtain the current denoised image frame includes:

[0012] The frequency domain characteristic parameters include the main interference frequency, the proportion of harmonic energy, and the noise bandwidth; the absolute value of the frequency difference between the main interference frequency in the frequency domain characteristic parameters and the pixel sampling frequency band in the working frequency band of the visual sensor is calculated. If the absolute value of the frequency difference is less than or equal to the preset frequency difference threshold, it is determined to be a frequency band coupling interference mode, and a zero-phase IIR notch filter with the main interference frequency as the center frequency and a bandwidth associated with the noise bandwidth in the frequency domain characteristic parameters is constructed to filter the frequency domain data of the current original image frame to obtain the current denoised image frame.

[0013] If the absolute value of the frequency difference is greater than the preset frequency difference threshold and the harmonic energy ratio in the frequency domain characteristic parameter exceeds the preset energy ratio threshold, it is determined to be a broadband noise interference mode, and the current original image frame is decomposed into a time domain sequence according to the line scanning signal and multi-layer wavelet packet decomposition is performed. The sub-band range to be suppressed is determined according to the noise bandwidth, and the sub-band with excessive harmonic energy is screened out and the denoised line scanning signal is reconstructed and reorganized into the current denoised image frame; grayscale equalization processing is performed on the current denoised image frame.

[0014] Furthermore, the method of determining that the frequency band coupling interference mode is a frequency band coupling interference mode and constructing a zero-phase IIR notch filter with a main interference frequency as the center frequency and a bandwidth associated with the noise bandwidth in the frequency domain characteristic parameter to filter the frequency domain data of the current original image frame to obtain the current denoised image frame includes:

[0015] Detect whether the main interference frequency causes a periodic shift in the pixel sampling phase due to the frequency pulling effect. If the periodic shift in the pixel sampling phase is detected to exceed a preset phase shift threshold, generate a phase correction filter based on the phase shift period and shift direction of the main interference frequency. The phase correction filter is used to compensate for the misalignment between pixel rows.

[0016] The phase correction filter and the zero-phase IIR notch filter are applied in series to the frequency domain data of the current original image frame. The main interference frequency and its integer multiple harmonic components are first suppressed by the zero-phase IIR notch filter, and then the filtered frequency domain data is phase compensated by the phase correction filter to restore pixel row and column alignment; an inverse Fourier transform is performed on the phase-compensated frequency domain data to generate a current denoised image frame after phase correction.

[0017] Furthermore, the method for detecting whether the main interference frequency causes a periodic shift of the pixel sampling phase due to a frequency pulling effect includes:

[0018] A two-dimensional Fourier transform is performed on the current original image frame to extract the spatial spectrum, the periodic peak component corresponding to the main interference frequency in the spatial spectrum is analyzed, and the spatial frequency and phase distribution of the peak component are calculated; the period of the pixel row misalignment is determined based on the spatial frequency, and the misalignment direction is determined based on the positive and negative polarity of the phase distribution; if it is detected that the amplitude of the periodic peak component exceeds the preset amplitude threshold and the spatial frequency is inversely proportional to the absolute value of the frequency difference between the main interference frequency, it is determined that the pixel sampling phase is periodically offset due to the frequency pulling effect.

[0019] Furthermore, the method of obtaining the obstacle position confidence map based on the spatiotemporal consistency analysis of the obstacle contour feature point set of the previous denoised image frame adjacent in time sequence includes:

[0020] The obstacle contour feature point set of the current denoised image frame is matched point by point with the feature point set of the previous denoised image frame, the displacement vectors of the successfully matched feature point pairs are calculated, and initial confidence weights are assigned to the successfully matched feature points; a motion vector field is constructed based on the direction and amplitude of the displacement vectors, and the direction and amplitude of the displacement vectors of the motion vector field are extracted as clustering features through a density clustering algorithm; the directional variance and amplitude standard deviation of the displacement vectors within each cluster are calculated. If the directional variance is lower than a preset directional consistency threshold and the amplitude standard deviation is lower than a preset amplitude stability threshold, the cluster is determined to be a static obstacle feature point set; otherwise, the cluster is determined to be a dynamic interference noise feature point set.

[0021] For the static obstacle feature point set, the adjustment coefficient is obtained by calculating the cosine similarity between the displacement of the static obstacle feature point and the displacement of the cluster center, and the initial confidence weight is corrected according to the adjustment coefficient. For the dynamic interference noise feature point set, the noise intensity factor is calculated based on its directional variance and amplitude standard deviation, and the initial confidence weight is corrected according to the noise intensity factor. The obstacle position confidence map is obtained by performing time series smoothing filtering on the corrected initial confidence weights of the static obstacle feature point set and the dynamic interference noise feature point set.

[0022] Furthermore, when it is detected that the grayscale variance of pixels in a preset number of consecutive rows in the current denoised image frame is lower than a preset noise threshold, the method of determining that the image data loss area is caused by electromagnetic interference includes:

[0023] The pixel grayscale variance of the current denoised image frame is calculated row by row. If the pixel grayscale variance of a preset number of consecutive rows is lower than a preset noise threshold, it is determined to be a candidate image data loss area. The real-time electromagnetic interference intensity distribution data corresponding to the candidate image data loss area is obtained. If the electromagnetic interference intensity distribution data of the area exceeds a preset interference intensity threshold, it is determined to be an image data loss area caused by electromagnetic interference. If the electromagnetic interference intensity does not exceed the threshold, the candidate image data loss area is verified by using multiple frames of historical denoised images to determine whether there is a persistent pixel grayscale variance anomaly.

[0024] Furthermore, the method of mapping the point cloud coordinates to an image coordinate system to obtain a three-dimensional geometric completion image includes:

[0025] A rigid body transformation matrix is ​​calculated based on the joint calibration parameters of the visual sensor and the lidar, wherein the rigid body transformation matrix includes a rotation matrix and a translation vector; the point cloud coordinates of the image data missing area are obtained by the lidar; the point cloud coordinates are projected into the image coordinate system through the rigid body transformation matrix to obtain a three-dimensional geometric completion point cloud projection corresponding to the image data missing area; the three-dimensional geometric completion point cloud projection is surface reconstructed to generate a triangular mesh model, and the triangular mesh model is rendered as a three-dimensional geometric completion image consistent with the resolution and viewing angle of the current denoised image frame.

[0026] Furthermore, the method of constructing a dynamic occupancy grid map of the underground environment based on the three-dimensional geometric completion image and the obstacle position confidence map includes:

[0027] The three-dimensional geometric completion image is rasterized and segmented to generate an initial grid map, and the geometric structure integrity parameters of each grid cell in the initial grid map are extracted. The confidence values ​​of the corresponding grid cells in the obstacle position confidence map are mapped to the same grid coordinate system, and the initial occupancy probability of each grid cell is calculated based on the weighted fusion of the geometric structure integrity parameters and the confidence values.

[0028] The historical occupancy probability is updated in time series based on the real-time coordinate information of the mining equipment. The historical occupancy probability is exponentially decayed by a dynamic attenuation factor, and the initial occupancy probability is integrated to generate an updated occupancy probability. If the updated occupancy probability exceeds the preset static obstacle determination threshold, it is marked as an impassable area; otherwise, it is marked as a passable area. The impassable area and the passable area are combined with the initial grid map to obtain a dynamic occupancy grid map.

[0029] Furthermore, the method for extracting the geometric structure integrity parameter of each grid cell in the initial grid map includes:

[0030] Statistical filtering is performed on the point cloud data of the three-dimensional geometric completion image in the grid unit to obtain the number of valid point clouds, and the ratio of the number of valid point clouds to the volume of the grid unit is calculated as the point cloud density; a consistency analysis is performed on the normal vector distribution of the point cloud in the grid unit, and the covariance matrix eigenvalue of the normal vector is calculated by principal component analysis. If the ratio of the maximum eigenvalue to the minimum eigenvalue of the covariance matrix eigenvalue is lower than a preset characteristic threshold, the surface continuity is determined to be strong, otherwise it is determined to be a surface fracture; the point cloud density is normalized to obtain a density integrity factor, and the strong surface continuity and surface fracture are converted into a surface continuity coefficient. The geometric structure integrity parameter is obtained through weighted calculation based on the density integrity factor and the surface continuity coefficient.

[0031] On the other hand, based on the same inventive concept, the present invention also provides a machine vision-based obstacle avoidance system for coal mining equipment, the system comprising: a data analysis module, an obstacle position confidence map acquisition module, a three-dimensional geometric completion image acquisition module, and an obstacle avoidance planning management module, wherein the modules are sequentially connected in communication;

[0032] The data analysis module is used to obtain the electromagnetic interference intensity distribution data generated by the operation of underground electromechanical equipment in real time, and synchronously collect the current original image frame output by the visual sensor; the electromagnetic interference intensity distribution data is subjected to spectrum analysis to extract the frequency domain characteristic parameters of the current electromagnetic noise;

[0033] An obstacle position confidence map acquisition module is configured to dynamically denoise the current original image frame based on the coupling relationship between the frequency domain characteristic parameters and the operating frequency band of the visual sensor to obtain a current denoised image frame, extract an obstacle contour feature point set from the current denoised image frame through multi-scale edge detection, and obtain an obstacle position confidence map based on spatiotemporal consistency analysis of the obstacle contour feature point set of a temporally adjacent previous denoised image frame;

[0034] A 3D geometric completion image acquisition module is configured to, upon detecting that the grayscale variance of a preset number of consecutive rows of pixels in the current denoised image frame is lower than a preset noise threshold, determine that the image data is lost due to electromagnetic interference, obtain the point cloud coordinates of the image data loss area, and map the point cloud coordinates to the image coordinate system to obtain a 3D geometric completion image;

[0035] The obstacle avoidance planning management module is used to construct a dynamic occupancy grid map of the underground environment based on the three-dimensional geometric completion image and the obstacle position confidence map, and plan the obstacle avoidance path of the mining equipment based on the dynamic occupancy grid map.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. By analyzing the coupling relationship between the frequency-domain characteristic parameters of electromagnetic interference and the operating frequency band of the visual sensor, targeted methods such as zero-phase IIR notch filters and wavelet packet decomposition are used to dynamically denoise the image, effectively removing electromagnetic noise interference. Furthermore, for areas of image data lost due to electromagnetic interference, three-dimensional geometric completion is performed by obtaining point cloud coordinates and mapping them to the image coordinate system. This resolves the issue of missing image information, significantly improves image quality, and provides a reliable data foundation for subsequent obstacle recognition and environmental mapping.

[0038] 2. The spatial and temporal consistency analysis is performed using the obstacle contour feature point sets of temporally adjacent denoised image frames. Through a series of operations such as point-by-point matching, motion vector field construction, density clustering, and confidence weight correction, the static obstacle feature point sets and dynamic interference noise feature point sets are accurately distinguished, and an accurate obstacle position confidence map is obtained. This effectively reduces the impact of dynamic interference noise on obstacle position judgment and improves the accuracy of obstacle position determination.

[0039] 3. Combining the 3D geometric completion image with the obstacle position confidence map, the 3D geometric completion image is rasterized and segmented. The geometric structure integrity parameters of the grid cells are extracted and weighted fused with the confidence values ​​to calculate the initial occupancy probability. The updated occupancy probability is then obtained through time-series updates and dynamic attenuation factors, thereby constructing a reliable dynamic occupancy grid map. This provides accurate environmental information for obstacle avoidance path planning, ensuring its rationality and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flowchart of the machine vision-based coal mining equipment obstacle avoidance method according to Example 1 of the present invention.

[0041] Figure 2 This is a schematic diagram of the module composition of the machine vision-based coal mining equipment obstacle avoidance system according to Example 2 of the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] Before giving examples, it is necessary to explain the application scenarios of the present invention. The present invention is a machine vision-based obstacle avoidance method and system for coal mine mining equipment. When applied to coal mine mining equipment, it can achieve reliable obstacle detection and obstacle avoidance path planning in a strong electromagnetic interference environment underground, effectively solving the problems of large image noise and data loss caused by electromagnetic interference underground, as well as dynamic interference noise affecting the accuracy of obstacle position judgment, and significantly improving the operational safety and operating efficiency of mining equipment.

[0044] Example 1: Figure 1 As shown, this embodiment provides a method for avoiding obstacles in coal mining equipment based on machine vision, the method comprising:

[0045] S1. Real-time acquisition of electromagnetic interference intensity distribution data generated by underground electromechanical equipment during operation, while simultaneously capturing the current raw image frames output by the visual sensor. Spectral analysis is performed on this electromagnetic interference intensity distribution data to extract the frequency domain characteristic parameters of the current electromagnetic noise. In underground coal mines, the operation of electromechanical equipment generates electromagnetic interference. Electromagnetic field intensity sensor arrays installed around the mining equipment capture changes in electromagnetic field intensity generated by large electromechanical equipment, including shearers, roadheaders, and conveyors. Simultaneously, visual sensors (including high-definition cameras and industrial cameras installed at the front of the mining equipment) simultaneously capture raw image frames of the current underground environment. For example, when a shearer cuts a coal seam or a roadheader drills into a rock formation, equipment such as the motor and inverter generate strong electromagnetic interference in the 75-120 Hz range, which directly affects image quality. Spectral analysis of the acquired electromagnetic interference intensity distribution data is performed using a fast Fourier transform (FFT) to extract the frequency domain characteristic parameters of the current electromagnetic noise. These include the main interference frequency (usually the motor's drive frequency and its harmonics), the harmonic energy percentage (reflecting the energy distribution of the interference signal), and the noise bandwidth (indicating the spectral breadth of the interference signal). For example, when a shearer cuts through a hard coal seam, the increased motor load may generate electromagnetic interference with a main frequency of 100 Hz, a harmonic energy percentage of 35%, and a noise bandwidth of approximately 20 Hz.

[0046] Based on the coupling relationship between the frequency domain feature parameters and the visual sensor's operating frequency band, the current original image frame is dynamically denoised to obtain the current denoised image frame. Multi-scale edge detection is then used to extract the obstacle contour feature point set from the current denoised image frame. The obstacle location confidence map is then generated based on the spatiotemporal consistency analysis of the obstacle contour feature point set from the previous denoised image frame. In underground coal mine environments, electromagnetic interference can cause image distortion such as noticeable horizontal streaks, snowflake noise, or periodic ripples. The multi-scale edge detection algorithm can be implemented using the Canny edge detection algorithm combined with a Gaussian pyramid. Canny edge detection is applied to Gaussian pyramid images at different scales to obtain edge information at different scales. This edge information is then fused to extract the obstacle contour feature point set. This feature point set describes the edge contours of potential obstacles in the underground environment, such as rock outbursts, support equipment, and pipeline systems.

[0047] When the grayscale variance of a preset number of consecutive rows of pixels in the current denoised image frame is detected to be below a preset noise threshold, the image data loss region is determined to be caused by electromagnetic interference, and the point cloud coordinates of the image data loss region are obtained. These point cloud coordinates are mapped to the image coordinate system to obtain a 3D geometrically completed image. When the grayscale variance of a preset number of consecutive rows of pixels in the current denoised image frame is detected to be below a preset noise threshold, these regions are determined to be caused by electromagnetic interference. This situation is particularly common during the startup of high-power underground mining equipment or when the load suddenly changes, such as when a coal mining machine cuts through a fault or a roadheader encounters a hard rock formation. Auxiliary sensors (such as lidar) are then used to obtain the point cloud coordinates of these image data loss regions. These point cloud coordinates provide the 3D spatial location information of the obstacle. The completed image compensates for the information lost in the original image and can capture complete information about the underground environment.

[0048] A dynamic occupancy grid map of the underground environment is constructed based on the 3D geometric completion image and the obstacle position confidence map. Obstacle avoidance paths for mining equipment are planned based on this dynamic occupancy grid map. An A* algorithm or Dijkstra algorithm is used to plan obstacle avoidance paths for mining equipment on this dynamic occupancy grid map, ensuring safe and efficient mining operations.

[0049] The method for obtaining a current denoised image frame by dynamically denoising the current original image frame according to the coupling relationship between the frequency domain characteristic parameters and the working frequency band of the visual sensor includes:

[0050] Frequency domain characteristic parameters include the main interference frequency, harmonic energy ratio, and noise bandwidth. The absolute value of the frequency difference between the main interference frequency in the frequency domain characteristic parameters and the pixel sampling frequency band within the visual sensor's operating frequency band is calculated. If the absolute value of the frequency difference is less than or equal to a preset frequency difference threshold, a frequency-band-coupled interference mode is determined. A zero-phase IIR notch filter centered on the main interference frequency and with a bandwidth associated with the noise bandwidth in the frequency domain characteristic parameters is constructed to filter the frequency domain data of the current raw image frame to obtain the current denoised image frame. The pixel sampling frequency of a visual sensor is typically in the range of 20-60 Hz, while the interference frequency of downhole motor equipment is mostly in the range of 50-120 Hz. A frequency-band-coupled interference mode is determined when the absolute value of the frequency difference is less than or equal to a preset frequency difference threshold (typically set to 15-20 Hz). In this frequency-band-coupled interference mode, a zero-phase IIR notch filter centered on the main interference frequency and with a bandwidth associated with the noise bandwidth is constructed. For example, if the primary interference frequency of a coal mining machine is detected to be 98 Hz and the noise bandwidth is 15 Hz, a notch filter with a center frequency of 98 Hz and a bandwidth of 15 Hz is constructed to filter the frequency domain data of the current original image frame, thereby generating the current denoised image frame. The zero-phase design ensures that the filtering process does not introduce phase distortion, maintaining the accuracy of the edge positions of objects in the image.

[0051] If the absolute value of the frequency difference is greater than the preset frequency difference threshold and the harmonic energy ratio in the frequency domain characteristic parameter exceeds the preset energy ratio threshold, it is determined to be a broadband noise interference mode, and the current original image frame is decomposed into a time domain sequence according to the line scanning signal and multi-layer wavelet packet decomposition is performed. The sub-band range to be suppressed is determined according to the noise bandwidth, and the sub-band with excessive harmonic energy is screened out and the denoised line scanning signal is reconstructed and reorganized into the current denoised image frame; grayscale equalization processing is performed on the current denoised image frame.

[0052] When the absolute value of the frequency difference exceeds the preset frequency difference threshold and the harmonic energy ratio exceeds the preset energy ratio threshold (typically set to 30%), a broadband noise interference pattern is identified. This situation is common when mining equipment experiences sudden load changes or when multiple devices are operating simultaneously. The current original image frame is decomposed into a time-domain sequence based on the line scan signal and then subjected to multi-layer wavelet packet decomposition. Daubechies wavelets can be used as the wavelet basis function. The subband range to be suppressed is determined based on the noise bandwidth. Subbands with excessive harmonic energy are filtered out, and the denoised line scan signal is reconstructed to form the current denoised image frame. For example, when broadband noise interference is detected, 3-5 layers of wavelet packet decomposition are performed to suppress the energy of the corresponding subbands based on the interference bandwidth characteristics. A histogram equalization algorithm is then used to perform grayscale equalization on the current denoised image frame to enhance image contrast and make obstacle outlines clearer.

[0053] The method of determining that the frequency band coupling interference mode is the frequency band coupling interference mode, constructing a zero-phase IIR notch filter with a main interference frequency as the center frequency and a bandwidth associated with the noise bandwidth in the frequency domain characteristic parameter to filter the frequency domain data of the current original image frame to obtain the current denoised image frame includes:

[0054] Detect whether the main interference frequency causes a periodic shift in the pixel sampling phase due to the frequency pulling effect. If the periodic shift in the pixel sampling phase is detected to exceed a preset phase shift threshold, generate a phase correction filter based on the phase shift period and shift direction of the main interference frequency. The phase correction filter is used to compensate for the misalignment between pixel rows.

[0055] Where ϕ(z) is a phase compensation function designed based on the phase offset period and offset direction of the main interference frequency. Linear or nonlinear phase compensation methods can be used to align the rows and columns of filtered pixels.

[0056] The phase correction filter and the zero-phase IIR notch filter are applied in series to the frequency domain data of the current original image frame. The main interference frequency and its integer multiple harmonic components are first suppressed by the zero-phase IIR notch filter, and then the filtered frequency domain data is phase compensated by the phase correction filter to restore pixel row and column alignment; an inverse Fourier transform is performed on the phase-compensated frequency domain data to generate a current denoised image frame after phase correction.

[0057] When high-power equipment operates underground in coal mines, unstable power grid fluctuations can cause small fluctuations in the motor drive frequency. This in turn causes periodic shifts in the visual sensor sampling phase, manifesting as periodic lateral misalignment in the image. When the periodic pixel sampling phase offset exceeds a preset phase offset threshold (typically π / 8), a phase correction filter is generated based on the phase offset period and offset direction of the main interference frequency. This phase correction filter compensates for inter-row pixel misalignment and prevents distortion of obstacle outlines caused by this misalignment. For example, if a phase offset period of every 20 rows and a rightward offset direction are detected, the corresponding correction filter parameters are generated. Finally, an inverse Fourier transform is performed on the phase-compensated frequency data to generate the current, phase-corrected, denoised image frame.

[0058] The method for detecting whether the main interference frequency causes a periodic shift of the pixel sampling phase due to the frequency pulling effect includes:

[0059] A two-dimensional Fourier transform is performed on the current original image frame to extract the spatial spectrum, the periodic peak component corresponding to the main interference frequency in the spatial spectrum is analyzed, and the spatial frequency and phase distribution of the peak component are calculated; the period of the pixel row misalignment is determined based on the spatial frequency, and the misalignment direction is determined based on the positive and negative polarity of the phase distribution; if it is detected that the amplitude of the periodic peak component exceeds the preset amplitude threshold and the spatial frequency is inversely proportional to the absolute value of the frequency difference between the main interference frequency, it is determined that the pixel sampling phase is periodically offset due to the frequency pulling effect.

[0060] A two-dimensional fast Fourier transform (2D-FFT) algorithm is used to process the current raw image frame, converting the image matrix into a frequency-domain matrix. For example, when the primary interference frequency is 100 Hz, a corresponding periodic peak component is searched for in the spatial spectrum. The calculated spatial frequency can be used to determine the period of pixel row misalignment, such as occurring every 15 or 20 rows. The direction of the misalignment, such as left or right, is determined based on the positive or negative polarity of the phase distribution. When the amplitude of the detected periodic peak component exceeds a preset amplitude threshold (typically set at 3-5 times the average energy of the spatial spectrum), and the absolute value of the frequency difference between the spatial frequency and the primary interference frequency is inversely proportional, it is determined to be a periodic shift in the pixel sampling phase caused by the frequency pulling effect. This phenomenon is particularly noticeable during mining equipment startup or sudden load changes.

[0061] The method for obtaining the obstacle position confidence map based on the spatiotemporal consistency analysis of the obstacle contour feature point set of the previous denoised image frame adjacent in time sequence includes:

[0062] The obstacle contour feature point set of the current denoised image frame is matched point by point with the feature point set of the previous denoised image frame. The displacement vectors of the successfully matched feature point pairs are calculated and initial confidence weights are assigned to the successfully matched feature points. A motion vector field is constructed based on the direction and amplitude of the displacement vectors. The displacement vectors of the motion vector field are then subjected to a density clustering algorithm to extract the direction and amplitude of the displacement vectors as clustering features. The directional variance and amplitude standard deviation of the displacement vectors within each cluster are calculated. If the directional variance is lower than a preset directional consistency threshold and the amplitude standard deviation is lower than a preset amplitude stability threshold, the cluster is determined to be a static obstacle feature point set. Otherwise, the cluster is determined to be a dynamic interference noise feature point set. The improved SIFT or ORB feature descriptors are used for matching, the displacement vectors of the successfully matched feature point pairs are calculated, and initial confidence weights are assigned to the successfully matched feature points. For example, when the interval between the two image frames is 100ms, the feature points of static obstacles such as support equipment and rock protrusions can be effectively tracked. A motion vector field is constructed based on the direction and magnitude of the displacement vectors. The displacement vectors in the motion vector field are then clustered using a density clustering algorithm (such as DBSCAN). The direction and magnitude of the displacement vectors are then extracted as cluster features. The directional variance and magnitude standard deviation of the displacement vectors within each cluster are calculated. If the directional variance is below a preset directional consistency threshold (typically 0.1-0.2 radians) and the magnitude standard deviation is below a preset magnitude stability threshold (typically 2-3 pixels), the cluster is identified as a static obstacle feature point set. Otherwise, it is identified as a dynamic interference noise feature point set. This method effectively distinguishes static obstacles (such as rock outbursts and support equipment in tunnels) from dynamic interference noise (such as underground dust, water mist, or temporarily moving objects).

[0063] For the static obstacle feature point set, an adjustment coefficient is calculated by calculating the cosine similarity between the displacement of the static obstacle feature points and the displacement of the cluster center. The initial confidence weight is then corrected based on this adjustment coefficient. For the dynamic interference noise feature point set, a noise intensity factor is calculated based on its directional variance and amplitude standard deviation. The initial confidence weight is then corrected based on the noise intensity factor. The corrected initial confidence weights of the static obstacle feature point set and the dynamic interference noise feature point set are then subjected to time-series smoothing filtering to produce an obstacle location confidence map. Correcting the initial confidence weights based on the noise intensity factor typically reduces their weight values. The corrected initial confidence weights of the static obstacle feature point set and the dynamic interference noise feature point set are then filtered using an exponentially weighted moving average to produce the obstacle location confidence map. This processing method enables accurate obstacle location determination even in conditions of high dust concentration or strong water mist interference underground.

[0064] The method of determining that the image data loss area is caused by electromagnetic interference when it is detected that the grayscale variance of pixels in a preset number of consecutive rows in the current denoised image frame is lower than a preset noise threshold includes:

[0065] The pixel grayscale variance is calculated line by line for the current denoised image frame. If the pixel grayscale variance for a preset number of consecutive lines is below a preset noise threshold, the region is identified as a candidate for image data loss. Real-time electromagnetic interference intensity distribution data corresponding to the candidate image data loss region is obtained. If the electromagnetic interference intensity distribution data for the region exceeds the preset interference intensity threshold, the region is identified as an image data loss region caused by electromagnetic interference. If the electromagnetic interference intensity does not exceed the threshold, the candidate image data loss region is verified using multiple frames of historical denoised images to determine whether there is a persistent pixel grayscale variance anomaly. The pixel grayscale variance is calculated line by line for the current denoised image frame. If the pixel grayscale variance for a preset number of consecutive lines (usually 10-15% of the image height) is below a preset noise threshold (usually 5-10 grayscale levels), the region is identified as a candidate for image data loss. For example, in a 1080p resolution image, if the pixel grayscale variance for 100 consecutive lines is less than 8 grayscale levels, these areas are marked as candidate for image data loss. Real-time electromagnetic interference intensity distribution data corresponding to candidate image data loss areas is obtained. If the electromagnetic interference intensity distribution data in this area exceeds a preset interference intensity threshold (typically set at 5-10 times the background electromagnetic field intensity), the image data loss area is determined to be caused by electromagnetic interference. If the electromagnetic interference intensity does not exceed the threshold, the candidate image data loss area is verified by using multiple frames of historical denoised images to determine whether there is persistent pixel grayscale variance anomaly. This verification effectively distinguishes image data loss caused by electromagnetic interference from low grayscale variance caused by a relatively uniform underground environment (such as a flat tunnel wall).

[0066] The method of mapping the point cloud coordinates to the image coordinate system to obtain a three-dimensional geometric completion image includes:

[0067] The joint calibration parameters of the visual sensor and lidar are used to calculate a rigid body transformation matrix, which includes a rotation matrix and a translation vector. The point cloud coordinates of the region of image data missing from the lidar are then acquired. These point cloud coordinates are projected into the image coordinate system using the rigid body transformation matrix to obtain a 3D geometrically completed point cloud projection corresponding to the region of image data missing. This 3D geometrically completed point cloud projection is then surface reconstructed to generate a triangular mesh model, which is then rendered as a 3D geometrically completed image consistent with the resolution and viewing angle of the current denoised image frame. Joint calibration is typically performed using a calibration plate before the device is put into use. The calibration process calculates the geometric relationship between the camera and lidar. The lidar is able to operate stably in electromagnetic interference environments and provide high-precision 3D structural information. For example, when data loss is detected in the central region of the image, the lidar point cloud data within that viewing angle is extracted and coordinate transformed. For example, a modified Poisson surface reconstruction algorithm or a greedy triangulation algorithm can be used to process unevenly distributed point cloud data. The triangular mesh model is rendered into a 3D geometric completion image that is consistent with the resolution and viewing angle of the current denoised image frame, thereby filling in the lost area information in the original image. This completion method can ensure that complete downhole environmental information can still be obtained under severe electromagnetic interference.

[0068] The method for constructing a dynamic occupancy grid map of an underground environment based on the three-dimensional geometric completion image and the obstacle position confidence map includes:

[0069] The three-dimensional geometric completion image is rasterized and segmented to generate an initial raster map, and the geometric structure integrity parameters of each grid cell in the initial raster map are extracted; the confidence values ​​of the corresponding grid cells in the obstacle position confidence map are mapped to the same grid coordinate system, and the initial occupancy probability of each grid cell is calculated based on the weighted fusion of the geometric structure integrity parameters and the confidence values; for example, the grid cell size is usually set to 10-20 cm, and such a resolution can take into account both computational efficiency and obstacle avoidance accuracy. The geometric structure integrity parameters of each grid cell in the initial raster map are extracted, and the geometric structure integrity parameters reflect the density and surface continuity of the point cloud distribution within the grid cell. For example, when the geometric structure integrity parameter of a grid cell is 0.8 and the corresponding confidence value is 0.75, an initial occupancy probability of 0.78 may be calculated.

[0070] The historical occupancy probabilities are updated sequentially based on the real-time coordinate information of the mining equipment. These historical occupancy probabilities are exponentially decayed using a dynamic attenuation factor and then fused with the initial occupancy probabilities to generate updated occupancy probabilities. If the updated occupancy probability exceeds a preset static obstacle threshold, the area is marked as impassable; otherwise, it is marked as passable. These impassable and passable areas are combined with the initial grid map to generate a dynamic occupancy grid map. In underground environments, the status of certain areas may change as mining operations progress. The dynamic attenuation mechanism ensures the timeliness of map information. If the updated occupancy probability exceeds a preset static obstacle threshold (typically set at 0.6-0.7), the area is marked as impassable; otherwise, it is marked as passable. The impassable and passable areas are combined with the initial grid map to generate a dynamic occupancy grid map, which serves as the basis for obstacle avoidance path planning.

[0071] The method for extracting the geometric structure integrity parameter of each grid cell in the initial grid map includes:

[0072] The point cloud data within the grid cell of the three-dimensional geometric completion image is statistically filtered to obtain the number of valid point clouds. The ratio of the number of valid point clouds to the grid cell volume is calculated as the point cloud density. A consistency analysis is performed on the distribution of normal vectors within the grid cell. The eigenvalues ​​of the covariance matrix of the normal vectors are calculated using principal component analysis. If the ratio of the maximum eigenvalue to the minimum eigenvalue of the covariance matrix eigenvalues ​​is lower than a preset characteristic threshold, the surface is determined to have strong continuity; otherwise, it is determined to have a surface fracture. The point cloud density is normalized to obtain a density integrity factor, and strong surface continuity and surface fracture are converted into a surface continuity coefficient. The geometric structure integrity parameter is obtained through a weighted calculation based on the density integrity factor and the surface continuity coefficient. For example, if the number of valid point clouds within a 10cm×10cm×10cm grid cell is 120, the point cloud density is 120 / (10×10×10) = 0.12 points / cm³. Strong surface continuity indicates a smooth and uniform surface within the grid cell; surface fracture indicates the presence of discontinuous structures such as edges and corners within the grid cell. The density integrity factor is normalized by normalizing the point cloud density, and strong surface continuity and surface fracture are converted into surface continuity coefficients. Typically, strong continuity is assigned a value of 0.9-1.0, and surface fracture is assigned a value of 0.5-0.7.

[0073] Example 2: Based on the same inventive concept, Figure 2 As shown, this embodiment also provides a coal mining equipment obstacle avoidance system based on machine vision, the system includes: a data analysis module, an obstacle position confidence map acquisition module, a three-dimensional geometric completion image acquisition module, and an obstacle avoidance planning management module, and each module is sequentially connected to each other;

[0074] The data analysis module is used to obtain the electromagnetic interference intensity distribution data generated by the operation of underground electromechanical equipment in real time, and synchronously collect the current original image frame output by the visual sensor; the electromagnetic interference intensity distribution data is subjected to spectrum analysis to extract the frequency domain characteristic parameters of the current electromagnetic noise;

[0075] An obstacle position confidence map acquisition module is configured to dynamically denoise the current original image frame based on the coupling relationship between the frequency domain characteristic parameters and the operating frequency band of the visual sensor to obtain a current denoised image frame, extract an obstacle contour feature point set from the current denoised image frame through multi-scale edge detection, and obtain an obstacle position confidence map based on spatiotemporal consistency analysis of the obstacle contour feature point set of a temporally adjacent previous denoised image frame;

[0076] A 3D geometric completion image acquisition module is configured to, upon detecting that the grayscale variance of a preset number of consecutive rows of pixels in the current denoised image frame is lower than a preset noise threshold, determine that the image data is lost due to electromagnetic interference, obtain the point cloud coordinates of the image data loss area, and map the point cloud coordinates to the image coordinate system to obtain a 3D geometric completion image;

[0077] The obstacle avoidance planning management module is used to construct a dynamic occupancy grid map of the underground environment based on the three-dimensional geometric completion image and the obstacle position confidence map, and plan the obstacle avoidance path of the mining equipment based on the dynamic occupancy grid map.

[0078] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0079] Finally, it should be noted that although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A coal mining equipment obstacle avoidance method based on machine vision, characterized in that: The method comprises: Real-time acquisition of electromagnetic interference intensity distribution data generated by underground electromechanical equipment during operation, and simultaneous acquisition of the current original image frame output by the visual sensor; extraction of frequency domain characteristic parameters of the current electromagnetic noise from the electromagnetic interference intensity distribution data through spectrum analysis; Dynamically denoising the current original image frame based on the coupling relationship between the frequency domain feature parameters and the operating frequency band of the visual sensor to obtain a current denoised image frame, extracting an obstacle contour feature point set from the current denoised image frame through multi-scale edge detection, and obtaining an obstacle position confidence map based on spatiotemporal consistency analysis of the obstacle contour feature point set of a temporally adjacent previous denoised image frame; When it is detected that the grayscale variance of pixels in a preset number of consecutive rows in the current denoised image frame is lower than a preset noise threshold, it is determined to be an image data loss area caused by electromagnetic interference, and the point cloud coordinates of the image data loss area are obtained; the point cloud coordinates are mapped to the image coordinate system to obtain a three-dimensional geometric completion image; A dynamic occupancy grid map of the underground environment is constructed based on the three-dimensional geometric completion image and the obstacle position confidence map, and an obstacle avoidance path of the mining equipment is planned based on the dynamic occupancy grid map.

2. The method for avoiding obstacles in coal mining equipment based on machine vision according to claim 1, characterized in that: The method for obtaining a current denoised image frame by dynamically denoising the current original image frame according to the coupling relationship between the frequency domain characteristic parameters and the working frequency band of the visual sensor includes: The frequency domain characteristic parameters include the main interference frequency, the harmonic energy ratio, and the noise bandwidth; the absolute value of the frequency difference between the main interference frequency in the frequency domain characteristic parameters and the pixel sampling frequency band in the working frequency band of the visual sensor is calculated; if the absolute value of the frequency difference is less than or equal to a preset frequency difference threshold, it is determined to be a frequency band coupling interference mode, and a zero-phase IIR notch filter with the main interference frequency as the center frequency and a bandwidth associated with the noise bandwidth in the frequency domain characteristic parameters is constructed to filter the frequency domain data of the current original image frame to obtain the current denoised image frame; If the absolute value of the frequency difference is greater than the preset frequency difference threshold and the harmonic energy ratio in the frequency domain characteristic parameter exceeds the preset energy ratio threshold, it is determined to be a broadband noise interference mode, and the current original image frame is decomposed into a time domain sequence according to the line scanning signal and multi-layer wavelet packet decomposition is performed. The sub-band range to be suppressed is determined according to the noise bandwidth, and the sub-band with excessive harmonic energy is screened out and the denoised line scanning signal is reconstructed and reorganized into the current denoised image frame; grayscale equalization processing is performed on the current denoised image frame.

3. The method for avoiding obstacles in coal mining equipment based on machine vision according to claim 2, characterized in that: The method of determining that the frequency band coupling interference mode is the frequency band coupling interference mode, constructing a zero-phase IIR notch filter with a main interference frequency as the center frequency and a bandwidth associated with the noise bandwidth in the frequency domain characteristic parameter to filter the frequency domain data of the current original image frame to obtain the current denoised image frame includes: Detecting whether the main interference frequency causes a periodic shift in the pixel sampling phase due to a frequency pulling effect, and if the detected periodic shift in the pixel sampling phase exceeds a preset phase shift threshold, generating a phase correction filter based on the phase shift period and shift direction of the main interference frequency, wherein the phase correction filter is used to compensate for the misalignment between pixel rows; The phase correction filter and the zero-phase IIR notch filter are applied in series to the frequency domain data of the current original image frame. The main interference frequency and its integer multiple harmonic components are first suppressed by the zero-phase IIR notch filter, and then the filtered frequency domain data is phase compensated by the phase correction filter to restore pixel row and column alignment; an inverse Fourier transform is performed on the phase-compensated frequency domain data to generate a current denoised image frame after phase correction.

4. The method for avoiding obstacles in coal mining equipment based on machine vision according to claim 3, characterized in that: The method for detecting whether the main interference frequency causes a periodic shift of the pixel sampling phase due to the frequency pulling effect includes: A two-dimensional Fourier transform is performed on the current original image frame to extract the spatial spectrum, the periodic peak component corresponding to the main interference frequency in the spatial spectrum is analyzed, and the spatial frequency and phase distribution of the peak component are calculated; the period of the pixel row misalignment is determined based on the spatial frequency, and the misalignment direction is determined based on the positive and negative polarity of the phase distribution; if it is detected that the amplitude of the periodic peak component exceeds the preset amplitude threshold and the spatial frequency is inversely proportional to the absolute value of the frequency difference between the main interference frequency, it is determined that the pixel sampling phase is periodically offset due to the frequency pulling effect.

5. The method for avoiding obstacles in coal mining equipment based on machine vision according to claim 1, characterized in that: The method for obtaining the obstacle position confidence map based on the spatiotemporal consistency analysis of the obstacle contour feature point set of the previous denoised image frame adjacent in time sequence includes: The obstacle contour feature point set of the current denoised image frame is matched point by point with the feature point set of the previous denoised image frame, the displacement vectors of the successfully matched feature point pairs are calculated, and initial confidence weights are assigned to the successfully matched feature points; a motion vector field is constructed based on the direction and amplitude of the displacement vectors, and the direction and amplitude of the displacement vectors of the motion vector field are extracted as clustering features using a density clustering algorithm; the directional variance and amplitude standard deviation of the displacement vectors within each cluster are calculated; if the directional variance is lower than a preset directional consistency threshold and the amplitude standard deviation is lower than a preset amplitude stability threshold, the cluster is determined to be a static obstacle feature point set; otherwise, the cluster is determined to be a dynamic interference noise feature point set; For the static obstacle feature point set, the adjustment coefficient is obtained by calculating the cosine similarity between the displacement of the static obstacle feature point and the displacement of the cluster center, and the initial confidence weight is corrected according to the adjustment coefficient. For the dynamic interference noise feature point set, the noise intensity factor is calculated based on its directional variance and amplitude standard deviation, and the initial confidence weight is corrected according to the noise intensity factor. The obstacle position confidence map is obtained by performing time series smoothing filtering on the corrected initial confidence weights of the static obstacle feature point set and the dynamic interference noise feature point set.

6. The method for avoiding obstacles in coal mining equipment based on machine vision according to claim 1, characterized in that: The method of determining that the image data loss area is caused by electromagnetic interference when it is detected that the grayscale variance of pixels in a preset number of consecutive rows in the current denoised image frame is lower than a preset noise threshold includes: The pixel grayscale variance of the current denoised image frame is calculated row by row. If the pixel grayscale variance of a preset number of consecutive rows is lower than a preset noise threshold, it is determined to be a candidate image data loss area. The real-time electromagnetic interference intensity distribution data corresponding to the candidate image data loss area is obtained. If the electromagnetic interference intensity distribution data of the area exceeds a preset interference intensity threshold, it is determined to be an image data loss area caused by electromagnetic interference. If the electromagnetic interference intensity does not exceed the threshold, the candidate image data loss area is verified by using multiple frames of historical denoised images to determine whether there is a persistent pixel grayscale variance anomaly.

7. The method for avoiding obstacles in coal mining equipment based on machine vision according to claim 1, characterized in that: The method of mapping the point cloud coordinates to the image coordinate system to obtain a three-dimensional geometric completion image includes: A rigid body transformation matrix is ​​calculated based on the joint calibration parameters of the visual sensor and the lidar, wherein the rigid body transformation matrix includes a rotation matrix and a translation vector; the point cloud coordinates of the image data missing area are obtained by the lidar; the point cloud coordinates are projected into the image coordinate system through the rigid body transformation matrix to obtain a three-dimensional geometric completion point cloud projection corresponding to the image data missing area; the three-dimensional geometric completion point cloud projection is surface reconstructed to generate a triangular mesh model, and the triangular mesh model is rendered as a three-dimensional geometric completion image consistent with the resolution and viewing angle of the current denoised image frame.

8. The method for avoiding obstacles in coal mining equipment based on machine vision according to claim 1, characterized in that: The method for constructing a dynamic occupancy grid map of an underground environment based on the three-dimensional geometric completion image and the obstacle position confidence map includes: The three-dimensional geometric completion image is rasterized and segmented to generate an initial grid map, and the geometric structure integrity parameters of each grid cell in the initial grid map are extracted; the confidence values ​​of the corresponding grid cells in the obstacle position confidence map are mapped to the same grid coordinate system, and the initial occupancy probability of each grid cell is calculated based on the weighted fusion of the geometric structure integrity parameters and the confidence values; The historical occupancy probability is updated in time series based on the real-time coordinate information of the mining equipment. The historical occupancy probability is exponentially decayed by a dynamic attenuation factor, and the initial occupancy probability is integrated to generate an updated occupancy probability. If the updated occupancy probability exceeds the preset static obstacle determination threshold, it is marked as an impassable area; otherwise, it is marked as a passable area. The impassable area and the passable area are combined with the initial grid map to obtain a dynamic occupancy grid map.

9. The method for avoiding obstacles in coal mining equipment based on machine vision according to claim 8, characterized in that: The method for extracting the geometric structure integrity parameter of each grid cell in the initial grid map includes: Statistical filtering is performed on the point cloud data of the three-dimensional geometric completion image in the grid unit to obtain the number of valid point clouds, and the ratio of the number of valid point clouds to the volume of the grid unit is calculated as the point cloud density; a consistency analysis is performed on the normal vector distribution of the point cloud in the grid unit, and the covariance matrix eigenvalue of the normal vector is calculated by principal component analysis. If the ratio of the maximum eigenvalue to the minimum eigenvalue of the covariance matrix eigenvalue is lower than a preset characteristic threshold, the surface continuity is determined to be strong, otherwise it is determined to be a surface fracture; the point cloud density is normalized to obtain a density integrity factor, and the strong surface continuity and surface fracture are converted into a surface continuity coefficient. The geometric structure integrity parameter is obtained through weighted calculation based on the density integrity factor and the surface continuity coefficient.

10. The machine vision-based obstacle avoidance system for coal mining equipment is characterized by: The system includes: a data analysis module, an obstacle position confidence map acquisition module, a three-dimensional geometric completion image acquisition module, and an obstacle avoidance planning management module, and each module is sequentially connected to each other; The data analysis module is used to obtain the electromagnetic interference intensity distribution data generated by the operation of underground electromechanical equipment in real time, and synchronously collect the current original image frame output by the visual sensor; the electromagnetic interference intensity distribution data is subjected to spectrum analysis to extract the frequency domain characteristic parameters of the current electromagnetic noise; An obstacle position confidence map acquisition module is configured to dynamically denoise the current original image frame based on the coupling relationship between the frequency domain characteristic parameters and the operating frequency band of the visual sensor to obtain a current denoised image frame, extract an obstacle contour feature point set from the current denoised image frame through multi-scale edge detection, and obtain an obstacle position confidence map based on spatiotemporal consistency analysis of the obstacle contour feature point set of a temporally adjacent previous denoised image frame; A 3D geometric completion image acquisition module is configured to, upon detecting that the grayscale variance of a preset number of consecutive rows of pixels in the current denoised image frame is lower than a preset noise threshold, determine that the image data is lost due to electromagnetic interference, obtain the point cloud coordinates of the image data loss area, and map the point cloud coordinates to the image coordinate system to obtain a 3D geometric completion image; The obstacle avoidance planning management module is used to construct a dynamic occupancy grid map of the underground environment based on the three-dimensional geometric completion image and the obstacle position confidence map, and plan the obstacle avoidance path of the mining equipment based on the dynamic occupancy grid map.

Images