Image-based evaluation method and device for cutting effect of lower drum of coal mining machine

Abstract

This invention relates to the field of automated control technology in fully mechanized coal mining, and discloses a method and apparatus for evaluating the cutting effect of the lower drum of a coal mining machine based on images. The method first performs denoising and format conversion on the acquired images, then uses a deep convolutional neural network to locate the drum and extract regions of interest, maintaining data validity through an effective pixel detection mechanism. Next, adaptive dual-threshold segmentation and morphological operations are performed to refine the extraction of the total rock powder area and the average brightness features of the rock powder region. Subsequently, a three-dimensional state feature vector containing the mean area, mean brightness, and area standard deviation is constructed based on a temporal sliding window. Based on this, the state of no-load operation, coal seam cutting, or rock cutting is determined, and a drum height adjustment control command is generated. This invention overcomes the interference of the harsh underground environment through multi-dimensional feature fusion and temporal statistical mechanisms, achieving high-precision identification of the cutting state and adaptive closed-loop control of the drum layer.

Description

Technical Field

[0001] This invention relates to the field of automated control technology for fully mechanized coal mining, specifically to a method and apparatus for evaluating the cutting effect of the lower drum of a coal mining machine based on images. Background Technology

[0002] The environment of fully mechanized coal mining faces is complex, and accurate identification of the cutting status of the coal shearing machine's drum is crucial for achieving unmanned automatic height adjustment control. Although existing cutting status recognition technologies have gradually incorporated machine vision methods, conventional image processing methods lack effective image enhancement and line-of-sight occlusion handling mechanisms in harsh underground environments with high dust, low illumination, and water mist interference. This results in insufficient continuity and anti-interference capabilities in feature extraction, making it difficult to obtain stable source data under visually limited conditions.

[0003] Meanwhile, traditional identification methods rely on a single grayscale threshold or texture feature to determine the working condition. The feature dimension is single and the coupling degree is high. They are easily affected by changes in ambient light and it is difficult to accurately distinguish the subtle differences in optical and geometric morphology between cut rock and cut coal seam through simple features. This results in a high misjudgment rate in the classification and identification of unloaded, coal cut and rock cut states.

[0004] In addition, existing control strategies are usually based on instantaneous detection results for feedback, lacking time-series statistics and consistency verification of historical data. This makes the system extremely sensitive to occasional sudden noise, which can easily lead to false alarms in state judgment. Consequently, it can cause frequent jumps and oscillations in the control command, failing to meet the actual production needs of smooth layering, adaptive adjustment, and protection of cutting teeth in coal mining machines. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an image-based method and apparatus for evaluating the cutting effect of the lower drum of a coal mining machine. This solves the problems of poor feature extraction and anti-interference capabilities in underground high-dust and visual obstruction environments, low accuracy of cutting state recognition due to reliance on a single feature, and frequent changes in control commands due to lack of temporal stability.

[0006] The first aspect of this invention provides an image-based method for evaluating the cutting effect of the lower drum of a coal mining machine, comprising:

[0007] First, basic denoising and format conversion processing is performed on the original color image of the current frame output by the image acquisition unit. Specifically, this includes extracting channel components to convert the color image into a grayscale image, and then using Gaussian convolution and median filtering to eliminate smoothing and discrete noise, thereby obtaining a basic denoised image.

[0008] Secondly, based on the denoised image, a deep convolutional neural network is used to locate the geometric position of the lower drum of the coal mining machine. Based on this, the region of interest image is extracted, and validity maintenance is performed. By detecting the proportion of valid pixels in the region of interest and comparing it with the occlusion determination threshold, when a line-of-sight occlusion is determined, the cached valid image from the previous frame is used to replace it, ensuring the continuity and validity of the input data.

[0009] Next, adaptive thresholding and morphological operations are performed on the region of interest image to extract rock powder features. Pixels are then classified using a dual threshold determined by the Otsu's method to generate a binarized image. The topology is optimized through morphological closing and opening operations, and the total rock powder area and average brightness of the rock powder region are calculated based on the filtered effective connected regions.

[0010] Subsequently, based on a temporal sliding window mechanism, the rock powder features of multiple frames are statistically aggregated to construct a three-dimensional state feature vector. This step statistically analyzes the mean area, mean brightness, and standard deviation of rock powder area within the sliding window to characterize the current cutting load, material reflectivity, and cutting volatility.

[0011] Finally, the final cutting state is determined based on the three-dimensional state feature vector, and height adjustment control commands are generated. The no-load state is distinguished by comparing the average rock powder area with the no-load dust generation threshold; the state of cutting rock and cutting coal seam is distinguished by constructing a lithology discrimination index that integrates the average brightness and the standard deviation of rock powder area and comparing it with the lithology discrimination threshold. After the final cutting state is determined through temporal consistency verification, corresponding height adjustment control commands for rock avoidance, coal exploration, or maintaining the stratum are generated, driving the coal mining machine's actuators to operate.

[0012] A second aspect of the present invention provides an image-based device for evaluating the cutting effect of the lower drum of a coal mining machine, applied to the method described in the first aspect above. The device includes:

[0013] The image acquisition unit is configured to acquire real-time video streams and output the original color image of the current frame; the region of interest processing unit is configured to perform image denoising and format conversion, extract the region of interest image based on the geometric position of the drum, and maintain image validity when the line of sight is obstructed using a caching mechanism; the rock powder feature refinement extraction unit is configured to perform adaptive dual-threshold segmentation and morphological operations, and quantify the total rock powder area and the average brightness of the rock powder area; the cutting state adaptive evaluation unit is configured to construct a three-dimensional state feature vector containing the mean and standard deviation based on a temporal sliding window, and determine the final cutting state accordingly; the feedback control execution unit is configured to generate and output height adjustment control commands based on the final cutting state and the current actual height.

[0014] This invention provides an image-based method and apparatus for evaluating the cutting effect of the lower drum of a coal mining machine. It has the following beneficial effects:

[0015] 1. This invention uses a deep convolutional neural network to accurately locate the roller and combines it with an effective pixel ratio detection mechanism. When the line of sight is obstructed, the cached image is automatically called up, ensuring the continuity and effectiveness of the feature extraction source data. With the help of dual threshold hysteresis segmentation and morphological opening and closing operations, it can filter background noise and artifacts in the high dust and low light environment downhole, ensuring that rock powder features can still be accurately extracted in harsh visual environments.

[0016] 2. This invention constructs a three-dimensional state feature vector that includes the average area of ​​rock powder, the average brightness, and the standard deviation of rock powder area, realizing feature fusion from three dimensions: the scale of dust generation distribution, the optical reflection characteristics of the material, and the stability of the cutting process. This multi-dimensional evaluation strategy can capture the differences between the cut rock and the cut coal seam in the image, solve the problem of misjudgment caused by the susceptibility of single features to changes in ambient light, and improve the recognition accuracy of the three states of no load, cut coal seam, and cut rock.

[0017] 3. This invention introduces a time-series sliding window statistical and confidence probability verification mechanism, which uses historical frame data to smooth instantaneous fluctuations, eliminates the impact of accidental mutations on state determination, and avoids frequent jumps in control commands. Based on the final determined cutting state, differentiated height adjustment control commands (rock avoidance, coal search) are generated, realizing adaptive closed-loop adjustment of the coal mining machine drum layer, which improves the efficiency of unmanned mining while protecting the cutting teeth and reducing wear. Attached Figure Description

[0018] Figure 1 A flowchart illustrating the overall process of an image-based method for evaluating the cutting effect of a lower drum in a coal mining machine, as provided in an embodiment of the present invention.

[0019] Figure 2 This is a detailed logic flowchart of image preprocessing and region of interest acquisition in an embodiment of the present invention;

[0020] Figure 3 This is a structural block diagram of an image-based coal mining machine lower drum cutting effect evaluation device provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The image-based method for evaluating the cutting effect of the lower drum of a coal mining machine provided by this invention runs on this hardware architecture, which mainly includes an image acquisition device, a central computing and processing terminal, and a coal mining machine control and execution mechanism.

[0023] The image acquisition equipment is configured to acquire real-time video streams including the lower drum of the coal mining machine and the cutting contact surface. In the specific physical deployment, the image acquisition equipment uses intrinsically safe or explosion-proof high-definition cameras, installed in an unobstructed position on the outside of the coal mining machine or the inside of the rocker arm. The camera's lens optical axis points towards the cutting area of ​​the lower drum of the coal mining machine, and the field of view (FOV) covers the drum body, the contact interface between the lower edge of the drum and the coal seam floor, and the areas of spilled material generated on both sides of the drum. The image acquisition equipment is connected to the central computing terminal via a fiber optic or gigabit Ethernet interface, at a frame rate of... (For example, 30fps) Output resolution is (For example ) digital video signal .

[0024] The central computing and processing terminal is the main execution unit of this method. It is configured to receive video signals and execute image processing and logical judgment algorithms. It adopts an embedded industrial control computer (IPC) or an AI processing box with edge computing capabilities, and needs to have an independent graphics processing unit (GPU) or neural network accelerator (NPU) to meet real-time computing requirements. The central computing and processing terminal establishes a bidirectional communication connection with the coal mining machine control and execution mechanism through an explosion-proof communication interface (such as RS485, CAN bus, or industrial Ethernet).

[0025] The control actuator of a coal mining machine mainly refers to the programmable logic controller (PLC) and its actuators. The control actuator is configured to receive status assessment results or control commands from the central computing terminal and adjust the traction speed of the coal mining machine accordingly. Drum cutting height Or it may trigger an audible and visual alarm device.

[0026] See attached document Figure 1 Step S100: Image Acquisition and Enhancement Preprocessing. This method first acquires a real-time video stream using a mining camera and performs serial processing on the original image, including grayscale conversion, cascaded denoising (Gaussian and median filtering), guided filtering for dehazing, and CLAHE visual enhancement. This step aims to eliminate the interference of low-light, high-dust, and water-mist environments in the mine on image quality, and output a base image with a high signal-to-noise ratio.

[0027] Step S200: Dynamic Region of Interest (ROI) localization and maintenance. A deep convolutional neural network (CNN) is used to accurately locate the geometric coordinates of the roller, and based on this, the ROI, including the cut contact surface, is dynamically cropped. Simultaneously, the system detects the effective pixel percentage within the ROI. If visual occlusion is detected, the system automatically retrieves cached data from the previous frame for compensation, ensuring the accuracy and continuity of the feature extraction region.

[0028] Step S300: Refined quantization of rock powder features. Within the ROI region, a dual-threshold adaptive segmentation algorithm is used to extract rock powder targets, and the topology is optimized through morphological closing and opening operations. Subsequently, the system calculates the total rock powder area (representing the truncating load) and average brightness (representing the truncated material) of the effectively connected regions, completing the transformation from image to physical features.

[0029] Step S400: Adaptive temporal state assessment. A temporal sliding window mechanism is introduced to calculate the mean and variance of features across multiple frames, constructing a three-dimensional state feature vector. Based on a multi-level logistic regression model, the system accurately classifies the current operating condition into three states: no-load, coal seam cutting, or rock cutting, by comparing parameters such as the no-load dust generation threshold and lithology discrimination index.

[0030] Step S500: Closed-loop feedback control commands are generated, and the timing consistency of the preliminary assessment results is verified to filter out random interference. Based on the final confirmed cutting status, the system generates corresponding height adjustment control commands (e.g., rapid lifting when encountering rock, slow lowering under no-load conditions, and maintaining the cutting level), and sends them to the coal mining machine controller after mechanical stroke constraints to achieve adaptive adjustment of the drum level.

[0031] In the implementation of this method, a digital video signal is defined. A series of consecutive image frames ,in Represents a time series. For any frame of an image... Establish a two-dimensional pixel coordinate system ,in This method aims to improve the performance of digital video signals. The analysis establishes a state space from the image space to the physical cutting state space of the coal mining machine. mapping relationship : ,in ,in This indicates that the coal mining machine drum is currently cutting the coal face; This indicates that the coal mining machine drum has come into contact with the rock; This indicates a state where the vision sensor has failed or there is excessive environmental interference.

[0032] In step S100, the system first performs basic processing on the original acquired video data. This process includes two consecutive stages: grayscale conversion and cascaded mixing denoising, which aims to unify the data format and remove the basic noise unique to the downhole environment.

[0033] The data processing unit first receives the current frame's original color image output by the image acquisition unit. For any pixel in the image plane, its horizontal coordinate is set to... The vertical coordinate is The system extracts the red, green, and blue channel components of the pixel.

[0034] To reduce the computational dimensionality of subsequent algorithms while preserving brightness information, the system uses a weighted average method to convert the three-channel color data into single-channel grayscale data. (Calculation of grayscale image) In pixel coordinates pixel grayscale value at The formula is as follows:

[0035] ;

[0036] in, Represents a grayscale image In pixel coordinates The pixel grayscale value at that location; Represents the original color image in pixel coordinates The red channel component value at that location; Represents the original color image in pixel coordinates The green channel component value at that location; Represents the original color image in pixel coordinates The blue channel component value at that location; , and These are the weighting coefficients for the corresponding color channels.

[0037] Obtaining grayscale images Next, the system performs the first stage of noise reduction, employing a Gaussian filtering algorithm to suppress smoothing noise generated by uniformly distributed suspended coal dust in the well. The system defines the Gaussian convolution kernel. as follows:

[0038] ;

[0039] in, Pi is a constant. This represents the standard deviation of the Gaussian distribution, with a value range of 0.6 to 1.0. The base of the natural logarithm; This represents the horizontal offset of the Gaussian convolution kernel in the local coordinate system. This represents the vertical offset of the Gaussian convolution kernel in the local coordinate system.

[0040] The system will convert grayscale images With Gaussian convolution kernel Perform convolution operations to generate an intermediate image after Gaussian filtering. The intermediate image. In pixel coordinates pixel grayscale value at It is a smoothed value obtained through convolution calculation.

[0041] Subsequently, the system processed the intermediate image. A second-level denoising process is performed, employing a median filtering algorithm to eliminate discrete impulse noise generated by large coal dust particles or sensor circuitry. The system defines a pixel coordinate system. centered filtering window The size of the filter window is set to ,in The value can be 3 or 5.

[0042] Output base denoised image In pixel coordinates Output pixel grayscale value The calculation formula is:

[0043] ;

[0044] in, This represents the base denoised image after denoising processing. In pixel coordinates Output pixel grayscale value at; The median operator is used to sort the values ​​in the set and take the middle value. Represented in pixel coordinates The filtering window centered on; Indicates the middle image Offset relative to the center point The pixel grayscale value of the location; This represents the horizontal index variable within the window; This represents the vertical index variable within the window; Indicates the coverage filter window The set of all pixel grayscale values.

[0045] After the above serial processing, the output is the basic denoised image. The color space conversion and basic noise filtering were completed, serving as input data for subsequent targeted water mist removal steps.

[0046] After performing basic processing, the system obtains a basic denoised image. Although basic denoising removed discrete noise, non-uniformly distributed water mist interference still remained in the image. Considering the physical characteristics of well water mist appearing as mid-frequency texture fluctuations while rock powder areas exhibit distinct structural edges in the image, the system employs a guided filtering algorithm to improve the basic denoising of the image. A second processing step is then performed. This step aims to smooth the water mist texture while preserving the gradient characteristics of the rock powder edges.

[0047] The system will denoise the base image. Simultaneously configured as a guide filter, the guiding image and input image The system defines the guided filter output image as... It also defines a pixel index A central local square window The local square window covers the area with The set of pixels in the neighborhood centered on the element. Assume a local square window... Internal, guided filter output image With guide image There exists a local linear relationship between them.

[0048] Assuming a local square window Internal, grayscale value of the guided filter With guide image There exists a local linear relationship. For local square windows... any pixel within Its linear model is defined as follows:

[0049] ;

[0050] in, Indicates the output image of the guided filter. At pixel The grayscale value at that location; This refers to the guide image (i.e., the base denoised image). ) at pixel The grayscale value at that location; Represents a partial square window Linear multiplication coefficients within; Represents a partial square window Linear addition intercept within; This indicates that the linear relationship applies to local square windows. All pixels included .

[0051] To find the optimal linear multiplication coefficients and linear addition intercept The system constructs a cost function that includes data fidelity terms and regularization terms. Output image by minimizing guided filtering. With input image The coefficients are determined by the differences between them. The formula for calculating the cost function is as follows:

[0052] ;

[0053] in, This represents the input image (i.e., the base denoised image). ) at pixel The grayscale value at that location; This represents the regularization parameter, used to control the smoothness of the filter. Its value is set to a fixed constant based on the range of downhole water mist concentration. Indicates a local square window All pixels Perform a summation operation; This represents the fitting error term, used to ensure that the output structure closely approximates the input structure; This represents a regularization term used to prevent linear multiplication coefficients. Too large.

[0054] Based on the principle of linear regression, the system solves the above cost function using the least squares method to obtain a local square window. Corresponding linear multiplication coefficients and linear addition intercept The calculation formula is as follows:

[0055] ; ;

[0056] in, Indicates a guide image In a local square window The grayscale variance within; Indicates a guide image In a local square window The mean gray level within the image. According to this formula, when the gray level variance of a local region of the image... Much larger Time (corresponding to high-frequency areas such as the edge of rock powder), Approaching 1, the guided filter output image Preserve edge features; when grayscale variance much smaller Time (corresponding to low-frequency regions such as flat water mist). Approaching 0, the guided filter output image It approaches the local mean, thus achieving smooth dehazing.

[0057] Because any pixel in the image It will be contained in multiple local square windows covering that point. In the middle, the system calculates all covered pixels. The average value of the coefficients corresponding to the window is used as the final linear coefficient for that pixel. Guided filtering output image. At pixel The final grayscale value at the location The calculation formula is as follows:

[0058] ; ; ;

[0059] in, Represents pixels The average multiplication coefficient at the location; Represents pixels The average intercept coefficient at; This indicates the base denoised image at the pixel level. Output pixel grayscale value at; This represents a constant indicating the total number of pixels within the window; This indicates that for all pixels that satisfy the condition " Located in a partial square window Pixel index of the condition "inside" Perform summation.

[0060] Through the above processing, the system generates a guided filter output image. The guided filter output image smooths out the mid-frequency grayscale fluctuations caused by water mist while preserving the boundary gradient information of the rock powder area, providing a high signal-to-noise ratio data foundation for subsequent feature extraction.

[0061] After performing targeted water mist removal, the system obtains a guided filter output image. To further enhance the contrast between the rock powder features and the background coal face in the image, and to eliminate local shadows caused by uneven underground lighting, the system performs guided filtering on the output image. The visual enhancement process is performed in two stages, consisting of Limit Contrast Adaptive Histogram Equalization (CLAHE) and Linear Gray Scale Stretching.

[0062] The system first filters the output image using a guided filter. Perform CLAHE processing. The system will then output the guided filtered image. Divided into There are three non-overlapping rectangular sub-regions, among which... This indicates the number of rectangular sub-regions divided in the vertical direction (row direction) of the image. This represents the number of rectangular sub-regions divided along the horizontal (column) direction of the image. For each rectangular sub-region, the system calculates its gray-level histogram. Let the gray-level range within the rectangular sub-region be... ,in This represents the total number of gray levels (usually 256). Definition The grayscale value of this sub-region The number of pixels. To prevent noise amplification, the system introduces a clipping threshold. Restrict the histogram.

[0063] The system calculates the corrected histogram. If the number of pixels at a certain gray level Exceeding the shear threshold The excess portion is then cropped and accumulated into the total excess pixels. The system will Distribute evenly across all gray levels. Corrected histogram. The calculation logic is as follows:

[0064] like ,but ;like ,but The final histogram after allocation. for:

[0065] ;

[0066] in, This represents the transient histogram after shearing. This represents the total number of excess pixels that were cropped, calculated using the following formula: ; This represents the total number of gray levels.

[0067] Based on the corrected final histogram The system calculates the cumulative distribution function (CDF) for each sub-region. For any pixel in the image, the system performs bilinear interpolation transformation based on its location and the CDFs of its four adjacent sub-regions to obtain the equalized intermediate image. This process enhances the contrast of local details while suppressing noise amplification in flat areas.

[0068] Subsequently, the system processed the intermediate image. Linear grayscale stretching is performed to fully utilize the display's dynamic range and further amplify the difference between the rock powder (high grayscale) and the background (low grayscale). The system statistically analyzes intermediate images. The global grayscale distribution is used to determine the lower threshold of the stretching interval. and upper limit threshold . The value is taken as the 2% quantile of the entire image's grayscale value. The value is taken as the 98th percentile of the grayscale value of the entire image.

[0069] Final enhanced image In pixel coordinates The formula for calculating the pixel grayscale value at a given location is as follows:

[0070] ;

[0071] in, This indicates the final output enhanced image in pixel coordinates. The pixel grayscale value at that location; Indicates the pixel coordinates of the intermediate image The pixel grayscale value at that location; This represents the lower limit threshold of the stretching range; Indicates the upper limit threshold of the stretching range; This represents the maximum gray level of an 8-bit grayscale image; This indicates the conditional judgment logic, meaning that when the inequality condition following it is met, the system uses the corresponding mathematical expression to perform the calculation.

[0072] Through the above steps, the system outputs an enhanced image with high contrast, uniform illumination, and distinct features. This completes all operations in the image preprocessing stage, providing standardized data input for subsequent region of interest construction and feature extraction.

[0073] See attached document Figure 2 In step S200, the system uses computer vision algorithms to automatically locate the precise position of the lower drum of the coal mining machine in the image coordinate system, providing a spatial reference for subsequent delineation of dynamic regions of interest (ROIs) containing rock powder features. Step S200 first performs a target detection task based on a deep convolutional neural network.

[0074] The data processing unit receives the enhanced image. As input data, the system pre-loads a deep convolutional neural network model trained on a labeled dataset of underground coal mining machine drums. This deep convolutional neural network model is configured as a single-stage object detection architecture, designed to achieve end-to-end real-time inference.

[0075] The system will enhance the image. Input deep convolutional neural network model Forward propagation computation is performed. This process maps the image from pixel space to feature space and regresses the target bounding box. The mathematical expression of the inference process is as follows:

[0076] ;

[0077] in, This represents the set tensor of multiple candidate detection boxes output by the network.

[0078] The system for set tensors Perform non-maximum suppression to remove redundant candidate boxes with excessive overlap, and then apply a preset confidence threshold. Filter out the optimal target bounding box of the roller .

[0079] Roller target bounding box Defined as a five-dimensional vector containing location information and confidence information, its mathematical definition is as follows:

[0080] ;

[0081] in, This represents the horizontal x-coordinate of the center point of the roller target bounding box in the image coordinate system. This represents the vertical ordinate of the center point of the roller target bounding box in the image coordinate system. This represents the width of the target bounding box in pixels. This represents the pixel value indicating the height of the target bounding box of the roller; This represents the predicted confidence probability value of the bounding box belonging to the lower drum category of the coal mining machine, and its value ranges from [0,1].

[0082] To ensure the effectiveness of the positioning, the system verifies the validity of the detection results. Only if certain conditions are met... At that time, the system determines that the detection is successful and locks the device. The coordinates of the roller's center. If no roller target meeting the conditions is detected in the current frame (i.e., ... The system will automatically call the bounding box data of the previous frame for Kalman filtering prediction to maintain the continuity of positioning.

[0083] By performing a target detection task based on a deep convolutional neural network, the system accurately obtained the location of the lower drum of the coal mining machine in the current video frame. These parameters will serve as direct input variables for dynamically defining the region of interest.

[0084] Obtaining the target bounding box of the roller After obtaining the location parameters, the system performs dynamic geometric calculation of the ROI region. Since the cutting products (coal blocks or rock particles) of the coal mining machine drum are mainly distributed below and on both sides of the drum under the action of gravity and centrifugal force during the cutting process, and the cutting contact surface is located at the bottom edge of the drum, the system establishes an ROI mapping model relative to the drum position.

[0085] The system obtains the physical resolution parameters of the image acquisition unit and sets the total image width to [value missing]. The total height of the image is Using the center coordinates of the roller and its dimensions The system calculates the starting coordinates and dimensions of the original region of interest. The x-coordinate of the top-left corner of the original region of interest is defined as... The top left vertical coordinate is The area width is The area height is The calculation formula is as follows:

[0086] ; ;

[0087] ; ;

[0088] in, This represents the lateral expansion coefficient of the ROI area relative to the width of the drum, and its value is set to a range of 1.2 to 1.5 to ensure that the ROI area covers the spilled material on both sides of the drum. This represents the longitudinal extension coefficient of the ROI area relative to the roller height, with a value range of 0.3 to 0.5, used to cover the cutting contact strip below the roller; This represents the vertical overlap correction factor, with a value range of 0.1 to 0.2, used to ensure that the upper edge of the ROI area is slightly higher than the bottom edge of the drum, thus including the contact line between the drum and the coal wall.

[0089] Due to the calculated original coordinates And the extended dimensions will exceed the physical boundaries of the image plane. The system performs boundary constraint processing on the ROI region to generate the final effective intercept coordinate range.

[0090] Define the horizontal starting index of the final ROI region in the image coordinate system. Horizontal End Index Vertical starting index and vertical end index The boundary constraint calculation logic is as follows:

[0091] ; ;

[0092] ; ;

[0093] in, This indicates the operation of finding the maximum value. This indicates the minimum value operation; This indicates a floor operation, used to convert floating-point coordinates into integer pixel indices; Indicates the total width of the image; This indicates the total height of the image.

[0094] The system is based on the defined index range and From enhanced images Extract the corresponding pixel matrix from the image to generate the ROI sub-image. The ROI sub-image Most background areas unrelated to the cutting process (such as hydraulic supports and coal mining machine bodies) were removed, and only the core area containing the cutting contact surface and potential rock powder features was retained as input data for subsequent feature extraction steps.

[0095] Generating ROI sub-images Subsequently, the system performs line-of-sight occlusion detection to prevent non-cutting targets such as the coal mining machine's rocker arm and hydraulic support side plates from obstructing the line of sight, or to prevent image data from becoming invalid due to the camera being temporarily blocked by coal slurry. This step quantifies the effectiveness of the image by analyzing the pixel grayscale distribution characteristics within the ROI region.

[0096] The system first counts the ROI sub-images. The number of valid pixels within the range. Define the valid grayscale range. This effective grayscale range represents the typical grayscale range of the cut area (including rock powder and coal face) under normal lighting conditions. The system iterates through each pixel in the ROI sub-image, and if the grayscale value of a pixel falls within this range, it is determined to be a valid pixel.

[0097] The system calculates the effective pixel percentage of the ROI sub-image. The calculation formula is as follows:

[0098] ;

[0099] in, This indicates that the gray values ​​in the ROI sub-image satisfy... The total number of pixels; Indicates the width of the ROI sub-image in pixels; This represents the height of the ROI sub-image in pixels; This represents the total number of pixels within the ROI region.

[0100] The system presets an occlusion detection threshold. The occlusion determination threshold is set to a constant between 0.4 and 0.6. The system will calculate the effective pixel ratio. With occlusion determination threshold Comparison. When When the system determines that the current ROI region is in a state of visual obstruction or invalidity; when At that time, the system determines that the current ROI region is a valid field of view.

[0101] Based on the judgment results, the system executes a compensation strategy to determine the final output region of interest image. The system includes a cache unit to store the previously confirmed valid ROI image. The mathematical expression for the compensation and update logic is as follows:

[0102] ;

[0103] in, This represents the final output region of interest image after detection and compensation, which is then sent to the next processing unit. This represents the original ROI sub-image obtained by cropping the current frame; This indicates the previously confirmed valid ROI image stored in the cache unit; This indicates the conditional judgment logic, meaning that when the inequality condition following it is met, the system adopts the corresponding value.

[0104] If the system determines that the current frame is valid (i.e.) ), in addition to In addition to output, the system will also Write to cache unit to update This information is used in subsequent frames. If the system determines that the current frame is occluded, it directly reuses the image features of the previous frame, utilizing the temporal redundancy of the video stream to fill in visual gaps, thus ensuring the continuity and stability of the input for subsequent feature extraction steps.

[0105] In step S300, the system processes the region of interest image output in step S200. The system performs refined feature extraction. Considering the grayscale difference between underground rock powder and coal wall, and the problem that single threshold segmentation is prone to breakage or noise under uneven lighting conditions, the system adopts a dual-threshold adaptive segmentation algorithm to generate a binarized rock powder feature mask.

[0106] The system first calculates the region of interest image. The global baseline threshold. System statistics. The grayscale histogram was used, and the optimal segmentation threshold was calculated using the Otsu's algorithm (maximum inter-class variance method). The optimal segmentation threshold maximizes the variance between the foreground target class (high grayscale pixels) and the background class (low grayscale pixels) segmented accordingly, thereby determining the overall brightness segmentation benchmark of the image.

[0107] Based on calculations The system constructs a high threshold through linear mapping. and low threshold A high threshold is used to filter out identified strong rock powder feature points, while a low threshold is used to retain weak rock powder feature points connected to the strong feature points. The formula for calculating the threshold is as follows:

[0108] ; ;

[0109] in, This represents the optimal segmentation threshold calculated using the Otsu's method; This represents the high threshold floating coefficient, with a value range of 0.1 to 0.2, used to improve the judgment criteria for strong feature points and reduce false noise; This represents the low threshold scaling factor, with a value range of 0.4 to 0.6, used to preserve pixel information in edge transition areas.

[0110] The system utilizes high thresholds and low threshold For region of interest images Each pixel coordinate Perform preliminary classification and labeling. If the pixel grayscale value is greater than or equal to... The system marks it as a strong feature point; if the pixel gray value is between and Between these values, the system marks them as weak feature points; if the pixel grayscale value is less than... The system marks it as a background point.

[0111] Subsequently, the system performs hysteresis processing based on connected components to determine the final rock powder pixels. The system traverses all weak feature points in the image and checks whether there are pixels marked as strong feature points in the 8-neighborhood (i.e., the 8 surrounding adjacent pixel positions) of the weak feature point.

[0112] If a weak feature point has at least one strong feature point within its 8-neighborhood, or if the weak feature point is connected to a strong feature point via a path formed by other weak feature points, the system upgrades the weak feature point to a rock powder pixel. Conversely, if a weak feature point is isolated and not connected to any strong feature point, the system classifies it as noise and categorizes it as background.

[0113] Based on the above logic, the system generates the final binarized rock powder feature image. For any pixel coordinate within the image The mathematical expression of its binarization result is as follows:

[0114] ;

[0115] in, This represents the output binarized rock powder feature image in pixel coordinates. The value at that location; This indicates that the pixel has been identified as a rock powder feature point; This indicates that the pixel is identified as a background pixel; Indicates a conditional decision logic term; This represents the input region of interest image in pixel coordinates. The grayscale value at that location; This represents the high threshold constant for determining strong features obtained through pre-calculation. This represents the logical OR operator, which means that either the preceding or following condition must be satisfied. This represents a high threshold constructed through a linear mapping; This represents the logical AND operator, which requires both conditions to be met simultaneously. Represents the spatial connectivity state variable, when the current pixel coordinates The state is true when there is a pixel in the 8-neighborhood that has been marked as a strong feature point, or when it is connected to a strong feature point through other weak feature point paths. Words indicating logical transitions, referring to those that do not meet the above conditions. All other cases of the condition.

[0116] Through the above processing, the system effectively extracted continuous rock powder areas, repaired edge fractures caused by insufficient illumination using a hysteresis threshold strategy, and suppressed isolated noise points, providing an accurate binary data foundation for subsequent morphological filtering.

[0117] In generating binary rock powder feature images Afterwards, although the main rock powder areas have been extracted, due to the inhomogeneity of the rock powder surface texture and the random reflection of dust from the well, the binary image inevitably contains small voids (black dots) inside the rock powder areas and discrete isolated noise points (white dots) in the background areas. To ensure the accuracy of subsequent calculations of the rock powder cut-off area, the system... Perform a combination of morphological closing and opening operations to optimize the topological structure of the image.

[0118] The system first defines a structuring element for morphological operations. The structuring element is configured with a radius of A circular flat plate structure, in which The value of is set to a fixed constant (e.g., 3 to 5 pixels) based on the image resolution to match the minimum connected feature size of rock powder particles in the image.

[0119] The system utilizes structural elements Binarized rock powder feature images Perform morphological closing operations to generate intermediate transition images. The closing operation involves expansion followed by corrosion, aiming to fill the tiny cavities within the connected domains of rock powder and connect adjacent fractured rock powder areas, fusing them into a complete patch.

[0120] Subsequently, the system utilizes the same structural elements. For intermediate transition images Perform morphological opening operations to generate the final optimized mask image. The opening operation involves erosion followed by dilation, aiming to eliminate isolated noise points smaller than the structuring element in the background and smooth the edge burrs in the rock powder area, making the boundary more rounded and natural.

[0121] The mathematical operation process of the above morphological topology optimization is expressed as follows:

[0122] ;

[0123] ;

[0124] in, This represents an intermediate transition image that has had its internal holes filled after the closing operation. This represents the original binary image of rock powder features. Represents a predefined structural element; Represents the morphological closing operator; Represents the morphological dilation operator; Represents the morphological erosion operator; This represents the final optimized mask image after the opening operation has filtered out background noise. Represents the morphological opening operator.

[0125] Through the aforementioned sequential morphological processing of closing followed by opening, the system eliminates the interference of holes and noise caused by segmentation threshold sensitivity while maintaining the overall morphology of the rock powder region, resulting in an optimized mask image. It exhibits high connectivity and edge smoothness, accurately characterizing the true distribution range of rock powder within the region of interest.

[0126] Optimize the mask image in the output. Subsequently, the system further extracts the target features of rock powder in the image. To eliminate connected regions that resemble rock powder but are actually interference objects (such as reflections from cables or hydraulic lines), and to quantify the dust generation degree of the current frame, the system performs connected component analysis on the mask image and combines it with the region of interest image. Calculate the combined features of geometric and grayscale dimensions.

[0127] The system first optimizes the mask image. Perform a connected component labeling algorithm to identify all independent white connected regions in the image. Assume a total of [number missing] white connected regions are identified in the image. The nth connected component is defined as the nth connected component. The connected regions are ,in For each connected region The system calculates its geometric characteristic parameters, including the area of ​​the region. and the perimeter of the area .

[0128] To distinguish between clump-like rock powder and linear or irregular noise points left over from morphological operations, the system calculates the compactness characteristics of each connected region. Compactness describes how close a target shape is to a circle, and its calculation formula is as follows:

[0129] ;

[0130] in, Indicates the first The compactness of each connected region, whose value ranges from 1 to 2. ; This represents the constant value of pi.

[0131] The system constructs a feature filtering function, and sets a minimum rock powder area threshold based on the physical characteristics that rock powder clouds typically exhibit a certain area and a relatively full shape. and minimum tightness threshold The system traverses all connected regions and determines their validity. Only a connected region that simultaneously satisfies the conditions of having a sufficiently large area and not being elongated in shape is retained as a valid rock powder region. The definition of the first... Validity markers for connected components The calculation logic is as follows:

[0132] ;

[0133] in, Indicates the first Validity markers for each connected region; Indicates reservation. Indicates removal; This represents the minimum rock powder area threshold, used to remove large particle noise that was not eliminated during morphological calculations. This represents the minimum compactness threshold, used to eliminate linear interference such as pipes and cables. This represents the logical AND operator; Indicates a conditional decision logic term; Words indicating logical transitions, referring to those that do not meet the above conditions. All other cases of the condition.

[0134] Based on the filtered effective connected regions, the system combines the region of interest image. Calculate the temporal slice feature vector of the current video frame. This temporal slice feature vector contains two dimensions: spatial distribution dimension (total rock powder area) and optical reflectance dimension (average brightness of the rock powder area). Total rock powder area and average brightness of rock powder area The calculation formula is as follows:

[0135] ; ;

[0136] in, This indicates that the summation process is performed on all identified connected regions. Represents the region of interest image in pixel coordinates The grayscale value at that location; Indicates that the coordinate point belongs to the first... One connected region; This represents a small constant used to prevent the denominator from being zero.

[0137] The system will calculate the feature groups obtained from the current frame. Associate the current time series The data was stored in the temporal feature queue, completing the transformation from a single-frame static image to spatiotemporal multidimensional feature data, eliminating non-rock powder interference, and providing accurate data support for the subsequent quantitative evaluation of the cutting effect.

[0138] In the spatiotemporal multidimensional feature calculation and filtering step, the system has already calculated and output the time series corresponding to the current video frame. Total rock powder area and average brightness of rock powder area Considering the complexity of the underground coal mining environment, the feature data of a single frame is easily affected by instantaneous airflow disturbances or equipment vibrations, resulting in abrupt changes that cannot be directly used to determine the continuous cutting state. Therefore, a temporal sliding window mechanism is introduced into the state feature vector construction step to statistically aggregate features from multiple frames and construct a state feature vector that reflects the stability of the cutting behavior within the current time period.

[0139] The system establishes a fixed-length first-in-first-out (FIFO) feature queue in memory. The time span of the sliding window is defined to include... Frame image, in which This is a preset integer constant, representing the time granularity at which the system performs state evaluation. For the time series of the current video frame... The system retrieves the data stored in the queue from... arrive A series of basic feature data.

[0140] The system calculates the average area of ​​rock powder within the sliding window. and average brightness This quantifies the average dust generation intensity and average reflectivity within the current time period. Simultaneously, to assess the stability of the cutting process, the system calculates the standard deviation of the rock powder area. It is used to characterize the degree of fluctuation in dust production.

[0141] The formulas for calculating the above time-series statistical characteristics are as follows:

[0142] ; ;

[0143] ;

[0144] in, This represents the time sequence of the current video frame; This represents a constant indicating the number of frames contained in the sliding window; This represents the backtracking index variable within the sliding window; In time series The corresponding total rock powder area value for a single frame; In time series The corresponding average brightness value of the rock powder area in a single frame; Indicates the current window's internal time series The calculated average area of ​​rock powder; Indicates the current window's internal time series The calculated average brightness value; Indicates the current window's internal time series The calculated standard deviation of the rock powder area; This indicates a summation operation.

[0145] Based on the calculated statistical components, the system constructs the current time series. 3D state feature vector This three-dimensional state feature vector will serve as the direct input for subsequent classifiers to determine the current cutting medium (such as cutting coal seams, cutting rock, or unloaded material). The mathematical definition of the feature vector is as follows:

[0146] ;

[0147] in, In time series The completed three-dimensional state feature column vector; This represents the matrix transpose operation, used to convert a row vector into a column vector.

[0148] Through the state feature vector construction step, the system transforms discrete, fluctuating single-frame image features into feature vectors with temporal statistical significance. Reflects the overall cutting load, This reflects the reflective properties of the material of the object being cut. This reflects the continuous stability of the cutting process; together, these three factors constitute a description of the lower drum of the coal mining machine in a time series. A complete set of information on the instantaneous working status.

[0149] After constructing the three-dimensional state feature vector Subsequently, the system performs dynamic threshold determination based on multi-level logistic regression to map the current cutting state to a specific working condition category. The system presets three standard working condition categories: no-load state (marked as 0), coal seam cutting state (marked as 1), and rock cutting state (marked as 2). To accurately distinguish these three states, the system constructs a hierarchical decision model. This model first determines whether cutting behavior should occur based on dust generation, and then determines the hardness properties of the cutting medium based on optical characteristics and volatility.

[0150] The system first defines the cut-off discriminant function. Using the average area of ​​rock powder in the three-dimensional state feature vector Compared with the preset dust generation threshold under no-load conditions A comparison was made. This is the no-load dust generation threshold. It is a constant set by the system based on the ambient background noise, representing the maximum amount of suspended dust interference allowed around the drum in the non-cutting state.

[0151] Subsequently, the lithology discrimination index was constructed. This lithological discrimination index incorporates the mean brightness value from the state eigenvector. and rock powder area standard deviation Because the water mist mixed with rock powder generated during rock cutting has a high reflectivity (high brightness), and the dust generation fluctuates greatly due to drastic changes in drum load during hard rock cutting (high variance), the system uses a weighted linear combination method to calculate this index. Lithology discrimination index The calculation formula is as follows:

[0152] ;

[0153] in, This represents the mean brightness value in the three-dimensional state feature vector; The weighting coefficient representing the brightness feature ranges from 0.6 to 0.8 and is used to enhance the contribution of highly reflective features to the discrimination result. This represents the standard deviation of rock powder area in the three-dimensional state feature vector; The weighting coefficient, which represents the fluctuation characteristics, ranges from 0.2 to 0.4 and is used to help determine the stability of the cutting process.

[0154] Based on the cutting discrimination logic and lithology discrimination index, the system determines the preliminary cutting state. The system introduces a lithology determination threshold. This lithology threshold is used to define the critical point between pure coal cutting and lithological cutting. Preliminary cutting status. The mathematical expression of the decision logic is as follows:

[0155] ;

[0156] in, This indicates the initial cutoff state at the current moment as output by the system; This indicates an unloaded state, meaning the drum is not in contact with the coal wall or is only rotating idly; This indicates the state of rock cutting, meaning the drum is cutting interbedded rock or top and bottom rock, accompanied by high brightness and high volatility. This indicates the state of coal seam cutting, meaning the drum is cutting the coal body normally. Dust production is significant, but brightness and volatility are lower than the rock cutting standard. This represents the dust generation threshold for determining an unloaded system; This indicates the lithological threshold for classifying rock cutting as rock sectioning. This indicates a logical AND relationship; This indicates the conditional decision logic.

[0157] By performing dynamic threshold determination based on multi-level logistic regression, the system achieves automated and real-time classification of the working status of the lower drum of the coal mining machine. It transforms abstract image feature vectors into intuitive semantic labels of working conditions, which can sensitively identify instantaneous rock-collision behavior or continuous no-load process, providing a direct decision basis for subsequent coal mining machine height adjustment control strategies.

[0158] Output the initial truncation state of the current frame. Subsequently, considering that transient interferences in the underground environment (such as splashing coal blocks obscuring the camera view or airflow disturbances caused by localized gas outbursts) can lead to jumps in single-frame judgment results, the system performs timing consistency verification to filter out random noise. The system adopts a sliding window-based majority voting and state-preserving mechanism to ensure the stability of the signals output to the coal mining machine control system in the time dimension.

[0159] The system establishes a length of The verification queue, in which A preset odd constant (e.g., 5 or 7) is used to store the most recent consecutive... Preliminary frame determination results. For the current time series. The verification queue contains a set .

[0160] The system iterates through the verification queue, counts the frequency of occurrence of the three operating conditions (no load, coal seam cutting, rock cutting), and calculates the confidence probability of each condition within the current window. A state category variable is defined. These correspond to the states of no load, coal seam, and rock, respectively. The confidence probability for each state is given. The calculation formula is as follows:

[0161] ;

[0162] in, In time series Within the verification window, the state category variable The probability of the state that appears; Indicates the length of the verification queue; This represents the backtracking index within the window; In time series The initial judgment result is output; Represent the Kronecker function, when The function value is 1 if the time condition is met, and 0 otherwise; it is used to statistically analyze state categorical variables. The number of times it appears.

[0163] Based on the calculated confidence probability, the system determines the optimal candidate state within the current window. The optimal candidate state satisfies To avoid frequent state transitions, the system introduces a consistency confirmation threshold. (The value typically ranges from 0.6 to 0.8). The system only updates the final output state when the proportion of candidate states exceeds this threshold; otherwise, it maintains the previous output state. Final cut-off state. The update logic is as follows:

[0164] ;

[0165] in, This indicates the final cutting status at the current moment after timing verification, and this signal will be transmitted to the coal mining machine height adjustment control system. This represents the most frequently occurring optimal candidate state within the current verification window; This indicates the preset consistency confirmation threshold; This indicates the final cut state output by the system at the previous moment; This indicates the conditional decision logic; This means that when the confidence level of the optimal candidate state does not reach the threshold, the system determines that it is currently in a state transition period or that there is noise interference, and forcibly maintains the output of the previous state.

[0166] Through the above processing, the system effectively smooths out the classification spikes caused by image noise, ensuring accurate state switching when the cutting conditions change (such as coal seam thinning leading to continuous rock cutting), and maintaining the continuity and stability of the control signal under occasional interference.

[0167] The final cut state, verified by timing, was output. Subsequently, the system generates a height adjustment control command for the lower drum of the coal mining machine based on this status. This step aims to build a closed-loop feedback system that, by adjusting the drum height in real time, ensures that the coal mining machine always remains at the optimal cutting level, thus avoiding cutting the bottom rock (preventing wear on the cutting teeth) and eliminating the no-load phenomenon (improving the recovery rate).

[0168] The system obtains the actual height of the lower drum of the coal mining machine at the current moment. This value is obtained in real time from the rocker arm angle sensor of the coal mining machine. The system has preset adjustment step size parameters for different working conditions: the rock avoidance adjustment step size is defined as... This value is a large positive number, used to quickly raise the drum to detach it from the rock strata when rock cutting is detected; the coal-finding adjustment step size is defined as... This value is a small positive number used to slowly lower the drum to locate the coal seam floor when no load is detected.

[0169] System defines height adjustment increment function The adjustment direction and magnitude for the next control cycle are determined based on the current cutting status. The calculation logic is as follows:

[0170] ;

[0171] in, This represents the state-based height adjustment increment; This represents the final cut-off state, where Corresponding to the state of rock cutting, Corresponding to the no-load state, Corresponds to the normal coal seam cutting state; This indicates increasing the height of the rollers (raising the bottom); This indicates lowering the roller height (underground).

[0172] Based on the calculated adjustment increment, the system generates the altitude for the next time step. To prevent control commands from exceeding the mechanical travel limits of the coal mining machine, the system introduces physical boundary constraint functions. Let the minimum allowable mechanical height of the lower drum be... Maximum mechanical height is .

[0173] The formulas for calculating and constraining the target control height are as follows:

[0174] ;

[0175] ;

[0176] in, Indicates the height of the unconstrained theoretical target; This indicates the actual height feedback value of the roller at the current moment; This is a function to maximize the value, used to ensure that the target height is not lower than the minimum mechanical height. ; This is a function to minimize the target height, used to ensure that the target height does not exceed the maximum mechanical height. ; This indicates the altitude sent to the hydraulic control system of the coal mining machine at the next moment.

[0177] The system will calculate The signal is converted into an electro-hydraulic control signal, driving the rocker arm cylinder of the coal mining machine to move. When the increment is zero, the system maintains the current stratum advancement, realizing an adaptive cutting control strategy of rapid lifting when there is rock, slow lying down when there is no coal, and stable cutting when coal is seen.

[0178] See attached document Figure 3 The coal mining machine's lower drum cutting effect evaluation device mainly includes an image acquisition unit, a region of interest (ROI) processing unit, a rock powder feature refinement extraction unit, a cutting state adaptive evaluation unit, and a feedback control execution unit. These units rely on an explosion-proof industrial control computer platform installed on the coal mining machine body, and achieve data interaction through an internal high-speed bus. Each unit is configured to execute corresponding algorithm logic to achieve automated evaluation and control.

[0179] The image acquisition unit is configured as an intrinsically safe high-speed mining camera, installed at the root of the coal mining machine's rocker arm or on the side wall of the machine body, with its optical axis precisely aligned with the cutting face of the lower drum. This unit is responsible for continuously acquiring video streams containing the drum, coal face, and cutting products at a fixed high frame rate, converting the optical signals into a digital image matrix, and transmitting them to the region of interest processing unit.

[0180] The region of interest (ROI) processing unit is configured to receive raw image data and crop out ROI sub-images containing key truncating features based on the geometric position information of the roller. The region of interest (ROI) processing unit integrates an occlusion detection and repair module, which is programmed to calculate the percentage of effective pixels in the ROI sub-image. and compare it with the preset occlusion determination Comparison. When detected. At that time, the occlusion detection and repair module automatically retrieves the valid ROI image from the previous frame stored in the cache. Replace the current frame; otherwise, update the buffer and output the region of interest image of the current frame. This ensures that the image data input to subsequent stages remains clear at all times.

[0181] The rock powder feature refinement extraction unit is connected to the region of interest processing unit for use in extracting rock powder features. Medium-quantification of rock powder features. This rock powder feature refinement extraction unit first uses a dual-threshold segmentation module to binarize the image using an adaptive threshold determined by the maximum inter-class variance method, generating a binary rock powder feature image. Subsequently, the rock powder feature refinement extraction unit utilizes the morphological operation module, through a pre-set radius structural element... Perform closing and opening operations sequentially on the binary image to output an optimized mask image with improved topology. .

[0182] The rock powder feature refinement extraction unit further includes a feature calculation module, configured to perform connected component analysis on the mask image and utilize a small rock powder area threshold. and minimum tightness threshold Disruptive connected components are removed, and the total rock powder area of ​​the remaining effective region is calculated. and average brightness of rock powder area As a fundamental feature quantity.

[0183] The truncation state adaptive evaluation unit receives basic feature quantities and is composed of a cascaded feature vector construction module, a preliminary judgment module, and a temporal verification module. The feature vector construction module is configured with a length of... A first-in-first-out queue is used to calculate the average area of ​​rock powder within the sliding window. Average brightness and rock powder area standard deviation Generate a three-dimensional state feature vector .

[0184] The initial determination module is configured to receive state feature vectors and perform multi-level logical operations: firstly, by comparing... Dust generation threshold under no-load conditions Distinguish between unloaded and cut-off states; secondly, calculate the lithological discrimination index. and lithological determination threshold The module compares and distinguishes between the cut coal seam and the cut rock state. It outputs the preliminary cutting status at the current moment. .

[0185] The timing verification module is configured to verify the initial truncation state. Perform consistency filtering. This timing verification module maintains a length of... The verification queue is used to calculate the confidence probability of each state appearing in the queue. Only when the probability of the optimal candidate state exceeds the consensus confirmation threshold. Update the output state in real time; otherwise, maintain the state from the previous time step, thus outputting the smoothed final cut state. .

[0186] The feedback control execution unit is communicatively connected to the coal mining machine's height adjustment hydraulic system and is configured to adjust based on the final cutting state. Generate control commands. This feedback control execution unit stores the rock-avoidance adjustment step size. Coal-finding adjustment step size When a rock cutting status signal is received, a positive large-scale adjustment command is generated; when an idle status signal is received, a negative fine-tuning command is generated; when a coal seam cutting signal is received, the command remains at zero. This feedback control execution unit also integrates limit constraint logic to ensure the generated target height is maintained. Always in and Ultimately, the solenoid valve is driven through the D / A conversion interface to achieve closed-loop adaptive adjustment of the roller layer.

Claims

1. An image-based method for evaluating the cutting effect of the lower drum of a coal mining machine, characterized in that, Includes the following steps: S100: The region of interest processing unit performs basic denoising and format conversion processing on the original color image of the current frame output by the image acquisition unit to obtain a basic denoised image. S200: The region of interest processing unit locates the geometric position of the lower drum of the coal mining machine based on the basic denoised image, extracts the region of interest image according to the geometric position, and performs validity maintenance on the region of interest image. S300, the rock powder feature refinement extraction unit performs adaptive threshold segmentation and morphological operations on the region of interest image to extract rock powder features, including the total rock powder area and the average brightness of the rock powder region; S400, the truncated state adaptive evaluation unit performs statistical aggregation on the rock powder features of multiple frames based on the temporal sliding window mechanism, and constructs a three-dimensional state feature vector containing temporal statistical information; S500, the cutting state adaptive evaluation unit determines the final cutting state based on the three-dimensional state feature vector, and the feedback control execution unit generates a height adjustment control command for the lower drum of the coal mining machine based on the final cutting state. In step S400, the step of the truncation state adaptive evaluation unit constructing a three-dimensional state feature vector containing temporal statistical information specifically includes: The mean value of rock powder area and the mean value of brightness are obtained by calculating the arithmetic mean of the total rock powder area and the average brightness of the rock powder region of the N frames of data cached within the sliding window, respectively. The standard deviation of the total rock powder area is obtained by calculating the standard deviation of the rock powder area over time. The mean value of rock powder area, the mean value of brightness and the standard deviation of rock powder area are combined to form the three-dimensional state feature vector. In step S500, the step of the adaptive evaluation unit for the cutting state determining the final cutting state based on the three-dimensional state feature vector specifically includes: The average area of ​​the rock powder is compared with the no-load dust generation threshold. If it is less than the no-load dust generation threshold, it is determined to be in a no-load state. If the value is not less than the no-load dust generation threshold, the lithology discrimination index is calculated by multiplying the mean brightness value by a preset brightness weighting coefficient, multiplying the standard deviation of rock powder area by a preset fluctuation weighting coefficient, and adding the product of the mean brightness value and the brightness weighting coefficient and the product of the standard deviation of rock powder area and the fluctuation weighting coefficient. The lithology discrimination index is then compared with the lithology discrimination threshold. If the index is not less than the lithology discrimination threshold, the state is determined to be rock cutting; otherwise, the state is determined to be coal seam cutting, and the preliminary cutting state is output.

2. The image-based method for evaluating the cutting effect of the lower drum of a coal mining machine according to claim 1, characterized in that, In step S100, the steps of the region of interest processing unit performing the basic denoising and format conversion processing specifically include: Extract the red, green, and blue channel components of the original color image, and convert the original color image into a grayscale image using a weighted average method. The grayscale image is convolved using a Gaussian convolution kernel to suppress and smooth noise, generating an intermediate image. The intermediate image is subjected to median filtering using a filter window of a preset size to eliminate discrete impulse noise, and the basic denoised image is output.

3. The image-based method for evaluating the cutting effect of the lower drum of a coal mining machine according to claim 1, characterized in that, In step S200, the step of the region of interest processing unit locating the geometric position of the lower drum of the coal mining machine and extracting the region of interest image specifically includes: The base denoised image is input into a deep convolutional neural network model for detection to obtain the center coordinates and size parameters of the target bounding box of the roller. Based on the center coordinates and size parameters, and combined with the preset lateral expansion coefficient, longitudinal extension coefficient, and vertical overlap correction coefficient, the original region of interest coordinates are calculated. The original region of interest coordinates are subject to physical boundary constraints, and a sub-image of the ROI is cropped from the base denoised image based on the constrained coordinate range.

4. The image-based method for evaluating the cutting effect of the lower drum of a coal mining machine according to claim 3, characterized in that, In step S200, the step of the region of interest processing unit performing validity maintenance on the region of interest image specifically includes: The number of effective pixels in the ROI sub-image whose gray values ​​fall within the effective gray range is counted, and the effective pixel ratio is calculated by dividing the number of effective pixels by the total number of pixels in the ROI sub-image. The effective pixel percentage is compared with a preset occlusion determination threshold; When the percentage of effective pixels is less than the occlusion determination threshold, it is determined to be a line-of-sight occlusion state, and the previously confirmed effective ROI image stored in the cache unit is called as the output of the region of interest image. When the percentage of effective pixels is not less than the occlusion determination threshold, it is determined to be an effective field of view, the ROI sub-image is output as the region of interest image, and the cache unit is updated using the ROI sub-image.

5. The image-based method for evaluating the cutting effect of the lower drum of a coal mining machine according to claim 1, characterized in that, In step S300, the step of the rock powder feature refinement extraction unit performing the adaptive threshold segmentation specifically includes: The optimal segmentation threshold for the region of interest image is calculated using the maximum inter-class variance method. Based on the optimal segmentation threshold, a high threshold for determining strong feature points and a low threshold for retaining weak feature points are constructed through linear mapping. The pixels of the region of interest image are classified and labeled using the high threshold and the low threshold, and weak feature points connected to strong feature points are identified as rock powder pixels through connected component hysteresis processing, thereby generating a binarized rock powder feature image.

6. The image-based method for evaluating the cutting effect of the lower drum of a coal mining machine according to claim 5, characterized in that, In step S300, the step of the rock powder feature refinement extraction unit performing the morphological operation specifically includes: A morphological closing operation is performed on the binarized rock powder feature image using a structuring element with a preset radius to generate an intermediate transition image that fills the internal cavities. The morphological opening operation is performed on the intermediate transition image using the structuring element to generate an optimized mask image with background noise filtered out. Step S300 also includes using the optimized mask image to guide the calculation of the rock powder features.

7. The image-based method for evaluating the cutting effect of the lower drum of a coal mining machine according to claim 1, characterized in that, In step S300, the step of the rock powder feature refinement extraction unit extracting the rock powder features specifically includes: Perform connected component analysis on the image after the morphological operations to identify independent connected regions; The area of ​​a region is obtained by counting the total number of pixels in each of the connected regions, and the compactness is obtained by calculating the ratio of the area of ​​the region to the square of the perimeter of the connected region. Effective connected regions are screened using minimum rock powder area threshold and minimum compaction threshold; The total rock powder area is obtained by summing the areas of all effective connected regions, and the average brightness of the rock powder area is obtained by calculating the sum of the gray values ​​of the corresponding pixels of all effective connected regions and dividing it by the total number of pixels in the effective connected regions.

8. The image-based method for evaluating the cutting effect of the lower drum of a coal mining machine according to claim 1, characterized in that, In step S500, the process by which the feedback control execution unit generates the height adjustment control command specifically includes: A verification queue is established to record the initial truncation state of multiple consecutive frames. The confidence probability of each state is calculated, and the state with the highest confidence probability and exceeding the consistency confirmation threshold is determined as the final truncation state. The height adjustment increment is determined based on the final cutting state, wherein the rock cutting state corresponds to a positive rock avoidance adjustment step, the no-load state corresponds to a negative coal-seeking adjustment step, and the coal seam cutting state corresponds to zero increment. The height adjustment increment is added to the current actual height of the lower drum of the coal mining machine, and after mechanical stroke constraint, the height adjustment control command is generated.

9. An image-based device for evaluating the cutting effect of a lower drum in a coal mining machine, characterized in that, The image-based method for evaluating the cutting effect of the lower drum of a coal mining machine, as described in any one of claims 1-8, includes: The image acquisition unit is configured to acquire a real-time video stream containing the lower drum of the coal mining machine and the cutting contact surface and output the original color image of the current frame; The region of interest processing unit is configured to perform basic denoising and format conversion processing on the original color image, locate the geometric position of the lower drum of the coal mining machine and extract the region of interest image, and is also configured to detect the validity of the region of interest image and perform image replacement when the line of sight is obstructed. The rock powder feature refinement extraction unit is configured to perform adaptive threshold segmentation and morphological operations on the region of interest image, and calculate the total rock powder area and the average brightness of the rock powder region based on the filtered connected regions; The cutting state adaptive evaluation unit is configured to construct a three-dimensional state feature vector by statistically analyzing the total rock powder area and the average brightness of the rock powder region based on a time-series sliding window mechanism, and to determine the final cutting state based on the three-dimensional state feature vector. The feedback control execution unit is configured to receive the final cutting state and generate a height adjustment control command based on the current actual height of the lower drum of the coal mining machine.

Images