Bolt miner remote controller fault image diagnosis system driven by CMC chip

Abstract

The invention relates to the technical field of computer vision and industrial equipment fault diagnosis, in particular to a fault image diagnosis system for a remote controller of a bolter miner driven by a CMC chip. Comprising a data sensing module for synchronously acquiring a CMC chip internal logic state vector and a real-time observation image, and preprocessing to obtain target visual data; the generative reconstruction module maps the logic state vector to a high-dimensional space to generate an ideal feature map; the visual analysis module is used for extracting an observation feature map through a convolutional neural network, generating an interference distribution mask based on frequency domain texture statistics, and identifying an unstructured environmental noise region; the difference analysis module is used for calculating a semantic residual image by combining the ideal feature map, the observation feature map and the interference distribution mask, and generating an image difference analysis result; according to the method, ideal features are generated by using the internal logic state vector of the CMC, and the residual calculation of the coal dust noise area is shielded by the interference mask, so that the false alarm caused by the environmental noise is effectively eliminated, and the closed-loop verification that what you see is what you control is realized.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and industrial equipment fault diagnosis technology, specifically to a fault image diagnosis system for a tunneling and anchoring machine remote control driven by a CMC chip. Background Technology

[0002] Image-based fault diagnosis of roadheader remote controllers refers to the detection and evaluation of the operating status of interactive equipment in underground coal mines based on the acquired panel images. Currently, there are three methods for fault diagnosis of roadheader remote controllers: manual periodic inspection, image comparison based on traditional machine vision, and logic detection based on simple electrical signals.

[0003] However, when performing fault diagnosis based on existing technologies, on the one hand, the identification of unstructured noise in the high dust and low light environment of underground is not accurate enough, which is prone to false alarms; on the other hand, due to the lack of a joint verification mechanism between internal control logic and external visual appearance, it is difficult to effectively distinguish between visual defects caused by environmental interference and actual device faults, which will reduce the accuracy of fault diagnosis. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a fault image diagnosis system for a CMC chip-driven tunneling and anchoring machine remote controller. Specifically, the technical solution of this invention includes:

[0005] The data sensing module is used to synchronously acquire real-time observation images and internal logic state vectors of the target interactive device, and preprocess the real-time observation images to obtain target visual data.

[0006] A generative reconstruction module, connected to the data perception module, is used to receive the internal logical state vector and map the internal logical state vector to a preset high-dimensional feature space, so as to generate an ideal feature map corresponding to the internal logical state vector through a decoding network.

[0007] The visual analysis module is used to encode the target visual data through a convolutional neural network to extract observation feature maps, and to generate an interference distribution mask based on the frequency domain texture statistics of the target visual data, wherein the interference distribution mask is used to identify the regions in the target visual data that are interfered with by unstructured environmental noise.

[0008] The difference analysis module is used to calculate a semantic residual map based on the ideal feature map, the observed feature map, and the interference distribution mask, and to generate an image difference analysis result based on the semantic residual map. The image difference analysis result is used to characterize the visual difference between the target visual data and the ideal feature map.

[0009] Preferably, the difference analysis module includes:

[0010] The weighted calculation unit is used to perform a difference operation on the ideal feature map and the observed feature map to obtain the original difference tensor, and to use the interference distribution mask to perform spatial weighted filtering on the original difference tensor to reduce the difference weight of the unstructured environmental noise interference region.

[0011] The difference quantization unit is used to perform norm calculation on the weighted filtered original difference tensor to obtain the global difference degree, and generate image difference analysis results based on the global difference degree.

[0012] Preferably, the generative refactoring module includes:

[0013] A manifold mapping unit is used to project the discrete internal logic state vectors onto a continuous latent space manifold to obtain latent state encodings;

[0014] The decoding and generation unit is used to read the latent state code through a pre-trained generative adversarial network generator and reconstruct the ideal feature map corresponding to the internal logic state vector, wherein the ideal feature map includes the expected panel display content, indicator light on / off state and button physical displacement features.

[0015] Preferably, the visual analysis module includes:

[0016] The texture separation unit is used to perform wavelet transform on the target visual data and extract the fractal dimension features of the high-frequency components;

[0017] The mask generation unit is used to identify irregular texture regions based on the fractal dimension features and mark the irregular texture regions as high-confidence environmental interference regions to generate the interference distribution mask.

[0018] Preferably, the data sensing module includes:

[0019] A synchronous triggering unit is used to listen to the change edge of the internal logic state vector and trigger an image acquisition command when a state change is detected, so as to ensure that the real-time observation image is time-aligned with the internal logic state vector.

[0020] The illumination compensation unit is used to calculate the ambient illuminance based on the histogram statistical characteristics of the real-time observed image, and to dynamically adjust the exposure gain of the acquisition device based on the ambient illuminance.

[0021] Preferably, the target interactive device is a remote controller for a tunneling and anchoring machine, and the internal logic state vector originates from the register data of the CMC control chip;

[0022] The unstructured environmental noise is caused by coal dust or oil pollution covering the area underground in the coal mine.

[0023] The image difference analysis results are used to analyze the visual performance of the screen display, button status, or indicator light status of the tunneling and anchoring machine remote control.

[0024] Preferably, the difference analysis module is further used for:

[0025] Based on the global difference degree, the original difference tensor is reverse-mapped to locate the difference region on the target visual data;

[0026] Based on the geometric morphological features of the difference regions, the image difference analysis results are classified into regions.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. This system constructs a generative residual diagnostic architecture based on logical priors, utilizes the internal logic state vector of the CMC control chip to generate an ideal feature map, and performs difference analysis between it and the real-time acquired observation feature map under the constraint of interference distribution mask. It can effectively distinguish between visual defects caused by coal dust obstruction and visual defects caused by device failure in the feature space. Thus, while retaining the ability to detect minor faults, it eliminates false alarms caused by unstructured environmental noise in coal mines, and realizes closed-loop verification of visual appearance and control logic.

[0029] 2. This system employs a texture separation mechanism based on wavelet transform and fractal dimension features. It utilizes the physical difference between the natural irregular fractal structure of coal dust and the smooth surface of industrial products to generate an interference distribution mask. This enables the accurate identification of high-confidence environmental interference areas, thereby solving the problem of traditional color segmentation failure caused by the similar color of coal dust and remote control casing in coal mine tunneling faces. It ensures robust segmentation of unstructured noise under complex lighting and color confusion scenarios.

[0030] 3. This system uses hardware synchronization triggering and dynamic illumination compensation mechanisms in the data perception module to monitor changes in logic state to trigger instantaneous image acquisition, and adjusts the exposure gain in real time based on histogram statistical features. This can eliminate dynamic blur caused by the asynchronous timing of logic and image, and ensure that the image maintains the optimal dynamic range under severe downhole illumination fluctuations, thereby providing high-quality input data with strict spatiotemporal alignment and stable clarity for the neural network.

[0031] 4. This system uses the inverse mapping positioning and geometric morphology classification functions of the difference analysis module to back-project the high-dimensional residual tensor onto the original image coordinate system to generate a visual heat map. Combined with the eccentricity and compactness characteristics of the connected domain, the system classifies faults into regions, which can intuitively locate and identify specific fault types such as screen cracks, burnt-out indicator lights, or mechanically stuck buttons. This improves fault diagnosis from simple alarm prompts to visual and precise positioning, effectively guiding targeted repairs and reducing downtime. Attached Figure Description

[0032] The present invention will be further explained below with reference to the accompanying drawings and embodiments:

[0033] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0035] First embodiment:

[0036] Please see Figure 1 A fault image diagnostic system for a CMC chip-driven tunneling and anchoring machine remote control, the system includes:

[0037] The data perception module is used to synchronously acquire real-time observation images and internal logic state vectors of the target interactive device, and preprocess the real-time observation images to obtain target visual data.

[0038] The generative reconstruction module is connected to the data perception module. It is used to receive the internal logical state vector and map the internal logical state vector to a preset high-dimensional feature space so as to generate an ideal feature map corresponding to the internal logical state vector through the decoding network.

[0039] The visual analysis module is used to encode the target visual data through a convolutional neural network to extract the observation feature map, and to generate an interference distribution mask based on the frequency domain texture statistics of the target visual data. The interference distribution mask is used to identify the regions in the target visual data that are interfered with by unstructured environmental noise.

[0040] The difference analysis module is used to calculate the semantic residual map based on the ideal feature map, the observed feature map, and the interference distribution mask, and to generate image difference analysis results based on the semantic residual map. The image difference analysis results are used to characterize the visual difference between the target visual data and the ideal feature map.

[0041] This embodiment details the construction process of a generative residual diagnostic architecture based on logical priors. This architecture aims to solve the problem of interference from unstructured environmental noise in coal mines on the visual inspection of precision instruments. The system constructs a synchronous snapshot of the physical world and the digital logical world through a data perception module. This module does not simply acquire images, but acts as a multimodal synchronizer, reading the register states of the CMC control chip inside the tunneling and anchoring machine remote controller in real time through a hard-wired interface to form an internal logical state vector. This vector contains all the current discrete logic states of the remote control, such as the on / off position of the LED indicator, the numerical code of the digital tube display, and the trigger flag of the button;

[0042] Simultaneously, this module controls an explosion-proof industrial camera to capture real-time images from the remote control panel. The image is raw RGB data without denoising, preserving the characteristics of coal dust, oil stains, and light reflection at the scene. To ensure that the input data matches the tensor dimension and distribution requirements of the convolutional neural network in the subsequent visual analysis module, the system performs standardized preprocessing operations on the real-time observed images to obtain the target visual data: a bilinear interpolation algorithm is used to uniformly scale the raw images of different resolutions to [resolution value missing]. Fixed pixel size; perform photometric normalization, first reducing the image pixel values ​​from... Divide the integer field by Linear mapping to The floating-point field is used to obtain Then according to the formula Perform pixel value remapping, where and These are preset channel mean and standard deviation vectors, which are derived from the statistical values ​​of the ImageNet dataset to eliminate differences in dynamic range of illumination and accelerate neural network convergence.

[0043] The generative reconfiguration module establishes the baseline truth for fault diagnosis. This module receives the internal logic state vector based on a deep generative model. It maps to a predefined high-dimensional feature space to generate an ideal feature map. This feature map is a multi-channel feature tensor that includes the panel's geometry, texture prediction, and brightness distribution prediction of the indicator lights. It should be noted that the panel's geometry is not directly derived from its internal logic state vector. Derived or calculated out of thin air; the decoding network of the generative reconstruction module, during the pre-training phase, has learned the inherent static physical spatial layout and geometric shape of the remote control panel through a large number of real image samples of dust-free panels, and has solidified this prior spatial knowledge into the weight parameters of the network; internal logical state vector This serves only as a conditional guidance signal, instructing the decoding network to dynamically and directionally render the corresponding indicator light on / off states and screen display features based on the extracted basic static geometry.

[0044] The visual analysis module executes the semantic encoding stream and the interference perception stream in parallel. On one hand, it utilizes a lightweight convolutional neural network, such as a ResNet-18 or MobileNetV2 backbone network with the fully connected layers removed, and extracts the output of the Stage-3 layer as features to encode the target visual data and extract observation feature maps representing the current actual physical state. To ensure that the convolutional neural network can extract effective and discriminative panel features, this embodiment clarifies the training source and method of the network: weights pre-trained on the ImageNet dataset are loaded as the initialization parameters of the backbone network to obtain general edge and texture extraction capabilities.

[0045] Transfer learning fine-tuning was performed on a dedicated dataset for anchor blocker remote controllers, which includes normal and faulty panel images under different angles and lighting conditions. During training, triplet loss was used to optimize the feature space, narrowing the feature distance between similar states and widening the feature distance between different states, thereby ensuring the quality of the extracted feature maps. It possesses high semantic discriminativeness. To resolve the ambiguity of collaborative training between modules, a phased training strategy is explicitly defined: In the first phase, the visual parsing module is trained independently until convergence, and its weight parameters are saved. In the second phase, when training the generative reconstruction module, the weights saved in the first phase are loaded, and the visual parsing module is set to a parameter-frozen state, so that it only serves as a fixed feature extractor to calculate the feature alignment loss, without participating in the backpropagation update of the gradient, thereby ensuring the stability of the feature alignment benchmark.

[0046] On the other hand, based on the frequency domain texture statistics, regions in the image that do not belong to the inherent structure of the remote control are identified, and an interference distribution mask is generated. The numerical value in the mask represents the confidence level that the location is covered by unstructured environmental noise; the difference analysis module calculates the consistency between visual and logical representations based on the ideal feature map. Observation feature map and interference distribution mask Calculate the semantic residual map and generate image difference analysis results;

[0047] In this embodiment, in the scenario of a fully mechanized tunneling face with high dust and low illumination in an underground coal mine, the system introduces CMC logic as prior knowledge to generate an ideal feature map. The system can effectively distinguish between visual defects caused by coal dust obstruction and visual defects caused by device failure. Specifically, by using interference distribution masking to shield the residual calculation of noisy areas, the system can perform high-sensitivity fault comparison only in clean areas. This retains the ability to detect minor faults while eliminating false alarms caused by environmental noise such as coal dust coverage, achieving a closed-loop verification effect of "what you see is what you control".

[0048] The difference analysis module includes:

[0049] The weighted calculation unit is used to perform differential operations on the ideal feature map and the observed feature map to obtain the original difference tensor, and to use the interference distribution mask to perform spatial weighted filtering on the original difference tensor to reduce the difference weight of the unstructured environmental noise interference area.

[0050] The difference quantization unit is used to calculate the norm of the weighted and filtered original difference tensor to obtain the global difference degree, and generate image difference analysis results based on the global difference degree.

[0051] This embodiment further refines the quantification logic of the deviation between visual representation and logical intent in the difference analysis module; the weighted calculation unit performs anti-interference difference operation in the feature space, and this unit calculates the ideal feature map. With observation feature map The original difference; considering the interference distribution mask. It is generated based on the original image resolution, while the feature map The convolutional network downsampling results in a smaller spatial size. The system uses adaptive average pooling to mask the image. downsampling to Same spatial dimensions ;

[0052] Here, it is made clear and The numerical determination method is as follows: given that the input image is preprocessed as The pixel dimension is determined by the CNN backbone network used in the visual parsing module, such as ResNet-18, with a total downsampling stride of 16 in Stage-3. and The adaptive average pooling layer is based on this target size. Dynamically calculate the pooling kernel size and step size;

[0053] Using the aligned interference distribution mask Spatially weighted filtering is applied to the original differences to obtain the weighted residual tensor. The specific calculation formula is as follows:

[0054]

[0055] in, : The ideal feature map originates from the generative reconstruction module, and its physical meaning is the expected visual features based on logical inference; : The observed feature map originates from the visual analysis module, and its physical meaning is the actual visual features based on real-time image encoding; The interference distribution mask is derived from the texture analysis results and then downsampled and aligned. Its physical meaning is the probability that a pixel is covered by environmental noise. A matrix of all ones, used to logically reverse a mask; The Hadamard product, in physical terms, performs element-wise weighted operations; the difference quantization unit compresses the high-dimensional residual tensor into a scalar index that can be used for decision-making, and obtains the global difference degree through norm calculation. The specific calculation formula is as follows:

[0056]

[0057] in, : Derived from the spatial dimension setting of the feature map, its physical meaning is the height dimension of the feature map; : Derived from the spatial dimension setting of the feature map, its physical meaning is the width dimension of the feature map; and : These represent the row index and column index of the feature map, respectively; : Derived from the spatial location of the weighted residual tensor The eigenvector at that location; Derived from preset system parameters, its physical meaning is the normalization factor, which is usually taken as the total number of spatial pixels in the feature map. This is explicitly defined here. That is, the product of the feature map height and width, to ensure a strict correspondence between the formula expression and the parameter definition; The L2 norm, in physical terms, is the Euclidean modulus of the eigenvector at a given location.

[0058] In order to determine the global difference The system generates image difference analysis results with clearly defined binary decision logic for normal or faulty conditions. The difference quantization unit incorporates an adaptive threshold determination mechanism based on statistical process control: data is pre-collected under fault-free conditions. Calculate the mean of the global variability of continuous reference data frames. with standard deviation Set dynamic alarm thresholds In real-time diagnosis, if the calculation is done in real time... If the result is positive, the visual abnormality is determined; otherwise, it is determined to be normal, thus realizing automated fault screening based on statistical significance.

[0059] This embodiment introduces inverse mask weights in differential calculations to force the difference weights of high-confidence interference areas to be reduced to zero, thereby realizing an intelligent attention mechanism in the harsh working conditions of underground coal mines. Specifically, even if the visual characteristics of the coal dust-covered area completely deviate from the ideal characteristics, it will not lead to an increase in global difference. The system only focuses on those areas that should be clearly seen and should be correct, but are actually inconsistent, such as LEDs that are not blocked but have abnormal brightness or cracked screens, thereby ensuring the accuracy of fault location.

[0060] The generative refactoring module includes:

[0061] Manifold mapping unit is used to project discrete internal logic state vectors onto a continuous latent space manifold to obtain latent state encoding;

[0062] The decoding and generation unit is used to read the latent state code through a pre-trained generative adversarial network generator and reconstruct an ideal feature map corresponding to the internal logic state vector. The ideal feature map includes the expected panel display content, indicator light on / off status, and button physical displacement features.

[0063] This embodiment further refines the specific mechanism by which the generative reconstruction module processes the feature mapping between discrete signals and continuous images; the manifold mapping unit addresses the problem of logic state sparsity, given the internal logic state vector output by the CMC. Typically composed of discrete register values, such as LED status bits and digital tube encoding, this unit utilizes an embedding layer to embed vectors... Projecting onto a continuous latent space manifold, specifically through a multi-layer fully connected network, high-dimensional sparse discrete vectors are mapped into low-dimensional dense latent state codes. For example, a 128-dimensional vector;

[0064] To enable those skilled in the art to reproduce the mapping process, this embodiment discloses the specific network architecture of the MLP: the network includes an input layer, two hidden layers, and an output layer; assuming the preprocessed input vector has a dimension of... This dimension depends on the number of registers, for example, 64. The first hidden layer contains 256 neurons, using the ReLU activation function; the second hidden layer also contains 256 neurons, using the ReLU activation function; the output layer contains 128 neurons, using the Tanh activation function to limit the output to 1 / 2. The interval is used to obtain the latent state encoding. ;

[0065] Given that pre-trained generative adversarial networks (GANs) for subsequent connections typically expect the input to follow a standard normal distribution or a specific distribution, in order to address the Tanh output range... To address the potential mismatch between the generator's input distribution and the output distribution, this embodiment employs a distribution adaptation strategy: a distribution reparameterization module is introduced after the MLP output layer. If the generator's pre-training is based on a standard normal distribution, scaling or inverse transformation is used to adjust the distribution. Adapt to the Gaussian domain of the generator; or, as a preferred implementation, directly use it during pre-training of the generator. A uniform or truncated normal distribution is used as a noise prior to ensure the MLP output. Strict alignment with the generator input domain in terms of physical and statistical properties prevents distortion of generated features due to distribution incompatibility;

[0066] Prior to this mapping, the system has an internal logical state vector Perform structured preprocessing: preserve Boolean state bits, such as LED switches, as binary scalars; convert enumerated states, such as gear codes, into One-Hot sparse vectors; and normalize continuous values, such as analog readings. The interval is used to concatenate all processed sub-vectors to form the input layer vector of the MLP, in order to eliminate the gradient influence of data with different dimensions and obtain dense latent state encoding. ;

[0067] The decoding-generating unit recovers the spatially meaningful structure from the latent encoding. This unit employs a pre-trained generative adversarial network generator, whose architecture includes linear projection layers and multiple cascaded transposed convolutional layers. A one-dimensional expansion of the fully connected layers reads the 128-dimensional latent state encoding. This is mapped to an intermediate vector containing 8192 elements to match the total number of elements in the subsequent tensor, and this intermediate vector is then reshaped into the initial feature tensor. This completes the geometric transformation from a low-dimensional vector to a three-dimensional tensor, and then, through layer-by-layer upsampling and batch normalization, the data is read... And reconstruct the ideal feature map ;

[0068] In this process, the generator not only reconstructs the static structure of the panel, but also dynamically simulates the expected panel display content, such as the specific icon texture generated according to the screen ID, and the highlight feature response of the corresponding position of the indicator light on / off state. It also simulates the physical displacement features of the buttons according to the button state, such as the shadow or deformation features generated by the button concavity. In addition, in order to ensure the physical consistency of the generated model and solve the alignment problem of the feature space, this embodiment clarifies the training strategy of the generator: a conditional generative adversarial network architecture is adopted, and the training data consists of the logic state vectors and dust-free panel images synchronously collected in the laboratory environment.

[0069] The specific process for constructing the training loss function is as follows: In order to generate ideal feature maps Aligning the observed feature maps extracted by the visual analysis module within the same semantic space, the system displays real cleanroom panel images. The input is fed into a pre-trained and parameter-frozen visual parsing module to extract the baseline ground truth feature map. Construct a combined objective function that includes adversarial loss and feature matching loss. :

[0070]

[0071] The adversarial loss component in the generator network specifically adopts the loss function form defined by least-squares generative adversarial networks:

[0072]

[0073] in, Represents the mathematical expectation. The potential noise vector follows a distribution. , For generator networks, For discriminator networks; used to constrain the authenticity of the distribution of generated features;

[0074] The L1 norm, specifically the Mean Absolute Error (MEO), is used to force the generation of features. Approximating the true value characteristics at the element level Among these factors, the setting of the tradeoff coefficient is crucial for model convergence. In this embodiment, to ensure that the generated feature map maintains the realism of high-frequency textures while strictly aligning with the geometry of the panel, the following parameters are set: ,set up This ratio, as verified experimentally, effectively prevents pattern collapse and accelerates convergence. By minimizing this combined loss, the generator is forced to learn a deterministic mapping relationship between logical states and visual features, rather than random generation. For model training optimization, the Adam optimizer is used for parameter updates, and a learning rate is set. Momentum parameters Furthermore, a discriminator based on the PatchGAN structure is employed to enhance the ability to discriminate local high-frequency texture details.

[0075] This embodiment addresses the combinatorial explosion problem of complex logic states in a tunneling and anchoring machine remote controller by employing manifold mapping technology, ensuring the system's generalization ability to unknown working conditions. Specifically, even when combinations of logic states not seen in the training set occur, the continuity of the latent space manifold ensures that the generator can still generate reasonable ideal feature maps through interpolation. This enables the system to accurately predict the expected visual performance of the remote controller under any logic state, providing a reliable benchmark for subsequent residual analysis.

[0076] The visual analysis module includes:

[0077] The texture separation unit is used to perform wavelet transform on the target visual data and extract the fractal dimension features of the high-frequency components;

[0078] The mask generation unit is used to identify irregular texture regions based on fractal dimension features and mark the irregular texture regions as high-confidence environmental interference areas to generate interference distribution masks.

[0079] This embodiment further refines the unsupervised interference segmentation mechanism based on physical characteristics in the visual analysis module. Based on physical characteristics specifically refers to utilizing the objective morphological difference between the naturally occurring irregular high-frequency fractal structure of the attached material and the regular low-frequency smooth surface of the industrial remote control casing. Unsupervised specifically means that the generation process of the interference distribution mask does not rely on a real dataset with manually pixel-level semantic annotation to train an additional image segmentation neural network. Instead, it directly relies on a preset discrete wavelet transform algorithm to extract the high-frequency subband of the real-time observed image, then uses the difference box dimension algorithm to calculate the fractal dimension within a sliding window, and finally directly substitutes it into a deterministic threshold formula based on sample distribution statistics to adaptively calculate the interference confidence of each pixel, thereby achieving automatic segmentation of unstructured noise. The texture separation unit captures the unique frequency domain features of coal dust. Given that coal dust particles appear as high-frequency, irregular noise points in the image, while the remote control panel appears as low-frequency or regular high-frequency edges, this unit performs target visual data... Discrete wavelet transform is performed, specifically using the Daubechies (db4) wavelet basis for two-level decomposition. The low-frequency approximation coefficients are set to zero, and only the high-frequency subband coefficients HL, LH, HH are used to perform inverse wavelet transform to reconstruct the high-frequency detail image. The reconstructed image is then subjected to Min-Max linear normalization to map its pixel values ​​to the grayscale range of [0, 255] to adapt to the grayscale depth parameters of subsequent fractal calculations.

[0080] Using the difference box dimension algorithm in Within a sliding window of pixels, with a stride of 4, calculate the fractal dimension feature within the local window. The specific calculation steps are as follows: Image sub-blocks are viewed as surfaces in three-dimensional space. Divide the space into scales of The grid; to ensure scale invariance in fractal calculations, the box height is defined as:

[0081]

[0082] in, grayscale depth The width is the window width; for each grid position Search for the maximum grayscale value of pixels within the grid coverage area. and minimum value Calculate the number of boxes required to cover the grayscale fluctuations of the image:

[0083]

[0084] Among them, symbols This function rounds up to the nearest integer, rounding to the nearest integer not less than the value in parentheses; it counts the total number of boxes. And by changing the scale ,like Perform multiple calculations;

[0085] Final fractal dimension For point-to-point The slope of the line obtained by least-squares fitting; because a sliding window with a step size of 4 was used when calculating the fractal dimension, the generated original... The matrix size is approximately equal to that of the original image. To achieve pixel-by-pixel mask generation, the system... The matrix is ​​subjected to bicubic interpolation upsampling to restore its spatial resolution to be consistent with the target visual data. Pixels, thus ensuring each image coordinate They all have corresponding fractal dimension characteristics ;

[0086] The mask generation unit generates an interference distribution mask based on the difference in texture complexity. The specific generation logic is as follows:

[0087]

[0088] in, Derived from the image coordinate system, its physical meaning is the horizontal and vertical coordinate indices of the currently processed pixel. The image, derived from the texture separation unit and upsampled and aligned, is located in coordinates... The local fractal dimension at the location; The texture complexity threshold is derived from statistical analysis of coal dust samples. Physically, it represents the critical fractal dimension that distinguishes regular panels from irregular coal dust. The specific statistical determination method involves pre-collecting a set of clean panel images and a set of coal dust-covered images, and then calculating the average fractal dimension of each set. and The average of the two values ​​is taken as the threshold, i.e. This is to ensure that the threshold is statistically separable; : Derived from preset parameters, its physical meaning is the steepness coefficient of the Sigmoid function, used to control the smoothness of the mask edges;

[0089] To enable those skilled in the art to implement this, specific parameter setting strategies are disclosed herein: due to the characteristics of fractal dimension The numerical fluctuation range is usually narrow, typically located in to Between, if The value is too small, such as less than This will cause the mask edges to be excessively blurred, making it impossible to effectively isolate interference; if The value is too large, such as greater than If the thresholding is not applied, it degenerates into binary thresholding, losing edge details; experimental testing on a typical coal mine underground environment dataset confirms... The preferred value range is In this embodiment, Specifically set as The basis for determining this value is: the deviation between the fractal dimension of the pixel and the threshold. achieve When the function output value is within the range of left and right, it can quickly converge to the left. or This means confidence saturation, which ensures strong suppression of obvious coal dust regions with high fractal dimension while retaining smooth weight transitions at transitional boundaries with small deviations, thus avoiding ringing effects or hard cutting artifacts during image fusion.

[0090] This embodiment utilizes fractal dimension as the core feature to distinguish between coal dust and the panel, significantly improving the segmentation robustness in color confusion scenarios. Specifically, since coal dust and the remote control casing are often both black, traditional color segmentation fails. However, this solution utilizes the physical difference between the natural irregular fractal structure of coal dust and the smooth surface of industrial products to accurately identify areas covered by coal dust of the same color, ensuring the physical interpretability and accuracy of the interference mask and providing a reliable basis for subsequent weighted analysis.

[0091] The data perception module includes:

[0092] The synchronous triggering unit is used to listen for the change edge of the internal logic state vector and trigger the image acquisition command when a state change is detected, so as to ensure that the real-time observed image is time-aligned with the internal logic state vector.

[0093] The illumination compensation unit is used to calculate the ambient illuminance based on the histogram statistical characteristics of the real-time observed image, and dynamically adjust the exposure gain of the acquisition device based on the ambient illuminance.

[0094] This embodiment further refines the hardware coordination mechanism for ensuring spatiotemporal consistency in the data perception module; the synchronization triggering unit aims to eliminate the time lag between the logical state and the visual image, and this unit listens to the internal logical state vector. The system responds to detected state transitions, such as the instant a button is pressed, by immediately triggering a hardware interrupt to snap the camera's shutter, ensuring that the captured image strictly corresponds to the current logical state. The illumination compensation unit addresses severe downhole illumination fluctuations by calculating the histogram statistical characteristics of the image in real time using the formula:

[0095]

[0096] in, Represents the grayscale value of a pixel, with a range of values. to integers, The gray-level histogram statistical function represents the gray-level values ​​of an image. The number of pixels; calculate the ambient illuminance characterization value. This refers to the average grayscale value of the image, and the exposure gain is dynamically adjusted accordingly. The specific adjustment logic is as follows:

[0097] in, Indicates the first Real-time grayscale mean of the frame image; The preset target grayscale mean value is preferred to be [value]. This is used to adjust the exposure to the optimal brightness.

[0098] in, The exposure gain value is derived from the actual exposure gain value acquired during the previous frame's image acquisition. Its dimension is defined as the integer step size of the image sensor register, with the unit being the dimensionless logical step size (step). Its value range is... to This is used as the integral benchmark for the current adjustment; The ambient illuminance characterization value is derived from the real-time calculation of the current frame based on histogram statistics. The current ambient illuminance is derived from real-time calculations based on histogram statistics, and its physical essence is the dimensionless average gray level. The value is derived from a preset optimal imaging brightness reference value, i.e., the target grayscale average value. In this embodiment, Set as In 8-bit grayscale space, it corresponds to approximately The dynamic range midpoint, this value is based on the average reflectivity of black rubber buttons and stainless steel panels in underground coal mines, and is designed to preserve details in dark areas while preventing overexposure of metal reflections. The proportional coefficient, derived from the preset negative feedback control loop, physically represents the adjustment sensitivity. Its dimension is set as [register step size] / [grayscale level], and it is used to account for deviations in the illumination characterization value. Mapped to gain adjustment step size;

[0099] In this embodiment, Adjusted to This parameter is determined using the critical proportionality method to ensure that the system... The system eliminates illumination deviations within a frame without causing oscillations; to prevent the calculated gain value from exceeding the linear range supported by the camera hardware or from numerical overflow, the system incorporates safety clamping logic.

[0100]

[0101] in, This is in accordance with the physical limits specified in the sensor manual; simultaneously, to avoid video flickering caused by sudden changes in illumination in a single frame, a recursive smoothing filter is used to update the actual gain.

[0102]

[0103] in, The preset time smoothing factor, such as ;

[0104] In addition, the system will during initialization Set as To ensure control stability during the cold start phase;

[0105] In this embodiment, a hard synchronization mechanism eliminates dynamic blur and false alarms caused by asynchrony between logic and image in the dynamically changing underground environment, which is crucial for capturing transient faults. Simultaneously, dynamic illumination compensation ensures that the image input to the generator network remains within the optimal dynamic range under extreme lighting conditions such as direct light from a mine lamp or in a dark corner, guaranteeing the stability of feature extraction. It is worth noting that the illumination compensation unit operates based on the continuous video preview stream output by the camera. When the synchronization trigger unit detects a state change, the camera directly uses the converged exposure parameters of the current preview frame to perform instantaneous capture, thereby avoiding the delay caused by the cold start exposure convergence process and ensuring that the acquired transient image has optimal brightness performance.

[0106] The target interactive device is a remote control for a tunneling and anchoring machine, and its internal logic state vector is derived from the register data of the CMC control chip;

[0107] Unstructured environmental noise is caused by coal dust or oil pollution covering underground in coal mines.

[0108] Image difference analysis results are used to analyze the visual performance of the screen display, button status, or indicator light status of the rockburst remote control.

[0109] This embodiment clearly defines the specific application scenarios of the system and the physical boundaries of the fault definition; the target interactive device of the system specifically refers to the remote control of the tunneling and anchoring machine used in the coal mine fully mechanized tunneling face, and the internal logic state vector is directly read from the CMC control chip, such as the memory-mapped register of the ARM Cortex-M series or FPGA core; the unstructured environmental noise processed by the system is defined as physical attachments unique to underground coal mines, specifically including coal dust coverings that appear as high fractal dimension black particle aggregations, and oil stains that appear as irregular patches with specular reflection characteristics; the image difference analysis results are specifically applied to multi-dimensional fault diagnosis, including detecting screen display abnormalities such as screen flickering, missing strokes or system crashes, detecting physical sticking of buttons where the logic display button is not pressed but the visual image display button is in a recessed state, and detecting indicator lights that are burned out or attenuated when the logic is set to 1 but the visual brightness is below the threshold;

[0110] This embodiment solves the problem of diagnosing specific electromechanical coupling faults in special coal mine operation environments by limiting specific application scenarios; especially in the diagnosis of button jamming, this solution uses the conflict between visual physical position and logical electrical signal to judge the fault, which breaks through the limitation of traditional pure circuit detection that cannot detect mechanical jamming but contact disconnection, and improves the maintenance efficiency and safety of the tunneling and anchoring machine remote control.

[0111] The difference analysis module is also used for:

[0112] Based on the global dissimilarity, the original dissimilarity tensor is reverse-mapped to locate the dissimilarity region on the target visual data.

[0113] Based on the geometric morphological features of the difference regions, the image difference analysis results are classified into regions.

[0114] This embodiment further refines the feedback localization and classification functions of the difference analysis module; the system performs reverse mapping localization in response to the global difference degree. If the preset alarm threshold is exceeded, the high-dimensional weighted residual tensor is processed using activation mapping or bilinear interpolation upsampling algorithms. Upsampling back to the original image resolution involves the system calculating the weighted residual tensor along the channel dimension. L2 norm to generate two-dimensional difference intensity map Upsample the 2D image to match the target visual data. With the same spatial resolution, in target visual data The system generates a heat map to accurately locate the areas of difference, i.e., the fault points; the system classifies the fault regions based on the geometric morphological characteristics of the areas of difference.

[0115] The system performs adaptive binarization processing on the heatmap, specifically using the maximum inter-class variance method to calculate the optimal segmentation threshold based on the grayscale histogram of the heatmap. Values ​​greater than in the heatmap The regions are labeled as foreground, and connected components are extracted. The shape descriptor of each connected component is calculated, including the eccentricity. With density The specific formulas for calculating geometric features are as follows:

[0116]

[0117]

[0118] in, These represent the lengths of the major and minor axes of the ellipse fitted to the connected region, respectively. Specifically, this is achieved by constructing a covariance matrix through the second-order central moments of the connected region. The eigenvalues ​​of this matrix correspond to the squares of the ellipse axis lengths, thus allowing us to solve for the... and The numerical value; in order to enable those skilled in the art to reproduce the solution process, this embodiment discloses specific calculation steps: traversing the set of all pixels within the connected component. Calculate the centroid coordinates:

[0119]

[0120] Calculate the second central moment:

[0121]

[0122] Build Covariance matrix:

[0123] in, Calculate the eigenvalues ​​of this matrix:

[0124]

[0125] Calculate the major and minor axes based on the geometric relationship between eigenvalues ​​and the lengths of the ellipse axes: , here is set ; Let be the pixel area of ​​the connected region. Let be the pixel perimeter of the connected component; before performing location-based region classification, given that the difference regions are extracted from a preprocessed uniform space, for example... The actual physical location of the components is defined in the coordinate system of the original high-resolution image, while the actual physical location of the components is defined in the original high-resolution image coordinate system. To ensure the accuracy of spatial determination, the system performs a coordinate inverse mapping operation: reading the original image acquisition resolution... And according to the scaling ratio The centroid coordinates of the connected components in the different regions And mapping the bounding box coordinates back to the original image coordinate system, i.e. After coordinate alignment is completed, the system calls the component layout registry pre-stored in non-volatile memory. This registry records the original pixel coordinate range of each key functional component on the remote control panel, such as the screen, indicator lights, and button array, under a standard viewing angle. This coordinate system was obtained by manually annotating the standard template image during the installation phase;

[0126] The judgment thresholds used here are all determined based on the statistical distribution of a large number of historical fault samples: [Pre-collected data / data / etc.] For example, for image samples of various known faults, statistically analyze the probability density distribution of their morphological feature parameters, and select... The boundaries of the confidence interval are used as the decision threshold to ensure the robustness of the classification logic;

[0127] Specifically, in response to the eccentricity of the differential region Higher, for example Furthermore, this threshold is derived from statistical data. The screen crack sample eccentricity is located at It appears as a thin, elongated line with its centroid coordinates falling within the screen's ROI area, and is classified as a screen crack; responding to the density of different areas Higher, for example The value is close to 1.0 for a circle, and this threshold is derived from the fact that the density of the indicator light spot is usually greater than that of a circle. If the overlap between the circular circumscribed rectangle (AABB) and the indicator light's preset ROI rectangle is greater than 0.5, it is classified as an indicator light malfunction; if the difference area is large-area diffuse and located in a non-functional area, it is classified as housing wear.

[0128] To fully cover the fault types in the embodiment, the system also performs the following classification logic: in response to the geometric centroid of the difference region falling within the preset button coordinate ROI, and the area of ​​the connected region... The deviation from the standard button area is less than This value is derived from statistics on button mold tolerances and imaging distortion, and is classified as a button mechanical jamming fault; in response to the proportion of the screen ROI covered by the difference area exceeding And eccentricity This feature is derived statistically from the rectangular geometric features of full-screen faults and can cover conventional faults. or A significant portion of the screen area is classified as a screen signal loss, such as a black screen or a distorted screen.

[0129] This embodiment achieves an intelligent leap from fault alarm to fault location through visualized reverse mapping and morphological classification. Specifically, the system can not only indicate the presence of a fault, but also intuitively circle the fault location on the image and identify the fault type. This visualized diagnostic result can directly guide maintenance personnel to perform targeted module replacements, such as replacing only the screen module instead of the entire remote control, thereby greatly reducing maintenance costs and downtime while ensuring production continuity.

[0130] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A fault image diagnosis system for a CMC chip-driven tunneling and anchoring machine remote control, characterized in that, The system includes: The data sensing module is used to synchronously acquire real-time observation images and internal logic state vectors of the target interactive device, and preprocess the real-time observation images to obtain target visual data. A generative reconstruction module, connected to the data perception module, is used to receive the internal logical state vector and map the internal logical state vector to a preset high-dimensional feature space, so as to generate an ideal feature map corresponding to the internal logical state vector through a decoding network. The visual analysis module is used to encode the target visual data through a convolutional neural network to extract observation feature maps, and to generate an interference distribution mask based on the frequency domain texture statistics of the target visual data, wherein the interference distribution mask is used to identify the regions in the target visual data that are interfered with by unstructured environmental noise. The difference analysis module is used to calculate a semantic residual map based on the ideal feature map, the observed feature map, and the interference distribution mask, and to generate an image difference analysis result based on the semantic residual map. The image difference analysis result is used to characterize the visual difference between the target visual data and the ideal feature map.

2. The CMC chip-driven fault image diagnosis system for a tunneling and anchoring machine remote controller according to claim 1, characterized in that, The difference analysis module includes: The weighted calculation unit is used to perform a difference operation on the ideal feature map and the observed feature map to obtain the original difference tensor, and to use the interference distribution mask to perform spatial weighted filtering on the original difference tensor to reduce the difference weight of the unstructured environmental noise interference region. The difference quantization unit is used to perform norm calculation on the weighted filtered original difference tensor to obtain the global difference degree, and generate image difference analysis results based on the global difference degree.

3. The CMC chip-driven fault image diagnosis system for a tunneling and anchoring machine remote controller according to claim 1, characterized in that, The generative reconstruction module includes: A manifold mapping unit is used to project the discrete internal logic state vectors onto a continuous latent space manifold to obtain latent state encodings; The decoding and generation unit is used to read the latent state code through a pre-trained generative adversarial network generator and reconstruct the ideal feature map corresponding to the internal logic state vector, wherein the ideal feature map includes the expected panel display content, indicator light on / off state and button physical displacement features.

4. The CMC chip-driven fault image diagnosis system for a tunneling and anchoring machine remote controller according to claim 1, characterized in that, The visual analysis module includes: The texture separation unit is used to perform wavelet transform on the target visual data and extract the fractal dimension features of the high-frequency components; The mask generation unit is used to identify irregular texture regions based on the fractal dimension features and mark the irregular texture regions as high-confidence environmental interference regions to generate the interference distribution mask.

5. The CMC chip-driven fault image diagnosis system for a tunneling and anchoring machine remote controller according to claim 1, characterized in that, The data sensing module includes: A synchronous triggering unit is used to listen to the change edge of the internal logic state vector and trigger an image acquisition command when a state change is detected, so as to ensure that the real-time observation image is time-aligned with the internal logic state vector. The illumination compensation unit is used to calculate the ambient illuminance based on the histogram statistical characteristics of the real-time observed image, and to dynamically adjust the exposure gain of the acquisition device based on the ambient illuminance.

6. The CMC chip-driven fault image diagnosis system for a tunneling and anchoring machine remote controller according to any one of claims 1 to 5, characterized in that, The target interactive device is a remote control for a tunneling and anchoring machine, and the internal logic state vector is derived from the register data of the CMC control chip; The unstructured environmental noise is caused by coal dust or oil pollution covering the area underground in the coal mine. The image difference analysis results are used to analyze the visual performance of the screen display, button status, or indicator light status of the tunneling and anchoring machine remote control.

7. The CMC chip-driven fault image diagnosis system for a tunneling and anchoring machine remote controller according to claim 2, characterized in that, The difference analysis module is also used for: Based on the global difference degree, the original difference tensor is reverse-mapped to locate the difference region on the target visual data; Based on the geometric morphological features of the difference regions, the image difference analysis results are classified into regions.

Images