Coal flow multi-mode autonomous sensing monitoring method, system, equipment and medium

Abstract

The invention discloses a coal flow multi-mode autonomous sensing monitoring method, system, equipment and medium, and relates to the technical field of industrial intelligent monitoring and digital data processing, and the method comprises the steps: deploying equipment, carrying out real-time collection, and generating a multi-mode data set; performing preprocessing and feature extraction on the multi-modal data to obtain a joint representation vector; generating an abnormal probability based on cloud edge collaboration; an early warning signal is triggered in combination with the abnormal probability and the grading threshold value; and in combination with an early warning signal, a monitoring strategy is optimized by adjusting dynamic distribution, so that the coal flow sensing and monitoring precision is improved. According to the invention, various abnormalities and potential safety hazards such as deviation, foreign matters, coal piling and the like in operation are found in time, production accidents caused by equipment faults are avoided, and production safety is guaranteed. And meanwhile, statistical analysis is performed on abnormal information such as coal piling, foreign matters, deviation and carrier roller faults in the operation process, so that the safety and the production efficiency in the coal flow transportation process are further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial intelligent monitoring and digital data processing, and particularly to a multi-modal autonomous perception monitoring method, system, device and medium for coal flow. Background Art

[0002] With the continuous development of the coal industry, the coal flow transportation system plays a crucial role in mine production. As the core equipment of the coal flow transportation system, the operation status of the belt conveyor directly affects the smooth progress of the entire production process. However, due to the harsh coal mine environment, equipment aging and other reasons, various abnormal conditions are likely to occur during the operation of the belt conveyor, such as deviation, tearing, slipping, etc. These problems not only affect production efficiency but may also lead to serious safety accidents.

[0003] In order to ensure the normal operation of the coal flow transportation system, improve production efficiency and work safety level, it is necessary to develop a set of intelligent monitoring algorithms for the coal flow transportation scenario. By introducing artificial intelligence technology, real-time monitoring and fault warning of the belt conveyor can be realized, thereby reducing the risk of accidents and improving the reliability and service life of the equipment.

[0004] The single belt is long in distance, and the environmental working conditions are complex. Under the conditions of easy corrosion influencing factors such as large buried depth geothermal heat and high salinity water spray, there are many potential safety hazards in the actual operation of the mine coal flow transportation system. The occurrence of idler failures is relatively frequent, and due to the large base number, if a large number of idlers are damaged and cannot be replaced in time, it will seriously threaten the safe operation of the belt conveyor. The belt is easily worn during operation, and problems such as belt tearing, foreign objects, deviation, and coal stacking are likely to occur during the coal conveying process. If the belt is severely damaged, the whole belt needs to be replaced, which is costly. At present, to ensure the safe operation of the main coal flow system, multiple manual inspections need to be arranged per shift. The labor intensity of the personnel is high, the inspection distance is long, and it is very difficult to conduct a comprehensive and detailed inspection.

[0005] Based on the cloud-edge collaboration and large model AI architecture of learning while using, the present invention develops an intelligent monitoring algorithm for the coal flow transportation scenario, realizes the key technology of developing an AI monitoring model for belt anomalies, forms a set of health guarantee system for the coal flow system, integrates inspection and control with data analysis, realizes the tracking of belt damage conditions, timely warning of dangerous situations and business closed-loop, effectively assists on-site production and belt maintenance work, and is promoted and applied, achieving remarkable economic and social benefits. Summary of the Invention

[0006] In view of the above existing problems, the present invention is proposed.

[0007] Therefore, the problem to be solved by the present invention is: how to solve the problems of frequent faults of belts and rollers in the coal flow transportation system of coal mines, low efficiency of manual inspection, untimely abnormal detection, high equipment maintenance cost, and poor environmental adaptability through technical means such as AI vision detection, multi-modal data fusion, and cloud-edge collaborative architecture, and realize the intelligent, efficient, and safe operation of the coal flow transportation system.

[0008] To solve the above technical problems, the present invention provides the following technical solutions: a multi-modal autonomous perception and monitoring method for coal flow, including deploying devices to collect in real time and generate a multi-modal data set; preprocessing the multi-modal data and extracting features to obtain a joint representation vector; the feature extraction includes using a convolutional neural network to extract belt surface damage texture features and temperature gradient features, and using short-time Fourier transform and Mel frequency cepstral coefficients to extract sound spectrum features; generating an abnormal probability based on cloud-edge collaboration; the abnormal probability includes using cloud-edge collaboration, that is, fusing the detection results obtained by the real-time object detection algorithm at the edge end with the results obtained by the residual network analysis in the cloud to calculate the abnormal probability; combining the abnormal probability with a grading threshold to trigger a warning signal; combining the warning signal to optimize the monitoring strategy by adjusting dynamic distribution to improve the accuracy of coal flow perception and monitoring.

[0009] As a preferred solution of the multi-modal autonomous perception and monitoring method for coal flow of the present invention, wherein: the multi-modal data set includes images, temperature distribution, operating noise, belt speed, coal volume distribution, and idler vibration signals.

[0010] As a preferred solution of the multi-modal autonomous perception and monitoring method for coal flow of the present invention, wherein: the result of the preprocessing is expressed as: , wherein, represents a composite processing operation, represents a transformation of the original data, represents the th type of transformation parameters, represents the weight coefficient of the transformation of the th type of data, represents the number of operations, taking =6, represents an index variable, represents the result of the preprocessing, represents the th type of data, including image data, temperature distribution, operating noise, belt speed, coal volume distribution, and Idler vibration signal

[0011] As a preferred solution of the multi-modal autonomous perception and monitoring method for coal flow described in the present invention, wherein: the joint characterization vector includes a belt surface damage texture feature vector, a temperature gradient feature vector, and a sound spectrum feature vector; the belt surface damage texture feature and the temperature gradient feature are extracted by using a convolutional neural network and expressed as: , , wherein, represents the convolutional neural network parameters, represents the preprocessed image data, represents the extracted belt surface damage texture feature, represents the preprocessed temperature distribution data, represents the extracted temperature gradient feature, represents the convolutional neural network; the sound spectrum feature is extracted by using the short-time Fourier transform and the Mel frequency cepstral coefficients and expressed as: , , wherein, represents the preprocessed running noise data, represents the parameters of the short-time Fourier transform, represents the parameters of the Mel frequency cepstral coefficients, represents the obtained sound spectrum feature, represents the short-time Fourier transform result, represents the short-time Fourier transform, represents the Mel frequency cepstral coefficients.

[0012] As a preferred solution of the multi-modal autonomous perception and monitoring method for coal flow described in the present invention, wherein: the calculation of the abnormal probability is expressed as: , wherein, represents the abnormal probability detected by the real-time object detection algorithm, represents the abnormal probability obtained by the residual network, 、 represent the weighting coefficients, represents the parameters of the real-time object detection algorithm, represents the parameters of the residual network, Denote the anomaly probability; the detection results obtained by the real-time object detection algorithm include deploying a lightweight real-time object detection algorithm model at the edge side to detect abnormal objects in the belt surface image in real time, including foreign objects, large pieces of coal, and deviation, generating object detection results including object category, location, and confidence, and uploading them to the cloud; the results obtained by the residual network analysis include the cloud receiving the detection results uploaded from the edge side and performing joint analysis in combination with thermal imaging and sound data. The joint analysis includes using the residual network to extract temperature gradient features to judge the risk of local overheating; using the residual network to extract sound spectrum features to judge the risk of abnormal wear of idlers; generating the anomaly probability distribution of thermal imaging and sound data.

[0013] As a preferred solution of the multi-modal autonomous perception and monitoring method for coal flow described in the present invention, wherein: the warning signal includes three-level warnings, and alarm information is generated by combining the anomaly probability and the grading threshold; the three-level warnings include high-risk warnings, medium-risk warnings, and low-risk warnings, which are expressed as: , wherein, denotes the high-risk grading threshold, denotes the low-risk grading threshold; when , it indicates high risk, triggers the device linkage control module, automatically adjusts the belt speed, and pushes the real-time picture and the hidden danger location coordinates to the monitoring terminal through the dynamic alarm video wall; when , it indicates medium risk, and pushes the warning information to the monitoring terminal to prompt the operator to check and intervene; when , it indicates low risk, records the warning information for subsequent analysis.

[0014] As a preferred solution of the multi-modal autonomous perception and monitoring method for coal flow described in the present invention, wherein: the dynamic distribution optimization monitoring strategy is expressed as: , wherein, denotes the dynamic distribution optimization monitoring strategy, denotes that in the case of high risk, a high-priority dynamic distribution strategy is triggered, denotes that in the case of medium risk, a medium-priority response is taken, denotes that in the case of low risk, a low-priority response is taken, denotes the parameter adjustment for high-risk warnings, denotes the operating parameters for medium-risk warnings, denotes the operating parameters for low-risk warnings, Represents dynamic distribution parameters; when in a high-risk situation, increase the sampling frequency, prioritize the processing of important data, and perform real-time response; when in a medium-risk situation, optimize the sensor sampling rate and adjust resource allocation; when in a low-risk situation, reduce the system load and optimize data transmission efficiency.

[0015] Another object of the present invention is to provide a multi-modal autonomous perception and monitoring system for coal flow. By integrating functional modules such as data acquisition, feature extraction, anomaly detection, early warning linkage, and model optimization, it realizes the autonomous perception of multi-modal data, real-time anomaly detection, and precise early warning during the coal flow monitoring process, improves the intelligent and automated level of coal flow monitoring, ensures the safe operation of equipment, and reduces the dependence on manual intervention.

[0016] To solve the above technical problems, the present invention provides the following technical solutions: A multi-modal autonomous perception and monitoring system for coal flow, including a data acquisition module, a feature extraction module, an anomaly detection module, an early warning linkage module, and a monitoring optimization module; the data acquisition module is used to deploy equipment for real-time acquisition to generate a multi-modal data set; the feature extraction module is used to preprocess multi-modal data and extract features to obtain a joint characterization vector; the anomaly detection module is used to generate an anomaly probability based on cloud-edge collaboration; the early warning linkage module is used to combine the anomaly probability with a grading threshold to trigger an early warning signal; the monitoring optimization module is used to combine the early warning signal and adjust the dynamic distribution to optimize the monitoring strategy to improve the accuracy of coal flow perception and monitoring.

[0017] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the above-mentioned multi-modal autonomous perception and monitoring method for coal flow.

[0018] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps of the above-mentioned multi-modal autonomous perception and monitoring method for coal flow.

[0019] The beneficial effects of the present invention are as follows: The multi-modal autonomous perception and monitoring method for coal flow provided by the present invention can timely detect various anomalies such as deviation, foreign objects, and coal stacking during operation and potential safety hazards, avoid production accidents caused by equipment failures, and ensure production safety. The developed multi-modal autonomous perception and monitoring system for coal flow integrates computer vision, equipment control, and big data analysis technologies to realize intelligent detection and alarm of belt rollers and belt surfaces, and at the same time statistically analyzes abnormal information such as coal stacking, foreign objects, deviation, and roller failures during operation, realizing AI video speed regulation, video management and analysis, video stream management and analysis, and platform alarm prompts. Description of the Drawings

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 This is a flowchart of a multi-modal autonomous perception and monitoring method for coal flow provided in the first embodiment of the present invention.

[0022] Figure 2 This is a structural diagram of a multi-modal autonomous perception and monitoring system for coal flow provided in the second embodiment of the present invention.

[0023] Figure 3 This is an experimental comparison diagram of a multi-modal autonomous perception and monitoring method for coal flow provided in the third embodiment of the present invention.

[0024] In the figure, 100 is the data acquisition module; 200 is the feature extraction module; 300 is the anomaly detection module; 400 is the early warning linkage module; 500 is the monitoring optimization module. Detailed implementation manners

[0025] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific implementation manners of the present invention in conjunction with the drawings in the specification.

[0026] Many specific details are set forth in the following description to facilitate a thorough understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0027] Embodiment 1 Refer to Figure 1 , which is the first embodiment of the present invention. This embodiment provides a multi-modal autonomous perception and monitoring method for coal flow, including: deploying devices, performing real-time acquisition to generate a multi-modal data set; preprocessing and feature extraction on the multi-modal data to obtain a joint representation vector; generating an anomaly probability based on cloud-edge collaboration; combining the anomaly probability with a grading threshold to trigger an early warning signal; and combining the early warning signal to optimize the monitoring strategy by adjusting dynamic distribution to improve the accuracy of coal flow perception and monitoring.

[0028] S1. Deploy devices, perform real-time acquisition, and generate a multi-modal data set.

[0029] It should be noted that the multi-modal data set includes images, temperature distributions, operating noises, belt speeds, coal volume distributions, and idler vibration signals.

[0030] Furthermore, the generation of the multi-modal dataset relies on traditional sensor data acquisition, combined with computer vision and intelligent algorithms, to ensure that subtle anomalies can be accurately captured. For example, image data can help detect surface damage or foreign objects on the coal belt, while temperature distribution data can provide early signals of equipment overheating or local failures.

[0031] Even further, the combination of multi-dimensional data such as temperature distribution, coal volume distribution, and idler vibration signals not only improves the accuracy of the data but also enables the identification of potential risk points through intelligent analysis algorithms. For example, changes in the idler vibration signal may indicate idler failure, and combining with other data can help predict the occurrence of failures in advance, thereby reducing unexpected downtimes and equipment damage.

[0032] S2. Preprocess the multi-modal data and extract features to obtain a joint representation vector.

[0033] It should be noted that for data preprocessing, through composite processing operations, features of different modalities are extracted and fused, and then a joint representation vector is constructed. The result of the preprocessing is expressed as: , where represents the composite processing operation, represents the transformation of the original data, represents the th type of data transformation parameter, represents the th weight coefficient of the transformation of the data, represents the number of operations, taking = 6, represents the index variable, represents the result of the preprocessing, represents the th type of data, including image data, temperature distribution, operating noise, belt running speed, coal volume distribution, and idler vibration signal.

[0034] Further, the joint representation vector includes a surface damage texture feature vector, a temperature gradient feature vector, and a sound spectrum feature vector; the feature extraction steps include using a Convolutional Neural Network (CNN) to extract the texture features and temperature gradient features of the surface damage, and using the Short-Time Fourier Transform (STFT) and Mel-Frequency Cepstral Coefficients (MFCC) to extract the sound spectrum features.

[0035] Among them, using the convolutional neural network to extract the texture features and temperature gradient features of the surface damage is expressed as: , , Among them, represents the convolutional neural network parameters, represents the preprocessed image data, represents the extracted texture features of the surface damage, represents the preprocessed temperature distribution data, represents the extracted temperature gradient features, represents the convolutional neural network.

[0036] Using the short-time Fourier transform and Mel-frequency cepstral coefficients to extract the sound spectrum features is expressed as: , , Among them, represents the preprocessed running noise data, represents the parameters of the short-time Fourier transform, represents the parameters of the Mel-frequency cepstral coefficients, represents the obtained sound spectrum features, represents the result of the short-time Fourier transform, represents the short-time Fourier transform, represents the Mel-frequency cepstral coefficients.

[0037] Furthermore, in the process of preprocessing and feature extraction of multimodal data, after the data undergoes composite processing operations, features of different modalities are effectively extracted and fused to construct a high-dimensional joint representation vector. Specifically, the application of CNN in the extraction of belt surface damage texture features and temperature gradient features involves a refined feature abstraction process. Through hierarchical learning of multiple convolutional layers in the network, minute features of damage texture and temperature changes are revealed. When using CNN for feature extraction, the preprocessed image data and temperature distribution data are respectively input into the network for deep convolution to generate high-level feature representations. The texture features of belt surface damage are extracted through multiple convolutional kernels of CNN, reflecting the local damage situation of the coal flow. The temperature gradient features are obtained by capturing the spatial temperature distribution changes in the network, reflecting the potential overheating risk of the system. In the process of sound spectrum feature extraction, STFT can transform the running noise signal into the frequency domain to reveal potential frequency features.

[0038] S3. Generate an anomaly probability based on cloud-edge collaboration.

[0039] It should be noted that the anomaly probability utilizes cloud-edge collaboration, that is, the detection results obtained by the real-time object detection algorithm (YouOnly Look Once, YOLO) at the edge are fused with the results obtained by the residual network analysis in the cloud to calculate the anomaly probability, expressed as: , where, represents the anomaly probability detected by YOLO, represents the anomaly probability obtained by the residual network, , represents the weighting coefficient, represents the parameters of YOLO, represents the parameters of the residual network, represents the anomaly probability.

[0040] Furthermore, the detection results obtained by YOLO include deploying a lightweight YOLO model at the edge to real-time detect abnormal targets in the belt surface image, including foreign objects, large lumps of coal, and deviation; generating object detection results, including object category, location, and confidence, and uploading them to the cloud.

[0041] The results obtained by the residual network analysis include the cloud receiving the detection results uploaded from the edge and conducting joint analysis in combination with thermal imaging and sound data; including using the residual network to extract temperature gradient features to judge the local overheating risk; using the residual network to extract sound spectrum features to judge the abnormal wear risk of idlers; generating the anomaly probability distribution of thermal imaging and sound data.

[0042] Furthermore, the collaboration between the cloud and the edge significantly enhances the system's real-time response ability. Through precise joint analysis of multi-modal data (such as thermal imaging data and sound signals), the system can detect anomalies more efficiently and improve accuracy. For example, the lightweight YOLO model deployed at the edge can quickly identify potential risks such as large foreign objects, coal blocks, and belt deviation in images, while the residual network in the cloud conducts in-depth analysis on temperature gradient and sound spectrum data to accurately determine whether there are risks of overheating or abnormal wear of the equipment. By fusing these multi-dimensional data, the generated anomaly probability can more precisely reflect the operating state of the equipment, thus supporting more timely and effective early warning and processing decisions. Furthermore, the system provides great flexibility in the multi-modal data fusion process by dynamically adjusting the weight coefficients, and can optimize the priority of specific anomalies according to the specific operating state and environmental situation of the equipment. For example, the system can prioritize the weights of relevant data sources according to specific problems such as overheating or abnormal sound, so as to respond to these emergencies more quickly, ensuring that the system can still maintain efficient monitoring capabilities and react in a timely manner in complex and changing environments.

[0043] S4. Combine the anomaly probability with the grading threshold to trigger a warning signal.

[0044] It should be noted that the warning signal includes three levels of warnings, and alarm information is generated by combining the anomaly probability with the grading threshold.

[0045] Furthermore, the three levels of warnings include high-risk warnings, medium-risk warnings, and low-risk warnings, which are expressed as: , Among them, represents the high-risk grading threshold, represents the low-risk grading threshold.

[0046] Furthermore, when a high-risk warning is triggered, the system will immediately activate the device linkage control module and automatically regulate the device according to the judgment of the anomaly probability and the grading threshold. In the case of high risk, the belt speed will be automatically adjusted to relieve potential equipment overload or abnormal conditions. At the same time, the system will display the real-time monitoring screen through a dynamic alarm video wall, push the coordinates of the hidden danger location to the monitoring terminal in a timely manner, and provide comprehensive visual information for the operator to understand the specific location and scope of influence of the problem.

[0047] When the system identifies medium risks, it will push a series of warning messages to the monitoring terminal according to the preset weight coefficients and thresholds, prompting the operator to conduct inspections and interventions in a timely manner; such medium-risk warnings do not immediately trigger equipment regulation, but the system will provide detailed warning descriptions, including possible risk types, affected equipment parts, comparative analysis of historical data, etc.; the operator will obtain this information for further confirmation; the risk level will be dynamically adjusted based on the operator's feedback, and subsequent decisions will be guided according to the inspection results.

[0048] Among them, dynamically adjusting the risk level includes threshold adjustment, data processing frequency, and monitoring parameters.

[0049] Among them, threshold adjustment includes that if under certain environmental conditions, the abnormal probability values of the equipment (such as temperature, vibration, noise, etc.) are usually on the high side, the high-risk threshold will be appropriately increased; for example, if the temperature fluctuates greatly but does not always cause equipment failures, the high-risk threshold can be increased to avoid over-triggering high-risk alarms; if in some cases, if the temperature fluctuation is small and may not affect long-term operation, the low-risk threshold can be appropriately reduced at this time so as to detect potential minor problems earlier and make adjustments in a timely manner.

[0050] The data processing frequency includes that when the abnormal probability is high and the system identifies high-risk signals, the data collection frequency and processing frequency should be increased; this is achieved by increasing the sensor sampling rate and accelerating data processing, so as to capture any changes in real time and reduce the response delay; in the low-risk stage, reduce the data collection and processing frequency to avoid wasting computing resources due to frequent processing of unnecessary data.

[0051] The monitoring parameters include increasing the monitoring accuracy or coverage in high-risk situations, adding more monitoring points, parameters, or data sources to ensure that problems can be captured at the earliest stage; for example, adding more temperature sensors and using more sensitive vibration sensors for precise monitoring; in low-risk situations, the number of monitoring points or the monitoring scope can be reduced, unnecessary monitoring pressure can be reduced, and the accuracy requirements for monitoring parameters can be lowered.

[0052] In low-risk situations, the system will record the warning information to ensure that the data is properly archived, providing a basis for subsequent analysis and decision-making; low-risk warnings do not require immediate intervention measures, but the system will generate a detailed historical data report through the intelligent analysis platform for subsequent system evaluation, performance optimization, or strategy adjustment; at this stage, the operator views the historical trends, monitoring data, and prediction results to provide data support for future monitoring and risk control.

[0053] S5. Improve the detection accuracy through dynamic distribution.

[0054] It should be noted that the dynamic distribution and optimization of the monitoring strategy are expressed as: , Among them, represents the dynamic distribution optimization monitoring strategy, represents that in high-risk situations, a high-priority dynamic distribution strategy is triggered, represents that in medium-risk situations, a medium-priority response is taken, represents that in low-risk situations, a low-priority response is taken, represents the parameter adjustment for high-risk warnings, represents the operating parameters for medium-risk warnings, represents the operating parameters for low-risk warnings, represents the dynamic distribution parameters.

[0055] Furthermore, when the system detects a high-risk state, the system will automatically activate a high-priority dynamic distribution strategy. This strategy ensures the real-time collection of each key parameter by increasing the data sampling frequency, especially for important indicators such as temperature, pressure, and vibration, which are monitored more frequently. During this process, high-risk data is given priority for processing, and dynamic responses are made based on real-time data. When an anomaly occurs, the resource allocation is automatically adjusted, and more computing power is mobilized for real-time analysis to quickly locate the problem and execute preventive measures. For example, when it is detected that the device temperature rises sharply, the system will automatically increase the sampling frequency of the temperature sensor and perform immediate analysis on the collected data to ensure timely response to potential device failures.

[0056] In a medium-risk state, the system will enter a medium-priority response mode. At this time, the system will optimize the sampling rate of the sensors, reduce the system load while ensuring data quality, and dynamically adjust the resource allocation according to real-time feedback. For example, data collection devices with higher processing loads are preferentially scheduled to ensure that important data is collected and processed first. During this stage, the operator will receive optimized data pushes and can decide whether to perform manual intervention or continue to observe the system status based on this information. The system automatically pushes risk analysis reports and warning messages to remind the operator which devices or links have potential risks and help the operator make timely decisions.

[0057] In a low-risk state, the system reduces unnecessary resource consumption through a low-priority response mode to ensure the efficient operation of the system without excessive resource waste. The load is automatically reduced, the data transmission efficiency is optimized, the network burden is reduced, and the rate and accuracy of data transmission are improved. The data collection frequency will also be dynamically reduced according to requirements to improve the overall operation efficiency of the system.

[0058] The above is a schematic solution of a multi-modal autonomous perception and monitoring method for coal flow in this embodiment. It should be noted that the technical solution of the system of the multi-modal autonomous perception and monitoring method for coal flow belongs to the same inventive concept as the technical solution of the above-mentioned multi-modal autonomous perception and monitoring method for coal flow. For the details not described in detail in the technical solution of the multi-modal autonomous perception and monitoring system for coal flow in this embodiment, reference can be made to the description of the technical solution of the above-mentioned multi-modal autonomous perception and monitoring method for coal flow.

[0059] Embodiment 2 Refer to Figure 2 , which is the second embodiment of the present invention. Different from the previous embodiment, it provides a multi-modal autonomous perception and monitoring system for coal flow, including: a data acquisition module 100, a feature extraction module 200, an anomaly detection module 300, a warning linkage module 400, and a monitoring optimization module 500.

[0060] The data acquisition module 100 is used to deploy devices for real-time acquisition to generate a multi-modal data set.

[0061] The feature extraction module 200 is used to preprocess the multi-modal data and extract features to obtain a joint representation vector.

[0062] The anomaly detection module 300 is used to generate an anomaly probability based on cloud-edge collaboration.

[0063] The warning linkage module 400 is used to combine the anomaly probability with a grading threshold to trigger a warning signal.

[0064] The monitoring optimization module 500 is used to combine the warning signal to optimize the monitoring strategy by adjusting dynamic distribution, so as to improve the accuracy of coal flow perception and monitoring.

[0065] This embodiment also provides a computing device applicable to the situation of a multi-modal autonomous perception and monitoring method for coal flow, including: A memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement a multi-modal autonomous perception and monitoring method for coal flow as proposed in the above embodiment.

[0066] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements a multi-modal autonomous perception and monitoring method for coal flow as proposed in the above embodiment.

[0067] The storage medium proposed in this embodiment and the multi-modal autonomous perception and monitoring method for coal flow proposed in the above embodiment belong to the same inventive concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0068] If the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs, Read-Only Memories), random access memories (RAMs, Random Access Memories), magnetic disks, or optical discs.

[0069] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0070] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection part with one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, a computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or otherwise processing it as appropriate, and then storing it in a computer memory.

[0071] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0072] Example 3 Reference Figure 3 , the third embodiment of the present invention provides a multi-modal autonomous perception and monitoring method for coal flow. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0073] Experimental steps: Set up the experimental group: Optimize the monitoring and the federated learning framework through dynamic distribution, that is, the method of the present invention, so that the coal flow transportation system can achieve high-precision anomaly detection in a complex and changeable environment.

[0074] Set up the control group: Compare the differences between the traditional monitoring method and the system based on the federated learning framework in terms of anomaly detection accuracy, response time, etc.

[0075] Collect experimental data, as shown in Table 1.

[0076] Table 1 Experimental data , Experimental evaluation: Data collection and preprocessing: Adopt the same data collection method for the experimental group and the control group to ensure the accuracy of environmental change data (such as temperature, humidity, etc.).

[0077] Feature extraction and training: Use deep learning algorithms to extract multi-modal data features in the experimental group.

[0078] Use traditional threshold methods for feature extraction and anomaly detection in the control group.

[0079] Model training and real-time update: The experimental group uses the federated learning framework for real-time model update and parameter adjustment.

[0080] The control group uses a fixed threshold method for monitoring.

[0081] Evaluation and comparison: By comparing the detection accuracy, response speed, false alarm rate and other indicators of the two groups, the performance differences between the experimental group and the control group were evaluated.

[0082] The analysis of the experimental results is specifically as Figure 3 shown: Comparison of detection accuracy: Experimental group: In complex environments such as high temperature, high humidity, and overload, the experimental group can effectively adapt to environmental changes by dynamically adjusting thresholds and real-time model updates, and finally achieves a detection accuracy of 95%. Control group: Due to the use of fixed models and threshold methods, the detection accuracy of the control group is relatively low under these environmental changes, only 80%.

[0083] Abnormal probability: The experimental group can dynamically adjust the parameters of the model according to changing environmental factors (such as temperature, humidity, etc.), thereby reducing the false alarm rate and improving the accuracy of detection.

[0084] Since the control group fails to adjust the model according to environmental changes, its abnormal detection accuracy is relatively low, resulting in a relatively high false alarm rate.

[0085] Generally speaking, detection accuracy: The experimental group should be significantly better than the control group, especially in complex environments (such as high temperature, high humidity, overload, etc.), because the experimental group can adapt to environmental changes by dynamically adjusting thresholds and real-time model updates. Response time: The experimental group achieves fast response through edge computing and federated learning frameworks, reducing latency, while the control group may have relatively high response latency. False alarm rate: Due to the dynamic optimization of model parameters, the experimental group has a relatively low false alarm rate, while the false alarm rate of the control group is relatively high.

[0086] The dynamic optimization method of the experimental group significantly improves the abnormal detection accuracy of the coal flow transportation system, especially performing well in complex environments. The control group, due to the fixed monitoring method, can effectively adapt to environmental changes, thereby improving the detection accuracy.

[0087] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not restrictive. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A coal flow multi-modal autonomous sensing monitoring method, characterized in that: include, Deploy equipment to collect data in real time and generate multimodal data sets; Preprocess the multimodal data and extract features to obtain a joint representation vector; The feature extraction includes using a convolutional neural network to extract surface damage texture features and temperature gradient features, and using short-time Fourier transform and Mel frequency cepstrum coefficients to extract sound spectrum features; Generate anomaly probability based on cloud-edge collaboration; The anomaly probability includes utilizing cloud-edge collaboration, that is, fusing the detection results obtained by the real-time target detection algorithm at the edge with the results obtained by the residual network analysis at the cloud to calculate the anomaly probability; Combine the abnormal probability with the classification threshold to trigger the early warning signal; Combined with early warning signals, the monitoring strategy is optimized by adjusting dynamic distribution to improve the accuracy of coal flow perception monitoring.

2. A coal flow multi-modal autonomous sensing monitoring method according to claim 1, characterized in that: The multimodal data set includes images, temperature distribution, operating noise, belt speed, coal quantity distribution and roller vibration signals.

3. A coal flow multi-modal autonomous sensing monitoring method as claimed in claim 2, characterized in that: The result of the preprocessing is expressed as, ， in, Represents a compound processing operation, represents the transformation of the original data, Indicates Data Types The transformation parameters, Indicates The weight coefficient of the transformation of the data, Indicates the number of operations, take =6, Represents an index variable, represents the result of preprocessing, Indicates data types, including Image data, Temperature distribution, Operation noise, Belt speed, Coal distribution and Roller vibration signal.

4. A coal flow multi-modal autonomous sensing monitoring method as claimed in claim 3, characterized in that: The joint characterization vector includes a surface damage texture feature vector, a temperature gradient feature vector, and a sound spectrum feature vector; The convolutional neural network is used to extract the surface damage texture features and temperature gradient features, which are expressed as: ， ， in, represents the convolutional neural network parameters, Represents the preprocessed image data, represents the extracted surface damage texture features, represents the temperature distribution data after preprocessing, represents the extracted temperature gradient feature, represents a convolutional neural network; The sound spectrum features extracted using short-time Fourier transform and Mel frequency cepstrum coefficients are expressed as: ， ， in, represents running noise data after preprocessing, represents the parameters of the short-time Fourier transform, Parameters representing the Mel-frequency cepstral coefficients, Represents the obtained sound spectrum characteristics, represents the short-time Fourier transform result, represents the short-time Fourier transform, Represents Mel-frequency cepstral coefficients.

5. A coal flow multi-modal autonomous sensing monitoring method as claimed in claim 4, characterized in that: The calculated abnormal probability is expressed as, ， in, represents the probability of anomalies detected by the real-time target detection algorithm, represents the abnormal probability obtained by the residual network, , represents the weighting coefficient, represents the parameters of the real-time object detection algorithm, represents the parameters of the residual network, represents the probability of abnormality; The detection results obtained by the real-time target detection algorithm include deploying a lightweight real-time target detection algorithm model on the edge to detect abnormal targets in the belt surface image in real time, including foreign objects, large pieces of coal, and deviations, generate target detection results, including target category, location, and confidence, and upload them to the cloud; The results obtained by the residual network analysis include the detection results uploaded by the edge end received by the cloud, and joint analysis is performed in combination with thermal imaging and sound data. The joint analysis includes using the residual network to extract temperature gradient features to determine the risk of local overheating; using the residual network to extract sound spectrum features to determine the risk of abnormal wear of rollers; and generating abnormal probability distribution of thermal imaging and sound data.

6. A coal flow multi-modal autonomous sensing monitoring method as claimed in claim 5, characterized in that: The warning signal includes three levels of warning, which generates alarm information by combining abnormal probability and classification threshold; The three-level warning includes high-risk warning, medium-risk warning and low-risk warning, which are expressed as: ， in, Indicates the high risk classification threshold, represents the low risk classification threshold; when When the alarm is raised, it indicates a high risk, triggering the equipment linkage control module to automatically adjust the belt speed and push the real-time image and hidden danger location coordinates to the monitoring terminal through the dynamic alarm video wall; when When it is detected, it indicates medium risk, and an early warning message is pushed to the monitoring terminal to prompt the operator to conduct inspection and intervention; when When , it indicates low risk and the warning information is recorded for subsequent analysis.

7. A coal flow multi-modal autonomous sensing monitoring method as claimed in claim 6, characterized in that: The dynamic distribution optimization monitoring strategy is expressed as: ， in, Indicates the dynamic distribution optimization monitoring strategy, Indicates that in high-risk situations, a high-priority dynamic distribution strategy is triggered. Indicates that in medium risk situations, a medium priority response is taken. Indicates that in low-risk situations, a low-priority response is taken. Indicates parameter adjustment for high-risk warning. Indicates the operating parameters for medium risk warning. Indicates the operating parameters for low risk warning. Indicates dynamic distribution parameters; When the risk is high, increase the sampling frequency, prioritize important data, and respond in real time; When at medium risk, optimize sensor sampling rate and adjust resource allocation; When the risk is low, the system load is reduced and the data transmission efficiency is optimized.

8. A coal flow multi-modal autonomous sensing monitoring system, using a coal flow multi-modal autonomous sensing monitoring method as claimed in any one of claims 1 to 7, characterized in that: It comprises a data collection module (100), a feature extraction module (200), an anomaly detection module (300), an early warning linkage module (400) and a monitoring optimization module (500); The data acquisition module (100) is used to deploy equipment, perform real-time acquisition, and generate a multi-modal data set; The feature extraction module (200) is used to preprocess the multimodal data and extract features to obtain a joint representation vector; The anomaly detection module (300) is used to generate anomaly probability based on cloud-edge collaboration; The early warning linkage module (400) is used to trigger an early warning signal by combining the abnormal probability and the classification threshold; The monitoring optimization module (500) is used to improve the coal flow perception monitoring accuracy by combining the early warning signal and adjusting the dynamic distribution optimization monitoring strategy.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a coal flow multimodal autonomous perception monitoring method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a coal flow multimodal autonomous perception monitoring method described in any one of claims 1 to 7 are implemented.

Images