Inspection robot monitoring method and system based on digital twinning

Abstract

The invention relates to the field of abnormal state recognition and environment monitoring, in particular to an inspection robot monitoring method and system based on digital twinning. The method comprises the following steps: deploying a plurality of cameras in a similar environment to construct a dynamic environment feature data set; acquiring an environment image video stream acquired by a camera carried by the suspension type inspection robot, and environment and machine body monitoring data acquired by temperature, humidity, gas detection, infrared and event cameras, electric quantity, inertial navigation and other sensors; transmitting the data to a digital twinning remote monitoring system through an MQTT communication protocol, setting a multi-level alarm threshold value, and executing image state analysis; based on an alarm threshold value and an image analysis result, environment data monitoring is carried out, early warning is carried out during overrun or abnormal recognition, and a digital twin interface displays monitoring parameters and an early warning state in real time. According to the method, machine vision and a multi-sensor sensing technology are fused, and adaptive image enhancement and a digital twinborn system are combined, so that the problem of low recognition accuracy in a complex mine environment is solved, and the routing inspection reliability under a low-illumination working condition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of abnormal state recognition and environment monitoring, more particularly to a kind of based on digital twinning inspection robot monitoring method and system.The inspection robot is especially for hanging mine inspection robot. BACKGROUND

[0002] Mine inspection robot is an intelligent mobile platform applied in coal mine field, which can carry different sensor equipment according to the use environment, and monitor and collect data of underground equipment and environment in real time, and timely discover and warn potential problems.The technology can replace manual inspection in complex environment, reduce operation risk and improve inspection efficiency, and has important value for ensuring coal mine safety production and promoting intelligent mine construction.

[0003] At present, mine underground environment and equipment monitoring mainly adopts manual inspection mode.Workers need to go deep into the underground harsh environment, and conduct real-time inspection of various equipment and surrounding environment.This mode has significant drawbacks: the working conditions in the underground are harsh, the inspection task intensity is extremely high and the danger is great, and the whole inspection process is difficult to implement.At the same time, existing inspection robots face serious limitations in actual mine application.Different areas in the underground generally have high dust concentration and low light intensity, resulting in low visibility, which makes it difficult for most inspection robots to balance low cost and high efficiency operation requirements, especially in terms of identification accuracy of environmental state, equipment operation and personnel activity.In addition, conventional inspection robots generally lack remote control capability and real-time monitoring data feedback mechanism, resulting in low uploading efficiency of underground complex environment monitoring data and not intuitive presentation.

[0004] In summary, the current technology has the following deficiencies:

[0005] Manual inspection mode exposes workers to high-risk and high-intensity work environment, and personnel safety and inspection efficiency are difficult to guarantee;

[0006] In mine areas with dust and poor light conditions, the identification accuracy of existing robots for environmental factors, equipment state and personnel position is greatly reduced, which seriously restricts the inspection effect;

[0007] The monitoring system lacks intelligence, lacks multi-source data fusion analysis and visualization warning function, and cannot realize dynamic real-time warning of environmental abnormalities, equipment failures and personnel risks. SUMMARY

[0008] Therefore, the present application provides a kind of based on digital twinning inspection robot monitoring method and system, to solve the core problems of low inspection efficiency and low identification accuracy of environment, equipment and personnel of existing inspection robots in complex mine environment.

[0009] In order to achieve the above object, the present application adopts the following technical solutions:

[0010] A kind of based on digital twin's inspection robot monitoring method and system, it is characterized in that, comprising:

[0011] S1: in the similar environment of robot operation, deploy multiple cameras to build dynamic environment feature dataset;

[0012] S2: obtain the image video stream and multi-sensor environment monitoring data collected by robot-mounted camera;

[0013] S3: data is transmitted to digital twin system by MQTT protocol and threshold value is set;

[0014] S4: early warning and real-time state display are carried out based on threshold value and image analysis result.

[0015] In one specific implementation scheme, the step S1 includes:

[0016] S11: multiple high-resolution cameras are deployed in similar environment;

[0017] S12: Gaussian filter denoising and curvelet transform denoising processing are carried out on collected image;

[0018] S13: image enhancement is carried out using gray histogram equalization and canny operator edge extraction algorithm.

[0019] In one specific implementation scheme, the step S1 further includes:

[0020] S14: improved dictionary learning algorithm is used to extract coal face feature parameters;

[0021] S15: data integration is realized by training abnormal state image sample library through BP neural network.

[0022] In one specific implementation scheme, the multi-sensor in the step S2 includes:

[0023] Temperature sensor, humidity sensor, gas detection sensor, infrared camera, event camera, power sensor, ranging sensor, inertial navigation device and speed measuring device.

[0024] In one specific implementation scheme, the step S3 includes:

[0025] S31: lower computer collects each sensor data by serial port protocol;

[0026] S32: after data is packaged, it is sent to server subject pool by MQTT protocol;

[0027] S33: host computer reads data from server and imports digital twin system.

[0028] In one specific implementation, the step S4 comprises:

[0029] S41: setting differentiated thresholds for environmental abnormalities, equipment failures and personnel risks;

[0030] S42: calling an image sample library for abnormal state identification;

[0031] S43: displaying monitoring parameters and early warning states in real time through a digital twin interface.

[0032] A digital twin-based inspection robot monitoring system, characterized in that it comprises:

[0033] A visual monitoring module: real-time display of equipment operation state;

[0034] An environmental data monitoring module: environmental early warning based on sensor data;

[0035] An equipment and personnel monitoring module: equipment and personnel danger early warning;

[0036] A robot state monitoring module: monitoring of real-time operation parameters of the robot;

[0037] An image data analysis and early warning module: identification of abnormal states in video streams.

[0038] In one specific implementation, the environmental data monitoring module is configured to receive temperature, humidity and gas sensor data;

[0039] The robot state monitoring module is configured to receive power, inertial navigation and speed sensor data.

[0040] In one specific implementation, the image data analysis and early warning module includes a dynamic chart trend analysis unit for performing data state trend analysis.

[0041] In one specific implementation, the equipment and personnel monitoring module uses an infrared camera and an event camera to cooperatively collect images, and identifies equipment and personnel abnormalities through an improved dictionary learning algorithm.

[0042] Compared with the prior art, the digital twin-based inspection robot monitoring method and system has the following beneficial effects:

[0043] 1. Real-time collection of working face environment data through temperature, humidity and gas sensors, combined with multiple alarm thresholds set by the digital twin system, dynamic early warning of environmental risks such as gas overrun and temperature anomalies, improving the response speed of the system.

[0044] 2. Using adaptive image enhancement technology to process images collected in low-light environments, combining improved dictionary learning algorithms with image sample libraries trained by BP neural networks, the accuracy of equipment operation state and personnel position recognition has been improved.

[0045] 3. Through the use of multi-source sensors such as power sensors and inertial navigation devices to monitor the state of the inspection robot, combined with the digital twin interface to display real-time body power, position and speed parameters, ensuring stable operation during inspection.

[0046] 4. Based on the MQTT communication protocol to transmit sensor data to the digital twin system, through dynamic charts to realize environmental trend analysis, synchronously present real-time video stream and warning information, solve the problem of non-intuitive underground monitoring data.

[0047] The inspection robot monitoring method and system based on digital twin, through machine vision to determine the track condition and combine multiple sensors for environmental perception, adaptive image enhancement is performed on the images collected by the camera carried by the robot, and a series of image preprocessing algorithms such as distortion correction are performed on the enhanced images, combined with mine image dataset and other sensors to perceive and identify the environment equipment personnel, the identification result and numerical value are uploaded to the digital twin system, realizing multi-threshold alarm and real-time environment equipment video stream presentation, more effectively ensuring the stable operation of the target inspection robot and efficient identification of equipment personnel, improving the identification rate and reliability in unevenly illuminated and low-light environments. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.

[0049] Figure 1 The overall flowchart of the inspection robot monitoring method and system based on digital twin. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the scope of protection of the present application.

[0051] As shown in Figure 1 The monitoring method of the inspection robot based on digital twinning is applied to mine environment monitoring, and includes the following steps:

[0052] Step S1, dynamic environment feature dataset construction

[0053] In order to accurately adapt the inspection robot to complex work scenes such as mines, it is necessary to first construct a dynamic environment feature dataset that fits the actual working conditions. Specifically, in a simulated scene similar to the actual working environment of the robot (such as a coal mine underground tunnel, a mining working face, which needs to have similar lighting, dust, equipment layout, etc.), multiple high-resolution cameras are deployed. These cameras can cover different angles and areas, capturing environment images in all directions.

[0054] After the images are collected, noise reduction processing is first performed, and Gaussian filtering is used to preliminarily suppress random noise in the images caused by electronic noise, etc. Then, through curvelet transform, more accurate noise reduction is achieved for non-stationary and anisotropic noise caused by dust in the mine environment, and the basic quality of the image is improved. After completing the noise reduction, in order to enhance the recognition degree of key information such as equipment and personnel in the image, the gray histogram equalization algorithm is used to adjust the image gray distribution and improve the contrast, and then the canny operator edge extraction is combined to highlight the features such as equipment outline and personnel shape, making the subsequent feature extraction more efficient.

[0055] On this basis, in order to focus on the core monitoring objects of the mine scene (such as the equipment running state and personnel operation situation of the coal mine working face), an improved dictionary learning algorithm is used to extract feature parameters from the preprocessed images. This algorithm is optimized for specific features such as mine equipment texture and personnel action, and can accurately capture key features such as texture changes caused by loose equipment screws and action patterns of personnel entering dangerous areas in violation of regulations. After extracting the features, a large number of abnormal state image samples (including images of scenes such as equipment failure, personnel violation, and environmental mutation) are trained through a BP neural network to construct an abnormal state image sample library, integrating scattered image data into structured data sets with classification and recognition capabilities, providing a basis for subsequent monitoring and early warning.

[0056] Step S2, multi-source data acquisition

[0057] The inspection robot needs to carry a variety of sensors to collect data of the working environment and its own state. Among them, the environmental data collection relies on temperature sensors, humidity sensors, and gas detection sensors (such as monitoring gas and carbon monoxide concentration) to monitor the environmental temperature, humidity, and harmful gas content in real time. In combination with infrared cameras, it can capture clear images of device heating and personnel outlines in low-light and smog-filled mine environments. Event cameras can quickly respond to sudden changes in brightness and object movement in the environment, supplementing the monitoring blind spots of conventional cameras.

[0058] At the same time, to ensure the stable operation and precise positioning of the robot, the remaining power is monitored by a power sensor, the distance to the surrounding obstacles is determined with the assistance of a distance sensor, and the position, motion speed, and attitude (such as roll angle and pitch angle) of the robot are obtained in real time by an inertial navigation device combined with a speed measurement device. These multi-source data collectively form the "perception layer" of the monitoring system and provide comprehensive materials for subsequent analysis.

[0059] Step S3, data transmission and threshold setting

[0060] The collected multi-source data needs to be transmitted in order to the digital twin system for remote monitoring and intelligent analysis. The lower computer (robot end data collection unit) collects and preliminarily packages the data collected by each sensor through a serial protocol, and then sends the data to the server topic pool with the help of the MQTT protocol. This protocol has the characteristics of lightweight and low bandwidth consumption, and is suitable for data transmission requirements in complex network environments such as mines, ensuring efficient and stable data upload.

[0061] After the upper computer (monitoring end data processing unit) reads the data from the server, it is imported into the digital twin system. In the system, alarm thresholds can be set according to different monitoring needs. For example, for mine environments, set the gas concentration exceeding 0.8% and the environmental temperature exceeding 40°C as the danger threshold; for the robot's own state, set the power below 20% and abnormal fluctuations in motion speed (such as a change in speed exceeding ±1 m / s in a short period of time) as the warning threshold. By setting thresholds, the system can quickly identify abnormal states.

[0062] Step S4, warning and real-time state display

[0063] Based on the set threshold and image analysis results, the system carries out intelligent early warning and state display. For different scenarios such as environmental abnormalities (such as excessive gas), equipment failures (such as high motor temperature, belt tearing), personnel risks (such as entering restricted areas, not wearing safety equipment), etc., set different thresholds to ensure the accuracy of the warning. When the data triggers the threshold, the system calls the image sample library constructed in advance, combines the real-time image video stream, and judges the abnormal state of equipment and personnel through feature matching and algorithm recognition, such as identifying the image features of loose equipment screws matching the fault features in the sample library, or monitoring the movement trajectory of personnel entering unauthorized areas, and triggering the warning immediately.

[0064] At the same time, the digital twin interface can display various monitoring parameters in real time, such as the gas concentration, temperature, and humidity values of the environment, the position coordinates of the robot, the remaining percentage of the power, the movement speed, and the abnormal state markers after image recognition (such as highlighting the location of the faulty equipment in the environmental model, the dangerous area of the personnel), so that the management personnel can intuitively understand the on-site situation and realize the closed-loop management of "data collection-analysis-warning-display".

[0065] For example, the specific implementation of the suspension type inspection robot monitoring method based on digital twin technology includes:

[0066] Step S1: Multiple cameras are arranged at multiple points in a similar dynamic environment for the robot to collect environmental data, and a dynamic complex environment feature data set is made to obtain a similar complex environment feature data set;

[0067] Step S2: During the operation of the robot, the environmental image video stream collection data carried on the robot body and the environmental monitoring data information of each sensor are obtained;

[0068] Step S3: The environmental image video stream collection data and the environmental monitoring data information of each sensor are transmitted to the digital twin remote monitoring system, the system alarm threshold is set, and the image state is analyzed;

[0069] Step S4: According to the alarm threshold and the image state analysis result, the environmental data is monitored, and the warning is issued according to the threshold and the image features. The digital twin remote monitoring interface monitors the parameters and the warning state in real time according to the processing message.

[0070] In the step S1, the specific steps of making the dynamic complex environment feature data set are as follows:

[0071] Step S11: Deploy multiple cameras in the environment similar to the environment where the robot is applied;

[0072] Step S12: Use high-resolution cameras to collect images for image processing;

[0073] Step S13: feature extraction, labeling, classification, integration and data set making of various abnormal state images;

[0074] Step S14: database integration using deep learning method to train camera algorithm.

[0075] The specific steps of the step S12 are as follows:

[0076] Step S121: denoising processing of the collected images using Gaussian filtering, curvelet transform and other algorithms;

[0077] Step S122: enhancement processing of the collected images using gray histogram equalization and canny operator edge extraction algorithm.

[0078] In the step S13, the feature extraction, labeling, classification, integration and data set making of various abnormal state images are as follows:

[0079] Step S131: feature parameter extraction of the coal mine working face using improved dictionary learning algorithm and image feature point matching algorithm;

[0080] Step S132: image feature classification for device personnel warning, environmental factors and the like.

[0081] In the step S14, the database integration using deep learning method to train camera algorithm is as follows:

[0082] Step S141: BP neural network is used to train the coal mine working face image sample library, and data integration of the abnormal state of the coal mine working face is realized.

[0083] In the step S2, the specific steps of the environment monitoring data information acquisition during the robot operation are as follows:

[0084] Step S21: real-time temperature data acquisition using the temperature sensor carried by the robot;

[0085] Step S22: real-time humidity data acquisition using the humidity sensor carried by the robot;

[0086] Step S23: real-time gas and carbon monoxide concentration data acquisition using the sensitive gas detection sensor carried by the robot;

[0087] Step S24: different types of image information acquisition using the infrared camera and event camera carried by the robot;

[0088] Step S25: acquisition of the monitoring information such as the power, position, speed and body temperature of the robot body using the power sensor, distance measuring sensor, temperature sensor, inertial navigation device and speed measuring device carried by the robot.

[0089] In the step S3, the environmental image data and the environmental monitoring data information are transmitted to the digital twin remote monitoring system, the system alarm threshold is set, and the image state analysis is performed according to the following steps:

[0090] Step S31: Image data and environmental monitoring data are acquired through a camera and a sensor;

[0091] Step S32: The data is transmitted to the digital twin remote monitoring system through the MQTT communication protocol;

[0092] Step S33: The digital twin remote monitoring system sets the alarm threshold, compares the data, and alarms;

[0093] Step S34: The data state trend is analyzed through the dynamic chart inside the digital twin remote monitoring system.

[0094] In the step S32, the data is transmitted to the digital twin remote monitoring system through the MQTT communication protocol according to the following steps:

[0095] Step S321: The data communication center carried by the lower computer collects sensor data through a serial port protocol

[0096] Step S322: The lower computer packs and integrates the data and sends it to each topic pool of the MQTT server through the MQTT protocol;

[0097] Step S323: The upper computer reads the data from the MQTT server and imports it into the digital twin remote monitoring system.

[0098] In the step S4, the environmental data is monitored according to the alarm threshold and the image state analysis result, and a warning is issued according to the threshold and the image characteristics according to the following steps:

[0099] Step S41: Set the system alarm threshold, and set different alarm thresholds for different types of alarms;

[0100] Step S42: Call the coal face image sample library to realize data recognition and analysis of the uploaded image state coal face abnormal state;

[0101] Step S43: The digital twin remote monitoring system sends a warning message according to the data analysis result.

[0102] On the basis of the above technical scheme, the monitoring system architecture is further constructed, based on the "perception-analysis-display" logic of the foregoing method part, the following functional modules are constructed, and each module cooperates to support the whole process of inspection and monitoring:

[0103] The visual monitoring module, as the core window of the system interacting with the user, integrates and displays the device running state, environmental parameters, robot self-data and early warning information in real time. Through the three-dimensional scene constructed by digital twin technology, the working environment is dynamically restored, and the collected multi-source data is mapped into the virtual model. Management personnel can intuitively view the layout of the mine roadway, the real-time running posture of the equipment (such as the cutting position of the coal mining machine and the running state of the conveyor), the moving track of personnel, and the dynamic change curve of various data on the interface without going to the scene, realizing the monitoring effect of "one screen observing the overall situation".

[0104] The environmental data monitoring module focuses on receiving and processing data collected by temperature, humidity and gas sensors, and performs environmental early warning according to the set threshold. Taking mine operation as an example, when the gas detection sensor monitors that the gas concentration gradually rises and approaches the dangerous threshold, the module will first make a trend prediction. If it is determined that the concentration continues to rise, it will immediately trigger an audible and light warning, and highlight the area where the gas exceeds the standard in the digital twin interface, while linking historical data to analyze the rules of gas anomalies in this area (such as whether it frequently occurs after a specific time period or operation process), assisting management personnel in developing response strategies, extending from simple "alarm" to "early warning + analysis" proactive management.

[0105] The equipment and personnel monitoring module uses infrared cameras and event cameras to cooperatively collect images, making up for the monitoring defects of single cameras in complex environments. The infrared camera captures heat radiation information such as equipment heat and personnel outline, and the event camera quickly responds to abnormal vibrations caused by equipment failures and sudden actions caused by personnel violations. After data fusion, the improved dictionary learning algorithm identifies abnormal states such as equipment wear and deformation, and non-standard personnel operations (such as not wearing a safety helmet or crossing a barrier). For example, in a coal mine working face, the algorithm can accurately identify the slack deformation characteristics of the chain of a scraper conveyor or the behavior of a worker entering a dangerous area without wearing reflective clothing, and timely trigger equipment maintenance and personnel warning instructions to ensure production safety.

[0106] The robot state monitoring module receives and analyzes power, inertial navigation and speed measurement sensor data in real time, and monitors robot running parameters in all directions. Through the attitude data feedback by the inertial navigation device, it determines whether the robot is tilted or stuck due to roadway undulation or equipment collision. Combined with the speed measurement device and power sensor, it analyzes the motion energy consumption and efficiency of the robot. When it finds that the power consumption is too fast or the motion speed is abnormal (such as sudden speed drop due to track jam), it automatically generates a robot health diagnosis report and pushes it to the management end, plans the maintenance, energy charging or path adjustment of the robot in advance to avoid the impact of robot failure on the overall inspection task.

[0107] The image data analysis and early warning module, in addition to performing image anomaly recognition, has a dynamic chart trend analysis unit built in, which deeply mines the collected image data and associated sensor data. For example, for image sequences of mine equipment, the temperature variation trend of the equipment surface (combined with infrared camera data) and the evolution law of the wear degree over time (by tracking the texture changes of the equipment through feature extraction algorithms) are analyzed to predict equipment failure risks in advance; for personnel operation images, the time length of personnel staying in different areas and the operation frequency are counted to assist in optimizing personnel scheduling and safety control strategies, so that image data is upgraded from an "anomaly recognition tool" to a "decision basis for equipment health management and personnel behavior analysis".

[0108] For example, the specific implementation of the suspension type inspection robot monitoring system based on digital twin technology includes:

[0109] The visual monitoring module provides visual real-time running status of the equipment.

[0110] The environmental data monitoring module performs environmental monitoring and issues monitoring warnings according to the data provided by external sensors; mainly through temperature, humidity, and sensitive gas sensors, workface environmental data is collected to realize workface environmental monitoring.

[0111] The equipment and personnel monitoring module performs equipment operation monitoring and personnel danger early warning according to the data provided by external sensors; mainly through two cameras, equipment operation video data is collected to monitor the equipment operation status.

[0112] The robot state monitoring module detects the real-time running status of the robot according to the data provided by internal sensors; mainly through body temperature sensors, battery level sensors, RFID readers, inertial navigation sensors, and Hall sensors to detect the body status of the workface inspection robot.

[0113] The image data analysis and early warning module identifies and judges the abnormal state of the environment and equipment according to the image video stream data uploaded by the camera and issues early warnings.

[0114] The technical concept of the present application is derived from abandoning the single-dimensional monitoring idea, fully integrating the data of image vision (high-resolution cameras, infrared / event cameras), environmental perception (temperature and humidity, gas sensors), and robot state monitoring (power, inertial navigation, etc.), and constructing a closed loop of "physical working environment - virtual mapping model - intelligent analysis and decision" through a digital twin system, so that the monitoring is upgraded from "single-point alarm" to "global perception and trend prediction".

[0115] The core of the present application is to link with digital twinning, deeply integrate multi-source perception, intelligent algorithm and scenario application, and break the limitations of traditional inspection such as "scattered data, late warning, poor scene adaptation". Through the pre-deployed static camera group in the mine tunnel to collect environmental images, the dynamic environment feature dataset is constructed through a specific image processing process, combined with the multi-source sensor data returned by the suspended inspection robot in real time, and the multi-dimensional warning is realized on the digital twinning platform. The key technical principle is to overturn the traditional "feature extraction first and enhancement second" process, and the processing order of "enhancement first and feature extraction second" is created: first, the low-illumination image is denoised and the contrast is improved, and the input quality of feature extraction is optimized, then the key features are extracted through the improved dictionary learning algorithm, and the recognition reliability in harsh environment is significantly improved.

[0116] The present application uses multi-algorithm cooperative image preprocessing and feature extraction mechanism, aiming at the scene of low-illumination, high-dust, complex equipment / personnel features in mine, through Gaussian filter, curvelet transform step-by-step denoising, gray histogram equalization, canny operator to enhance details, and then use improved dictionary learning algorithm to accurately capture mine specific features, combined with BP neural network to build sample library, solve the problem of "low recognition rate and high false alarm rate" of general algorithm in mine scene; at the same time, the combination of MQTT protocol and digital twinning system ensures the efficient transmission and visual presentation of data in complex network environment, making remote monitoring and intelligent warning operable.

[0117] Taking the monitoring of mine working face equipment as an example, the traditional manual inspection relies on experience and is difficult to find subtle faults of equipment in time; the conventional robot inspection can only collect basic data and lacks deep adaptation to complex scenes. The present application captures multi-dimensional features of equipment temperature, vibration (combined with sensors) and appearance texture (image algorithm) through multi-source data acquisition, and uses digital twinning system to map the equipment state in real time. Once the algorithm identifies the correlation between texture change and temperature anomaly of the equipment (such as screw loosening leading to local friction heating), it will immediately warn and push maintenance suggestions, realizing the change from "passive fault maintenance" to "active health management", which is the embodiment of the value of technological innovation in practical scenarios.

[0118] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications of these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A monitoring method for inspection robots based on digital twins, characterized in that, include: S1: Deploy multiple cameras in a similar environment to the robot's operation to construct a dynamic environmental feature dataset; S2: Acquire image and video streams and multi-sensor environmental monitoring data captured by the robot's onboard camera; S3: Transmit data to the digital twin system via the MQTT protocol and set alarm thresholds; S4: Provide early warnings and real-time status display based on threshold and image analysis results.

2. The inspection robot monitoring method based on digital twin according to claim 1, characterized in that, Step S1 includes: S11: Deploy multiple high-resolution cameras in similar environments; S12: Perform Gaussian filtering and curvelet transform noise reduction on the acquired image; S13: Image enhancement is performed using grayscale histogram equalization and the Canny operator edge extraction algorithm.

3. The inspection robot monitoring method based on digital twin according to claim 2, characterized in that, Step S1 further includes: S14: Extract feature parameters of coal mine working faces using an improved dictionary learning algorithm; S15: Data integration is achieved by training an abnormal state image sample library through a BP neural network.

4. The inspection robot monitoring method and system based on digital twin as described in claim 1, characterized in that, The multiple sensors in step S2 include: Temperature sensors, humidity sensors, gas detection sensors, infrared cameras, event cameras, power sensors, distance sensors, inertial navigation devices, and speed measuring devices.

5. The inspection robot monitoring method based on digital twin according to claim 1, characterized in that, Step S3 includes: S31: The lower-level machine collects data from each sensor via a serial port protocol; S32: Pack the data and send it to the server topic pool via the MQTT protocol; S33: The host computer reads data from the server and imports it into the digital twin system.

6. The inspection robot monitoring method based on digital twin according to claim 1, characterized in that, Step S4 includes: S41: Set differentiated thresholds for environmental anomalies, equipment failures, and personnel risks; S42: Call the image sample library to identify abnormal states; S43: Real-time display of monitoring parameters and early warning status via a digital twin interface.

7. A monitoring system for inspection robots based on digital twins, characterized in that, include: Visual monitoring module: Displays the real-time operating status of the equipment; Environmental data monitoring module: performs environmental early warning based on sensor data; Equipment and Personnel Monitoring Module: Performs hazard warnings for equipment and personnel; Robot status monitoring module: Monitors the robot's real-time operating parameters; Image data analysis and early warning module: Identifies abnormal states in video streams.

8. The inspection robot monitoring system based on digital twin according to claim 7, characterized in that, The environmental data monitoring module is configured to receive data from temperature, humidity and gas sensors. The robot status monitoring module is configured to receive data from power, inertial navigation, and speed sensors.

9. The inspection robot monitoring method and system based on digital twin according to claim 7, characterized in that, The image data analysis and early warning module includes a dynamic chart trend analysis unit, which is used to perform data status trend analysis.

10. The inspection robot monitoring method and system based on digital twin according to claim 7, characterized in that, The equipment and personnel monitoring module uses an infrared camera and an event camera to collect images in collaboration, and identifies equipment and personnel anomalies through an improved dictionary learning algorithm.

Images