Remote camera and robot linkage patrol system and patrol method

Abstract

The invention relates to the technical field of remote monitoring, in particular to a remote camera and robot linkage patrol system and patrol method. The patrol system comprises a remote camera monitoring module which remotely monitors the patrol robot and the industrial facility through a remote camera; the robot inspection module inspects industrial facilities through an inspection robot; the cooperative control module is used for processing and analyzing data acquired by the remote camera and the inspection robot, and controlling the remote camera monitoring module and the robot inspection module based on an analysis result; the security protection module is used for encrypting and decrypting data transmitted in each module; when the inspection system is detected to have a fault, the safety protection module triggers a corresponding safety protection strategy based on a fault type; and the data analysis and optimization module is used for optimizing the parameters of the patrol system based on the data stored in the database module. According to the invention, the inspection efficiency is improved, the human input is reduced, and the inspection quality and safety are guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote monitoring, in particular to a remote camera and robot linkage patrol system and method. BACKGROUND

[0002] With the rapid development of modern industry, the safety and stability of industrial facilities have become key factors to ensure production continuity, personnel safety and economic benefits. Substations, chemical plants, oil and gas pipeline hubs and other industrial facilities, as the core nodes of energy supply and industrial production, their operating status is directly related to the regional economic lifeline and social stability. However, these facilities are often distributed in remote areas or complex environments, and traditional manual patrol methods are not only inefficient and costly, but also difficult to achieve real-time monitoring of all-weather and full-coverage. Therefore, developing an efficient, intelligent and reliable remote monitoring system has become an urgent need in the field of industrial safety.

[0003] Currently, various technical solutions have been proposed for remote monitoring of industrial facilities. Among them, the system based on video monitoring realizes real-time picture transmission of the facility by deploying a camera network, but this scheme highly depends on manual duty, cannot automatically identify abnormal state and trigger early warning, and the monitoring effect is greatly reduced in bad weather or night environment. Another solution is to use robots for autonomous patrol, which completes equipment inspection through pre-set paths or remote control. However, the existing robot system still faces significant technical bottlenecks: on the one hand, the path planning lacks adaptive ability and is difficult to cope with dynamic environmental changes (such as equipment failure, obstacle appearance, etc.), resulting in low patrol efficiency or task interruption; on the other hand, the robot's own state monitoring and fault warning mechanism is imperfect, often resulting in task failure due to sudden failure (such as wheel group stuck, sensor failure), and even causing secondary accidents.

[0004] Traditional monitoring systems still have many limitations: first, the real-time response capability is insufficient. Video monitoring relies on manual interpretation and cannot achieve automatic identification and second-level response of abnormal events; although the robot system has a certain autonomy, it is limited by communication delay and decision delay, and it is difficult to meet the rapid intervention needs in emergency scenarios. Second, the environmental adaptability is poor. Existing technologies are difficult to effectively cope with unstructured environments (such as narrow passages, high temperature and humidity areas) in complex industrial scenes, resulting in monitoring blind spots or increased equipment wear and tear. Third, the operation and maintenance cost is high. Manual patrol requires continuous investment of a large number of manpower, while the robot system has a high failure rate, resulting in a sharp increase in maintenance costs, both of which do not meet the core demand of cost reduction and efficiency improvement. Fourth, the level of intelligence is insufficient. Traditional systems lack the ability to fuse and analyze multi-source data (such as equipment vibration, temperature, gas concentration), making it difficult to achieve the leap from "passive monitoring" to "active prediction".

[0005] In view of the above problems, the industry urgently needs a new remote monitoring system integrating autonomous navigation, environment perception, intelligent decision-making and remote collaboration functions. SUMMARY

[0006] The purpose of the present application is to provide a remote camera and robot linked patrol system and method to solve at least one of the above technical problems.

[0007] The present application achieves the above-mentioned purposes through the following technical solutions: A remote camera and robot linked patrol system, comprising: a remote camera monitoring module, a robot inspection module, a collaborative control module, a safety protection module, a database module, and a data analysis optimization module. The remote camera monitoring module is used to remotely monitor the patrol robot and industrial facilities through a remote camera. The robot inspection module is used to patrol the industrial facilities through the patrol robot. The collaborative control module is used to process and analyze the data obtained by the remote camera and the patrol robot, and control the remote camera monitoring module and the robot inspection module based on the analysis results. The safety protection module is used to encrypt and decrypt the data transmitted in each module. When a fault is detected in the patrol system, the safety protection module is also used to trigger the corresponding safety protection strategy based on the fault type. The database module is used to store the operating data of each module and the patrol map generated based on the actual working scenario. The data analysis optimization module is used to optimize the parameters of the patrol system based on the data stored in the database module.

[0008] Further, the remote camera monitoring module uses a YOLOv8 model with an added attention mechanism to perform multi-target detection on the collected images. The detected targets include robot pose, obstacles, and equipment status. The patrol robot is provided with an environment perception unit that collects surrounding environment data of the patrol robot and transmits it to the robot inspection module. The robot inspection module uses a rain and fog compensation algorithm to judge the current operating environment.

[0009] Further, the patrol system further comprises a dynamic safety identification module and a positioning module. The dynamic safety identification module comprises intelligent signboards deployed at each industrial facility. The intelligent signboards display relevant data of the corresponding industrial facilities through dynamic QR codes. The patrol robot is provided with a QR code recognition unit for recognizing the intelligent signboards. The positioning module comprises: a UWB (Ultra-Wide Band) positioning base station network arranged in a monitoring area, a UWB tag arranged on the inspection robot, and an IMU (Inertial Measurement Unit) inertial unit; the positioning module obtains the positioning coordinates of the inspection robot by using the UWB positioning coordinates, the vision positioning coordinates obtained by the remote camera, and the inertial navigation correction amount provided by the IMU inertial unit.

[0010] Further, the cooperative control module comprises: an intelligent path planning unit and a control unit. The intelligent path planning unit is configured to generate a planned path for each inspection robot based on the inspection map, position information of the industrial facility to be inspected, and position information of each inspection robot. The control unit is configured to control the inspection robot to inspect the industrial facility based on the planned path, and control the corresponding remote camera to remotely monitor the inspection robot based on the planned path and the positioning coordinates of the inspection robot.

[0011] Further, a corresponding safety protection strategy is triggered based on the fault type, comprising: When the positioning coordinates of the target inspection robot deviate from the preset planned path by more than a first preset distance threshold and less than a second preset distance threshold, an automatic deviation correction instruction is sent to the target inspection robot, and the target inspection robot automatically corrects its position. When the positioning coordinates of the target inspection robot deviate from the preset planned path by more than a second preset distance threshold, the intelligent path planning unit re-generates a planned path for the target inspection robot. When the target inspection robot encounters an obstacle on the corresponding planned path and cannot continue to inspect, the target inspection robot brakes urgently and sends an alarm prompt to the cooperative control module.

[0012] Further, a corresponding safety protection strategy is triggered based on the fault type, comprising: when it is detected that the target intelligent signboard is illegally disassembled or has an environmental anomaly, the target intelligent signboard starts a self-destruction mechanism. Based on the operation data of the inspection robot, an LSTM model is used to predict the fault of the inspection robot, and an alarm prompt is sent to the cooperative control module at a preset time before the predicted time point of the fault of the inspection robot.

[0013] Further, the parameters of the inspection system are optimized, comprising: Based on the temperature data, vibration frequency, and appearance defects of the inspection robot, the device health degree of the inspection robot is obtained, and the maintenance frequency of the inspection robot is planned according to the device health degree. The patrol map is divided into a region risk level, and the region risk level includes low risk, medium risk and high risk; the intelligent path planning unit of the cooperative control module avoids high risk level regions and reduces the frequency of passing through medium risk level regions when generating a planned path.

[0014] Further, the patrol map is generated by using a digital twin modeling technology. Before the patrol robot patrols according to the planned path, the cooperative control module simulates each patrol robot according to the patrol plan through the digital twin system, and when the simulation result confirms that the plan is risk-free, the patrol robot patrols according to the planned path.

[0015] A patrol method of a remote camera and a robot linkage, which adopts the remote camera and the robot linkage of any one of the above-mentioned patrol systems, and the method comprises the following steps: Load the patrol map generated based on the digital twin modeling technology, deploy a UWB ultra-wideband positioning base station network and an intelligent signboard, start a self-checking program of each module and synchronize initial parameters; Obtain the positioning coordinates of each patrol robot, and generate an initial planned path for each patrol robot; Obtain a planned path by virtually simulating and verifying the initial planned path through the digital twin system; Each patrol robot patrols the industrial facility based on the corresponding planned path; the remote camera monitoring module controls the remote camera to cooperate with collecting information of the patrol robot and the industrial facility according to the planned path sent by the cooperative control module; The cooperative control module obtains a patrol result based on the information collected by the remote camera and the information collected by the patrol robot.

[0016] Further, when the cooperative control module determines that the patrol site of the patrol robot is located in an indoor scene, the patrol system starts an indoor scene enhancement scheme; The indoor scene enhancement scheme comprises: The patrol robot performs pedestrian perception based on radar detection, gait recognition and voiceprint recognition; When the patrol robot perceives a pedestrian, the distance between the patrol robot and the pedestrian is kept not less than a third preset distance; The patrol system comprises an indoor lighting module, and the indoor lighting module controls indoor light to illuminate the patrol path of the patrol robot in advance, so that the illumination intensity of the patrol path reaches an illumination intensity threshold.

[0017] The present application has the following advantages: Full autonomous patrol capability: Through multi-modal sensor fusion and reinforcement learning algorithm, dynamic planning and real-time adjustment of the path are realized, and efficient patrol in complex scenarios can be completed without human intervention, significantly reducing the dependence on operators and improving the patrol efficiency.

[0018] Adaptive environmental interaction mechanism: A three-dimensional model of the facility is constructed using digital twin technology, which maps the physical environment changes (such as equipment status, spatial layout, etc.) in real time, providing accurate navigation and obstacle avoidance support for the robot. At the same time, through rain and fog compensation algorithm, radar detection and other multi-modal sensing means, the environmental adaptability in complex environments (such as low light, rain and fog) is enhanced.

[0019] Multi-level fault warning system: Based on the collaborative architecture of edge computing and cloud computing, the device status data, environmental parameters and robot ontology data (temperature, vibration frequency, appearance defects, etc.) are jointly analyzed to build a multi-level warning mechanism of "device health assessment - regional risk level division - fault prediction", realizing early prediction and rapid positioning of faults.

[0020] Human-machine collaborative remote control mode: While retaining the human intervention interface, the robot's perspective and expert knowledge base are deeply integrated through AR (Augmented Reality) technology, significantly improving the accuracy and efficiency of remote control. For indoor scenarios, the "pedestrian perception - safe distance maintenance - intelligent lighting" enhancement scheme (radar detection, gait / voiceprint recognition pedestrians, automatic adjustment of path and pre-illumination of the patrol area) is started to further ensure the safety of the operation.

[0021] Complex environment adaptability breakthrough: Relying on digital twin modeling technology and robot path planning capability, all-around monitoring of industrial facilities such as substations is realized, solving the monitoring blind spot problem of traditional solutions in complex environments (such as dynamic obstacles, unstructured space). Through data analysis driven path adaptive optimization, the planned path is accurately adapted to the specific environmental requirements (such as indoor / outdoor, high-risk / low-risk areas).

[0022] Through the full-link technology innovation of "autonomous planning - environmental adaptation - fault warning - human-machine collaboration", the invention provides a revolutionary solution for the safe operation of industrial facilities, promoting the upgrading of industrial monitoring mode to intelligent and preventive direction. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 Structure diagram of the remote camera and robot linkage patrol system of an embodiment of the invention; Figure 2 Flowchart of the remote camera and robot linkage patrol method of an embodiment of the invention. DETAILED DESCRIPTION

[0024] The invention will now be discussed with reference to exemplary embodiments. It should be understood that the described embodiments are merely intended to enable those skilled in the art to better understand and thus implement the invention, and are not intended to imply any limitation on the scope of the invention.

[0025] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment".

[0026] Example 1 Figure 1 This is a schematic diagram of a remote camera and robot linkage inspection system according to one embodiment of the present invention. Figure 1 As shown, according to one embodiment of the present invention, a remote camera and robot linkage inspection system includes: a remote camera monitoring module, a robot inspection module, a collaborative control module, a security protection module, a database module, and a data analysis and optimization module; The remote camera monitoring module is used to remotely monitor and inspect robots and industrial facilities via a remote camera. The robot inspection module is used to inspect industrial facilities using inspection robots; The collaborative control module is used to process and analyze the data acquired by the remote camera and the inspection robot, and to control the remote camera monitoring module and the robot inspection module based on the analysis results; The security protection module is used to encrypt and decrypt the data transmitted in each module; when a fault is detected in the patrol system, the security protection module is also used to trigger corresponding security protection strategies based on the fault type. The database module is used to store the operational data of each module and the inspection map generated based on the actual operation scenario; The data analysis and optimization module is used to optimize the parameters of the inspection system based on the data stored in the database module.

[0027] In this embodiment, a remote camera and robot linkage patrol system is proposed, aiming to achieve efficient and safe patrol of industrial facilities through intelligent device coordination mechanism. The system includes remote camera monitoring module, robot inspection module, collaborative control module, safety protection module, database module and data analysis optimization module. These modules work together to form an efficient and reliable patrol system. The remote camera monitoring module uses remote cameras deployed at key locations to monitor the patrol robot and industrial facilities in real time. These cameras have high resolution, wide viewing angle and night vision function, which can adapt to different environmental conditions to ensure clear and accurate monitoring images. The cameras transmit the collected image and video data to the collaborative control module in real time through 5G network for further analysis and processing. In addition, the remote camera also has intelligent tracking function, which can automatically adjust the angle and focal length of the camera according to the position and motion trail of the patrol robot, and always keep effective monitoring of the patrol robot, so as to realize the whole process tracking of the patrol process. The robot inspection module is the core of the system, which is composed of multiple patrol robots with autonomous navigation capability. These patrol robots are equipped with various sensors such as infrared thermal imager, gas detection sensor and laser radar, which can conduct comprehensive and detailed inspection of industrial facilities. The robot moves autonomously in the industrial facility area through the pre-planned patrol path or the dynamic path planning based on real-time environmental perception, and monitors the running state and environmental parameters of the equipment in real time. When the robot detects abnormal conditions such as abnormal temperature rise of equipment and gas leakage, it will immediately send the abnormal information to the collaborative control module through 5G network, and trigger alarm if necessary to inform relevant personnel for processing. The collaborative control module is responsible for comprehensive processing and analysis of the data obtained by the remote camera and the patrol robot. This module uses advanced data fusion algorithm to integrate the image data of the remote camera and the detection data of the patrol robot sensors, so as to more comprehensively and accurately evaluate the running state of the industrial facilities. Based on the analysis results, the collaborative control module can control the remote camera monitoring module and the robot inspection module in real time. For example, when the patrol robot detects an abnormal area, the collaborative control module can instruct the remote camera to monitor the area, and adjust the patrol path of the patrol robot to make it check the area in more detail. In addition, the collaborative control module also supports multi-robot collaborative work, which designs a distributed scheduling system based on 5G ultra-low latency (<10ms) communication to realize dynamic role allocation among multiple robots, adopts "leader-follower" mode to ensure the cooperation of robots and avoid mutual interference. At the same time, the database module uses blockchain technology to store the task log, ensuring the traceability of the patrol operation, which is convenient for subsequent analysis and responsibility identification.The security protection module is responsible for encrypting and decrypting the data transmitted in each module, ensuring the security and integrity of the data. This module uses advanced encryption algorithms such as AES (Advanced Encryption Standard) or RSA (Asymmetric Encryption Algorithm) to encrypt camera image data, robot sensor data, and collaborative control instructions, preventing data from being stolen or tampered with during transmission. When the security protection module detects a fault in the patrol system, it will trigger the corresponding security protection strategy according to the fault type. For example, if a network attack is detected, the system will immediately disconnect from the external network and enter isolation mode, while starting a backup communication link to ensure the basic functions of the system are not affected; if the patrol robot fails, the security protection module will instruct the patrol robot to enter a safe shutdown state and notify maintenance personnel for repair. The database module is the data storage center of the system, used to store the running data of each module and the patrol map generated based on the actual work scene. This module uses a distributed database architecture to efficiently store and manage large amounts of data. Running data includes camera monitoring data, robot patrol data, collaborative control instruction records, etc., which provide a basis for subsequent data analysis and system optimization. The patrol map is automatically generated based on the layout of industrial facilities and the patrol path of the robot, and contains device locations, passage information, dangerous area annotations, etc., providing an important reference for the autonomous navigation and task planning of the robot. The database module also has data backup and recovery functions to ensure data security and reliability. The data analysis and optimization module is the key to continuous improvement of the system, which optimizes the parameters of the patrol system based on the data stored in the database module. This module uses machine learning algorithms and data mining techniques to analyze historical data, identify problems and potential optimization points in the patrol process. For example, by analyzing the patrol robot's patrol time in different areas, the frequency of detected abnormalities, etc., the patrol path and task allocation strategy of the patrol robot can be optimized to improve patrol efficiency; by analyzing camera monitoring data, the monitoring angle and shooting frequency of the camera can be adjusted to ensure the monitoring effect of key areas. In addition, the data analysis and optimization module can also predict potential equipment failures, arrange maintenance plans in advance, reduce equipment failure rates, and improve the operational stability of industrial facilities.

[0028] The remote camera and robot linkage patrol system proposed in the present application realizes efficient and safe patrol of industrial facilities through the collaborative work of each module; the system not only can respond to abnormal situations in real time, but also supports multi-robot collaborative work, has strong security protection capabilities and data analysis and optimization functions, and provides strong support for intelligent operation and maintenance of industrial facilities.

[0029] According to an embodiment of the present application, the remote camera monitoring module uses a YOLOv8 model with an added attention mechanism to perform multi-target detection on the collected images. The detected targets include robot pose, obstacles, and equipment status. The environment perception unit is arranged on the inspection robot, which collects the surrounding environment data of the inspection robot and transmits it to the robot inspection module. The robot inspection module uses a rain and fog compensation algorithm to judge the current running environment.

[0030] In this embodiment, the remote camera monitoring module and the robot inspection module achieve efficient monitoring and inspection of industrial scenes through advanced technical means and hardware deployment. The remote camera monitoring module uses an improved YOLOv8 model, which introduces an attention mechanism based on the traditional YOLOv8, making the model more focused on key areas in the image, thereby improving the accuracy and efficiency of detection. In this way, the model can simultaneously identify multiple targets such as robot pose, obstacles, and equipment status. In terms of hardware deployment, the camera end is equipped with a Cambrian MLU270 chip, which is a high-performance neural network processing unit (NPU) that can efficiently support the operation of deep learning models. With the powerful computing power of this chip, the system can analyze 4K video streams in real time, with a frame rate of up to 30fps, allowing it to quickly process high-resolution images collected to ensure the real-time and accuracy of monitoring data. The improved YOLOv8 model combined with the attention mechanism can better handle complex industrial scenes. For example, when detecting the robot pose, the model can accurately identify the position, direction, and attitude information of the robot, providing support for the navigation and task planning of the robot. At the same time, for obstacle detection, the model can quickly identify obstacles in the path and provide real-time data for the robot's obstacle avoidance. In addition, the equipment status detection function allows the system to promptly detect abnormal conditions of equipment, such as damage to equipment components and abnormal running states, so that measures can be taken in advance to avoid potential safety hazards.

[0031] The inspection robot is equipped with an environment perception unit for collecting environmental data around it. These data include but are not limited to distance, obstacle information, ambient light, humidity, etc., providing support for the robot's autonomous navigation and task execution. To cope with complex industrial environments, the robot end adopts a multi-sensor fusion scheme of millimeter wave radar and TOF (Time-of-Flight) camera. Millimeter wave radar can provide high-precision distance measurement and speed information, especially in long distance and complex weather conditions. The TOF camera can generate real-time three-dimensional point cloud data of the environment, providing detailed environmental depth information for the robot. Through the fusion of the two sensors, the robot can more comprehensively perceive the surrounding environment, achieving high-precision autonomous navigation and obstacle avoidance. In addition, the robot inspection module also develops a rain and fog compensation algorithm. This algorithm can judge the current operating environment and optimize the image data. In severe weather conditions with visibility below 5 meters, the rain and fog compensation algorithm can effectively reduce the impact of rain and fog on image quality, ensuring the normal operation of the robot in low-visibility environments. This means that the robot can continue to perform inspection tasks in adverse conditions such as rain and fog, without the need to suspend work due to weather, thereby improving the availability and reliability of the system.

[0032] Through the cooperative work of the camera end and the robot end, the system can achieve efficient monitoring and inspection tasks. This edge collaborative control mechanism fully utilizes the advantages of each device, achieving optimized resource allocation and efficient task execution.

[0033] The present application realizes efficient cooperative work of remote camera and robot inspection module through advanced technical means and hardware deployment, providing an intelligent and efficient solution for monitoring and inspection of industrial facilities.

[0034] According to an embodiment of the present application, the inspection system further comprises a dynamic safety identification module and a positioning module. The dynamic safety identification module comprises intelligent signboards deployed at each industrial facility, which display relevant data of the corresponding industrial facility through dynamic QR codes; the inspection robot is provided with a QR code recognition unit for recognizing the intelligent signboards. Preferably, the relevant data of the industrial facility includes identity information, environmental temperature information, environmental humidity information, and environmental light intensity of the industrial facility.

[0035] The positioning module comprises a UWB ultra-wideband positioning base station network deployed in the monitoring area, a UWB tag and an IMU inertial unit deployed on the inspection robot; the positioning module obtains the positioning coordinates of the inspection robot through the UWB positioning coordinates, the robot vision positioning coordinates obtained by the remote camera, and the inertial navigation correction amount provided by the IMU inertial unit.

[0036] In this embodiment, the patrol system further incorporates a dynamic safety identification module and a positioning module to enhance the system's intelligence, security, and positioning accuracy.

[0037] The dynamic safety identification module includes smart signs deployed at various industrial facilities. These signs display relevant data about the corresponding industrial facilities via dynamic QR codes. The smart signs utilize e-ink display technology, enabling clear display of the dynamic QR codes and featuring low power consumption, high contrast, and good outdoor visibility. The smart signs integrate multiple environmental sensing sensors, including temperature, humidity, and light sensors, to monitor environmental parameters around the industrial facilities in real time. Data collected by these sensors, such as ambient temperature, humidity, and light intensity, is encoded into the dynamic QR code along with the industrial facility's identification information. The QR code encoding rule is "substation ID@area ID@equipment ID@timestamp," and it is encrypted using the AES256 encryption algorithm to ensure data security and uniqueness. To enhance security, the smart signs also have built-in vibration sensors. Upon detecting abnormal vibration (such as unauthorized disassembly or vandalism), a data erasure mechanism is immediately triggered to prevent the leakage of sensitive information. The dynamic QR code on the smart signs is updated every 60 seconds, using a time-series encryption algorithm to generate a new verification code. This dynamic update mechanism ensures the validity and security of the QR codes, preventing them from being illegally copied or tampered with. The inspection robot is equipped with a QR code recognition unit that can identify the dynamic QR codes on smart signs in real time, thereby obtaining relevant data about the industrial facilities. This data not only provides important references for the inspection robot's tasks but also provides foundational information for subsequent data analysis and equipment maintenance.

[0038] The positioning module combines UWB (Ultra-Wideband) positioning, visual QR code positioning, and an inertial navigation system (INS), achieving centimeter-level positioning accuracy (±2cm) through a deep learning fusion algorithm. The positioning module first deploys a network of UWB base stations, evenly distributed within the monitoring area at intervals of less than 50 meters. UWB technology, with its high accuracy (up to ±10cm) and strong anti-interference capabilities, provides the robot with fundamental positioning data. The patrol robot is equipped with UWB tags to receive signals transmitted from the base stations and calculate its own position. Simultaneously, the robot is also equipped with an IMU (Inertial Measurement Unit) to provide inertial navigation corrections. The IMU can measure the robot's acceleration and angular velocity in real time, calculating the robot's position change through integration, thus providing continuous position information in a short time, compensating for the shortcomings of UWB positioning in situations such as signal obstruction. Combining UWB positioning coordinates, visual QR code positioning coordinates, and the inertial navigation corrections provided by the IMU, the system establishes a positioning data fusion model: position = a * (UWB_data) + b * (visual_position) + g * (INS_correction) Wherein, the position is the patrol robot position; a, b, g are the first weight, the second weight and the third weight respectively; UWB_data is the UWB positioning coordinates; visual_position is the visual positioning; INS_correction is the inertial navigation correction amount.

[0039] This multi-modal positioning method not only improves the positioning accuracy and reliability, but also enhances the adaptability of the system in complex environments. For example, in GPS signal denial or weak signal environment, the system can automatically switch to UWB and visual two-dimensional code positioning mode, ensuring that the robot can continuously and accurately position and navigate.

[0040] The present application significantly improves the intelligence, safety and positioning accuracy of the patrol system through the innovative design of the dynamic security identification module and the positioning module, providing strong support for efficient inspection and safety management of industrial facilities.

[0041] According to an embodiment of the present application, the cooperative control module comprises: an intelligent path planning unit, a control unit; The intelligent path planning unit is used to generate a planning path for each patrol robot based on the patrol map, the position information of the industrial facility to be patrolled, and the position information of each patrol robot; The control unit is used to control the patrol robot to patrol the industrial facility based on the planning path; the control unit is also used to control the corresponding remote camera to remotely monitor the patrol robot based on the planning path and the positioning coordinates of the patrol robot.

[0042] Preferably, the corresponding safety protection strategy is triggered based on the fault type, including: When the positioning coordinates of the target patrol robot deviate from the preset planning path by more than a first preset distance threshold and less than a second preset distance threshold, an automatic deviation correction instruction is sent to the target patrol robot, and the target patrol robot automatically corrects its position; When the positioning coordinates of the target patrol robot deviate from the preset planning path by more than the second preset distance threshold, the intelligent path planning unit regenerates a planning path for the target patrol robot; When the target patrol robot encounters an obstacle on the corresponding planning path and cannot continue to patrol, the target patrol robot brakes urgently and sends an alarm prompt to the cooperative control module.

[0043] In this embodiment, the cooperative control module includes an intelligent path planning unit and a control unit. The intelligent path planning unit is responsible for generating an optimal planned path for each inspection robot. The inputs of this unit include inspection target points, real-time weather data, and device priorities, etc. Based on these input data, the intelligent path planning unit trains a path planning model using the PPO (Proximal Policy Optimization) reinforcement learning algorithm to generate an optimal path and a backup path set. The outputs of the model include: 1. The optimal path, which is the path that the inspection robot should follow under normal circumstances, achieving a balance between time, energy consumption, and safety. For example, the path will try to avoid high energy consumption areas while ensuring that the robot can complete the inspection task within the specified time. 2. The backup path set, in order to deal with special situations such as heavy rain, strong wind, etc., the intelligent path planning unit will also generate a backup path set. These backup paths are optimized according to different scenarios to ensure that the robot can still safely and efficiently complete the task under extreme conditions. 3. Risk heat map: the path planning unit also generates a risk heat map to mark out potential risk areas such as water accumulation or icing areas. This provides important reference information for the inspection robot during the inspection process, helping it to avoid potential dangers in advance.

[0044] The control unit is responsible for controlling the inspection robot to inspect the industrial facility according to the planned path generated by the intelligent path planning unit. The control unit receives the real-time positioning coordinates of the inspection robot and compares them with the planned path to ensure that the robot can travel according to the predetermined path.

[0045] When the positioning coordinates of the target inspection robot deviate from the preset planned path, the control unit will take different measures according to the degree of deviation, including: 1. Automatic correction, if the positioning coordinates of the robot deviate from the preset planned path by more than a first preset distance threshold but less than a second preset distance threshold (for example, 15 cm), the control unit will send an automatic correction instruction to the target inspection robot. After receiving the instruction, the robot will automatically adjust its position to return to the planned path. The second preset distance threshold is greater than the first preset distance threshold. 2. Replanning the path, if the positioning coordinates of the robot deviate from the preset planned path by more than the second preset distance threshold, it means that the robot may encounter a problem that cannot be solved by automatic correction. At this time, the intelligent path planning unit will generate a new planned path for the target inspection robot to ensure that the robot can continue to complete the inspection task. 3. Emergency braking and warning, if the target inspection robot encounters an obstacle on the corresponding planned path and cannot continue to inspect, the robot will immediately execute emergency braking and send a warning prompt to the cooperative control module. After receiving the warning, the control unit will take appropriate measures, such as notifying maintenance personnel to handle the obstacle, or adjusting the tasks of other robots to ensure the smooth progress of the inspection task.

[0046] In addition, the control unit also controls the corresponding remote camera to remotely monitor the patrol robot according to the planned path and the positioning coordinates of the patrol robot. In this way, the operator can observe the patrol state of the robot in real time and timely discover and handle possible problems.

[0047] The present application significantly improves the intelligent level and operation efficiency of the patrol system through the intelligent path planning and dynamic control function of the cooperative control module, and provides strong support for efficient patrol and safety management of industrial facilities.

[0048] According to an embodiment of the present application, the corresponding safety protection strategy is triggered based on the fault type, including: when it is detected that the target intelligent signboard exists illegal disassembly or environmental abnormality, the target intelligent signboard starts a self-destruction mechanism; Based on the operation data of the patrol robot, the LSTM model is used to predict the fault of the patrol robot, and an alarm prompt is sent to the cooperative control module at a preset time before the predicted time point of the fault of the patrol robot.

[0049] In the present embodiment, the intelligent signboard of the dynamic safety identification module has advanced safety protection function to cope with illegal disassembly or environmental abnormality. Each intelligent signboard is built-in with multiple sensors, including vibration sensor and environmental monitoring sensor (such as temperature and humidity sensor). When the vibration sensor detects abnormal vibration (for example, the signboard is illegally disassembled or moved), or the environmental monitoring sensor detects that the environmental parameters (such as temperature and humidity) exceed the preset safety range, the signboard will immediately start a self-destruction mechanism. The design of the self-destruction mechanism is to ensure that the sensitive information (such as the identity information of the industrial facility, environmental data, etc.) stored in the signboard cannot be illegally obtained. The self-destruction mechanism can include erasing the data stored in the signboard, damaging the two-dimensional code display module or making the signboard enter an unrecoverable locked state. The triggering condition of this mechanism is determined by a preset threshold, for example, the threshold of the vibration sensor can be set according to the vibration intensity during normal operation, and once the vibration exceeding the threshold is detected, it is identified as illegal disassembly behavior.

[0050] Based on the operation data of the inspection robot, the inspection robot is predicted for failure through a long short-term memory (LSTM) model. LSTM is a special recurrent neural network (RNN) that can effectively process time series data and capture long-term dependencies in the data. The inspection robot will collect and transmit various operation data in real time during operation, including but not limited to motor speed, battery power, sensor readings, motion trajectory, etc. These data are stored in the database module of the system and used as input for the LSTM model. The LSTM model analyzes these time series data to learn the feature patterns of the robot under normal operation and failure conditions. When the model detects a pattern related to failure in the data, it predicts the time point when the failure may occur and sends an alarm to the collaborative control module at a preset time (e.g. 30 minutes) before the time point. This early warning mechanism provides enough time for maintenance personnel to take preventive measures, such as pre-maintenance of the robot or adjustment of task allocation, to avoid interruption or delay of the inspection task due to robot failure.

[0051] The self-destruction mechanism of the intelligent signboard and the failure prediction function based on the LSTM model provide strong security protection and reliability guarantee for the inspection system of industrial facilities, ensuring efficient and safe operation of the system.

[0052] According to an embodiment of the present application, the parameters of the inspection system are optimized, including: based on the temperature data, vibration frequency and appearance defects of the inspection robot, obtaining the equipment health degree of the inspection robot, and planning the maintenance frequency of the inspection robot according to the equipment health degree; The inspection map is divided into regions with different risks, including low-risk, medium-risk and high-risk. The intelligent path planning unit of the collaborative control module avoids high-risk areas and reduces the frequency of passing through medium-risk areas when generating the planned path.

[0053] In this embodiment, the data analysis optimization module analyzes the operation data of the inspection robot to obtain the equipment health degree, and optimizes the maintenance frequency and inspection path of the inspection robot accordingly. The data analysis optimization module is also responsible for dividing the inspection map into regions with different risks, and guiding the intelligent path planning unit to avoid high-risk areas and reduce the frequency of passing through medium-risk areas when generating the planned path.

[0054] In order to ensure the security, integrity and traceability of the data, the database module uses a blockchain-based inspection database, specifically the Hyperledger Fabric framework. The data analysis optimization module develops a preventive maintenance model to evaluate the equipment health degree of the inspection robot. The model is based on three key indicators of the inspection robot: temperature data, vibration frequency and appearance defects, and calculates the equipment health degree by weighted summation. The specific formula is: Device health = Σ (α1*temperature trend + β1*vibration frequency + γ1*appearance defects) Wherein, α1, β1, γ1 are the first coefficient, the second coefficient, the third coefficient respectively.Temperature trend reflects the temperature change during the robot running process, vibration frequency represents the vibration intensity received by the robot during the running process, and appearance defects identify the shell damage, missing parts and other problems of the robot through visual detection technology.

[0055] Through the evaluation of the device health, the data analysis optimization module can plan the maintenance frequency of the patrol robot according to the health level.For example, when the device health is below a certain threshold, the system will suggest increasing the maintenance frequency and performing early inspection and repair on the robot to avoid the interruption of the patrol task caused by equipment failure.This data-based preventive maintenance strategy can effectively prolong the service life of the robot and improve the operation efficiency of the system.

[0056] The data analysis optimization module is also responsible for dividing the patrol map into risk areas.According to the environmental conditions of industrial facilities, equipment status, historical failure data and other factors, the patrol area is divided into three levels of low risk, medium risk and high risk.Based on the results of regional risk division, the intelligent path planning unit of the collaborative control module will avoid high-risk areas when generating the planned path and reduce the frequency of passing through medium-risk areas.The path planning unit will dynamically adjust the planned path based on real-time device health data and regional risk information to ensure that the robot completes the patrol task in a safe area.In addition, the data analysis optimization module will generate path optimization suggestions every month, automatically update the risk area division in the patrol map, and adjust the path planning strategy to adapt to the changes in the industrial environment.

[0057] Through the multiple functions of the data analysis optimization module, the invention realizes the intelligentization, safety and efficient operation of the patrol system, providing strong support for the patrol of industrial facilities.

[0058] According to an embodiment of the present application, the patrol map stored by the database module is generated using digital twin modeling technology; Before the patrol robot patrols according to the planned path, the collaborative control module simulates each patrol robot according to the patrol plan through the digital twin system, and when the simulation result confirms that the plan is risk-free, the patrol robot patrols according to the planned path.

[0059] In this embodiment, the inspection map stored in the database module is generated using digital twin modeling technology. Digital twin modeling constructs a virtual model highly consistent with the physical world through high-precision three-dimensional modeling and semantic annotation, which is used to simulate and optimize the task execution of the inspection robot. The following steps are included: Step 1, laser radar scanning generates point cloud map. The laser radar (LiDAR) is used to scan the industrial facility to generate a high-precision point cloud map. The laser radar can capture three-dimensional information of the environment with millimeter-level precision, and the generated point cloud map has a precision of ±2 centimeters. This high-precision map provides basic data for subsequent modeling and simulation. Step 2, import BIM model for semantic annotation. After generating the point cloud map, it is combined with the building information model (BIM). The BIM model contains detailed design information of the industrial facility, such as equipment type, safety level, etc. By importing the BIM model into the point cloud map, the system can perform semantic annotation on the map, giving each device and area specific attributes and functions. For example, the positions and types of transformers, switch cabinets, and other equipment are labeled, as well as the safety levels of different areas. Step 3, construct a simulation environment driven by a physical engine. After completing the semantic annotation, the system constructs a simulation environment based on a physical engine. The physical engine can simulate real physical interactions such as collision, friction, etc., supporting collision prediction and path planning. Through this simulation environment, the system can pre-act the task execution process of the inspection robot, detect potential collision risks, and optimize path planning.

[0060] Before the inspection robot follows the planned path for inspection, the cooperative control module will perform task pre-act through the digital twin system to ensure the safety and feasibility of the task. Before the start of the inspection task, the digital twin system will perform pre-act according to the planned path. During the pre-act process, the system will perform collision detection to ensure that the robot will not collide with equipment or obstacles during task execution. At the same time, the system will estimate the energy consumption of the task and calculate the amount of electricity required for the robot to complete the task, ensuring that the robot has enough electricity to complete the task. In addition, the system will also evaluate the safety factor of the task, taking into account the risk level of the path and the health status of the robot. When the inspection robot starts moving according to the planned path, the system will perform cooperative tracking through the camera cluster. The camera cluster processes and transmits data through 5G MEC (Multi-Access Edge Computing) technology to ensure real-time monitoring of the robot's position and state. The robot updates its pose data every 200 milliseconds, which is obtained through UWB (Ultra-Wideband Positioning) and visual fusion technology, ensuring positioning accuracy within centimeter-level range. This real-time monitoring and data updating mechanism ensures that the robot remains safe and efficient during task execution.

[0061] Through digital twin modeling and dynamic task execution technology, the invention realizes the intelligentization, safety and efficient operation of the inspection system, providing strong support for the monitoring of industrial facilities.

[0062] Example 2 Figure 2 This is a flowchart illustrating a remote camera and robot linkage inspection method according to one embodiment of the present invention. Figure 2 As shown, according to one embodiment of the present invention, a patrol method involving a remote camera and a robot, employing any of the remote camera and robot linkage patrol systems of the present invention, includes the following steps: Step S102: Load the inspection map generated based on digital twin modeling technology, deploy the UWB ultra-wideband positioning base station network and smart signage, start the self-test program of each module and synchronize the initial parameters; Step S104: Obtain the positioning coordinates of each patrol robot and generate an initial planned path for each patrol robot; Step S106: The initial planned path is verified through virtual simulation using a digital twin system to obtain the planned path; In step S108, each inspection robot inspects the industrial facilities based on its corresponding planned path; the remote camera monitoring module controls the remote camera to collect information from the inspection robots and industrial facilities according to the planned path sent by the collaborative control module. In step S110, the collaborative control module obtains the inspection results based on the information collected by the remote camera and the inspection robot.

[0063] This embodiment proposes a remote camera and robot-linked inspection method. This method is based on a highly intelligent and automated inspection system, utilizing advanced technologies such as digital twin technology, UWB ultra-wideband positioning, intelligent signage, and collaborative control modules to achieve efficient and safe inspection of industrial facilities. The inspection method specifically includes: Before the inspection mission begins, the inspection system first performs initialization and deployment operations. This includes loading an inspection map generated based on digital twin modeling technology. This map is generated as a high-precision point cloud map using LiDAR scanning and semantically annotated using a BIM model, including information such as equipment type and security level. The map's accuracy reaches 0.1 meters, providing detailed environmental information for the inspection robot. A UWB ultra-wideband positioning base station network is deployed within the monitoring area, with a base station spacing of less than 50 meters to ensure positioning accuracy within ±10 centimeters. Simultaneously, smart signs are installed at various industrial facilities. These signs integrate temperature and humidity sensors, vibration sensors, and display relevant equipment information via dynamic QR codes. A self-test program is initiated to check the functionality of each module (such as the remote camera monitoring module, robot inspection module, and collaborative control module). The self-test program also synchronizes initial parameters to ensure smooth communication and collaboration between modules.

[0064] After the system initialization is completed, the intelligent path planning unit of the collaborative control module begins to work. Based on the patrol map, the location information of the industrial facilities to be patrolled, and the current position information of each patrol robot, the unit generates an initial planning path for each patrol robot. During path planning, the intelligent path planning unit uses the PPO reinforcement learning algorithm to consider factors such as time, energy consumption, and safety to generate the optimal path. At the same time, the system also generates a set of backup paths to deal with adverse weather or unexpected situations.

[0065] After generating the initial planning path, the collaborative control module performs virtual simulation verification on these paths through the digital twin system. During the simulation process, the simulated robot moves along the planned path, and collision detection, energy consumption estimation, and safety factor evaluation are performed. The digital twin system synchronizes the physical environment and the virtual model in real time through the simulation environment driven by the physics engine, ensuring the accuracy of the simulation results. If the simulation results show that the path has risks (such as collision risks or excessive energy consumption), the intelligent path planning unit will regenerate the planning path until the simulation results confirm that the path is risk-free.

[0066] After the planning path passes the simulation verification, each patrol robot starts to patrol the industrial facilities according to the corresponding planning path. During the movement of the robot, the UWB tag and visual two-dimensional code recognition unit are used to obtain real-time position information, and the IMU inertial unit is used for navigation correction to ensure that the positioning accuracy reaches centimeter level (±2 cm). At the same time, the remote camera monitoring module controls the remote camera to cooperate with the collection of information about the patrol robot and the industrial facilities according to the planning path sent by the collaborative control module. The remote camera transmits real-time high-definition video streams through the 5G network, and the collaborative control module analyzes the video streams to identify information such as robot pose, obstacles, and equipment status.

[0067] During the patrol process, the collaborative control module generates patrol results by integrating the information collected by the remote camera and the information collected by the patrol robot. The patrol results include device status, environmental parameters, robot health status, and other information. These information is stored in the inspection database through blockchain technology to ensure data security and traceability.

[0068] If abnormal situations (such as equipment failure or robot deviation from the path) are found during the patrol process, the collaborative control module will take appropriate measures according to the pre-set safety protection strategy. For example, when the robot deviates from the path beyond the pre-set threshold, the system will send automatic deviation correction instructions or regenerate the planning path; when illegal disassembly or environmental abnormalities are detected on the intelligent signboard, the signboard will activate the self-destruction mechanism to ensure information security.

[0069] The present application generates a high-precision three-dimensional semantic map through digital twin modeling technology, combines UWB ultra-wideband positioning and intelligent signboards, and realizes efficient and safe inspection of industrial facilities. The present application uses reinforcement learning algorithm for dynamic path planning, and verifies the safety and feasibility of the path through digital twin system for virtual simulation, ensuring the safety and feasibility of the path. During the inspection process, the system monitors and analyzes the information collected by the remote camera and the robot in real time, generates detailed inspection results, and stores the data using blockchain technology to ensure the safety and traceability of the information. In addition, the present application also has an intelligent safety protection mechanism, which can automatically respond to abnormal situations, further improving the reliability and intelligent level of the system.

[0070] According to an embodiment of the present application, when the cooperative control module determines that the inspection site of the inspection robot is located in an indoor scene, the inspection system starts an indoor scene enhancement scheme; The indoor scene enhancement scheme includes: The inspection robot performs pedestrian perception based on radar detection, gait recognition and voiceprint recognition; When the inspection robot perceives a pedestrian, the distance between the inspection robot and the pedestrian is kept not less than a third preset distance; The inspection system includes an indoor lighting module, which controls the indoor lights to illuminate the inspection path of the inspection robot in advance, ensuring that the illumination intensity of the inspection path reaches 200 lux.

[0071] In this embodiment, an indoor scene enhancement scheme is proposed to improve the running efficiency, safety and human-machine interaction experience of the inspection robot in the indoor environment. In the indoor environment, the inspection robot needs to coexist with pedestrians (such as maintenance personnel and visitors) safely and efficiently. Therefore, the indoor scene enhancement scheme of the present application specially designs a pedestrian perception system to ensure that the robot can accurately identify and respond to the presence of pedestrians. This includes: deploying a 60GHz millimeter wave radar array on the robot to detect vital signs in the surrounding environment. Millimeter wave radar can penetrate clothing and light barriers, and can monitor the micro-movements of the human body, such as breathing and heartbeat, to accurately perceive the position and activity state of the pedestrian. In order to distinguish between maintenance personnel and visitors, the inspection robot is equipped with an advanced gait recognition algorithm. This algorithm analyzes the walking posture, pace frequency and gait characteristics of the pedestrian to accurately identify the identity of different personnel. The gait patterns of maintenance personnel and visitors are different, and through the gait recognition algorithm, the robot can quickly determine the identity of the pedestrian, so as to take appropriate interaction strategies. Combined with voiceprint recognition technology, the inspection robot can recognize specific voice commands such as "pause inspection". When the robot receives this command, it will immediately respond and pause the current task. The voiceprint recognition system analyzes the frequency, pitch and rhythm of the voice to ensure the accuracy and safety of the command.

[0072] Narrow spaces and complex layouts are common in indoor environments, which pose higher requirements for the navigation capabilities of robots. To ensure the safety and efficient navigation of robots in narrow spaces, a hybrid positioning system based on SLAM (Simultaneous Localization and Mapping) and RFID (Radio Frequency Identification) is constructed. SLAM technology enables robots to construct maps and locate themselves in real-time in unknown environments, while RFID tags provide accurate navigation reference points for robots. This hybrid positioning system can achieve centimeter-level positioning accuracy (±5 centimeters), ensuring precise navigation of robots in narrow spaces. The patrol robot is equipped with a flexible anti-collision structure that automatically retracts when the force exceeds 10 Newtons, protecting the robot and surrounding objects from damage. This design not only improves the safety of the robot, but also enhances its adaptability in complex environments. Social etiquette rules are added to the path planning to ensure that the robot maintains a distance of at least 0.6 meters from pedestrians. This rule not only conforms to human social habits, but also reduces potential conflicts in human-robot interactions, improving the efficiency of the robot.

[0073] To ensure that the robot can clearly perform tasks in an indoor environment, the invention introduces an indoor lighting module. The patrol system also includes an indoor lighting module that can control indoor lights to illuminate the robot's patrol path in advance. Before the robot enters a certain area, the lighting module automatically turns on the lights in that area, ensuring that the robot can clearly identify the surrounding environment. Based on the image from the remote camera, the intelligent lighting system can dynamically adjust the light intensity to maintain the illumination intensity of the patrol path at 200 lux. This illumination level meets the needs of the robot's vision system and does not cause discomfort to indoor personnel. Through intelligent lighting coordination, the efficiency and safety of the robot in indoor environments have been significantly improved.

[0074] The indoor scene enhancement scheme proposed by the invention solves the complex problems faced by the patrol robot in the indoor environment through a series of innovative technologies, providing an efficient, safe and intelligent solution for indoor patrol tasks.

[0075] Embodiment Three According to an embodiment of the present invention, a remote camera robot linkage patrol method comprises the following steps: Step one, dynamic identification deployment; To build an intelligent industrial safety protection system, a dual-mode dynamic safety identification system is deployed: an electronic ink screen terminal is used in the core area to generate a dynamic two-dimensional code containing two-dimensional code encoding rules through AES256 encryption technology, such as: substation ID@area ID@equipment ID@timestamp; the display content is automatically refreshed every minute, and a temperature and humidity sensor and a vibration monitoring module are integrated, which immediately triggers a security protocol when detecting environmental parameter abnormalities or signs of physical tampering, and performs a data storage erasure operation; intelligent signboards are placed at key nodes in the inspection site (such as equipment intervals), which generate a new verification code every 60 seconds using a time series encryption algorithm, and have a multi-parameter environmental sensing unit built-in to monitor temperature, humidity, and light intensity changes in real time, and when illegal disassembly or environmental threshold breaches occur, a self-destruction program is activated to ensure the security of the identification information. Through the deep integration of dynamic encryption, multi-source sensing, and active defense mechanisms, the system not only realizes real-time authentication and path tracing of equipment identity, but also builds a dual protection system at the physical and data layers, significantly improving the safety and control efficiency of industrial facilities.

[0076] Step two, build a patrol positioning system, and initialize the multi-modal positioning of the patrol robot; To build a high-precision industrial patrol positioning system, a multi-modal fusion positioning architecture is used to achieve centimeter-level spatial perception through three-layer technology collaboration: first, deploy a UWB ultra-wideband positioning base station network (base station spacing ≤ 50 meters) to build a global coordinate framework in the monitoring area, and integrate a UWB tag, a nine-axis IMU inertial unit, and a high-definition two-dimensional code recognition module into the patrol robot to form a multi-source sensing terminal; at the algorithm level, a dynamic weighted fusion model is built based on a deep learning framework: position = a * (UWB_data) + β * (visual_position) + γ * (INS_correction) where position is the robot position; a, β, and γ are the first, second, and third weights, respectively; UWB_data is the UWB positioning coordinate; visual_position is the visual positioning; and INS_correction is the inertial navigation correction; By real-time calibration of UWB's ±10cm absolute positioning, visual two-dimensional code's sub-centimeter relative correction, and INS's millimeter-level motion prediction, the final comprehensive positioning accuracy is ≤±2cm; the system has an intelligent switching mechanism built-in, which can seamlessly switch to a combined positioning mode of pure inertial navigation + visual SLAM when detecting GPS signal denial or environmental interference, ensuring continuous and stable positioning output capability in complex scenarios such as substation cable trenches and underground pipe corridors, providing a global high-precision spatial reference for autonomous patrol robots.

[0077] Step three, digital twin modeling Step 1: Laser radar scan generates point cloud map (accuracy ±2cm); Step 2: Import BIM model for semantic labeling (device type / safety level); Step 3: Build a physics engine-driven simulation environment (support collision prediction).

[0078] Based on laser radar point cloud scanning (accuracy ±2cm), a high-precision three-dimensional scene base is constructed, and a globally consistent digital map is generated through SLAM algorithm; the BIM building information model is imported into the virtual space, and the elements such as power equipment and pipeline valves are semantically labeled (including device type, voltage level, safety permission and other attribute information), forming a three-dimensional digital twin with engineering semantics; On this basis, the physical engine module is integrated to build a virtual simulation environment that can simulate fluid dynamics and rigid body collision, realize pre-collision detection and dynamics verification of robot motion trajectory. Further deploy AI-driven path planning system, which carries 0.1 meter level precision three-dimensional semantic map engine, uses deep reinforcement learning algorithm (such as PPO or SAC) for dynamic path optimization, establishes real-time data synchronization channel between digital twin and physical world, so that the robot can preform the inspection path in the virtual environment, and simultaneously calculate more than 20 process indicators such as motor torque, battery energy consumption and regional safety factor, and finally generate the optimal navigation strategy considering efficiency and safety.

[0079] Step four, intelligent path planning; The intelligent path planning system takes the inspection target point, real-time weather data and equipment operation priority as input, constructs a dynamic decision model through the proximal policy optimization (PPO) reinforcement learning algorithm. After training in the digital twin simulation environment, the model can generate the optimal navigation path considering time efficiency, energy consumption and spatial safety, and simultaneously output the backup path cluster and regional risk heat map (including water accumulation prediction, icing warning and other spatial annotations) in extreme weather scenarios such as heavy rain and strong wind. To support real-time decision-making, deploy edge intelligent sensing network on the inspection robot and fixed monitoring nodes, integrate neural network processing unit (NPU) on the camera end to realize local analysis of 4K@30fps video stream, use lightweight improved YOLOv8 detection architecture to realize multi-target detection, and complete tasks such as inspection robot body pose estimation, dynamic obstacle trajectory prediction and power equipment meter reading identification in parallel, providing millisecond-level environmental perception feedback for the path planning system, forming a closed-loop control architecture of "cloud digital twin planning-edge real-time sensing correction".

[0080] Step five, edge collaborative control; Step 1: Camera end deployment: Real-time analysis is achieved using Cambrian MLU270 chips; improved YOLOv8 model (with attention mechanism) is run; Step 2, robot end deployment; Adopt millimeter wave radar + TOF camera multi-sensor fusion; develop rain and fog compensation algorithm (can work at visibility <5m).

[0081] Multi-robot coordination mechanism, design distributed scheduling system based on 5G ultra-low latency (<10ms) communication, support multi-robot dynamic role allocation (leader-follower mode). Through the blockchain technology to realize the task log evidence, ensure the operation traceable.

[0082] Step six, dynamic task execution; Start the task before the digital twin system rehearsal (collision detection / energy consumption estimation); when the robot moves along the path, the camera cluster cooperates with the tracking through 5G MEC, updates the pose data (UWB + vision fusion) every 200ms, and adopts the exception handling mechanism during task execution.

[0083] The exception handling mechanism includes: Detect the distance difference between the current actual position and the expected position through the deviation detection module; Judge whether the distance difference exceeds the second preset distance threshold, which is set to 15cm in this embodiment, and can be set to other values in actual application combined with specific scene requirements; If the distance difference does not exceed 15cm, automatically correct the deviation through the system; If the distance difference exceeds the second preset distance threshold, further judge whether there is an obstacle that cannot continue to patrol; if not, re-plan the path for the patrol robot; otherwise, brake the patrol robot urgently, and send an alarm prompt to the main control system.

[0084] Step seven, build a safety protection system; Adopt five protection mechanisms, including: 1, hardware TEE protection control instruction; 2, quantum key distribution (QKD) encrypted communication; 3, LSTM model predicts device failure (early warning 30min); 4, federated learning anomaly detection (multi-station cooperative training); 5, core module physical self-destruction design.

[0085] Step eight, data analysis optimization; Establish a patrol inspection database based on blockchain (Hyperledger Fabric), develop a preventive maintenance model, and the model uses the following formula: Device health = Σ(α1*temperature trend + β1*vibration frequency + γ1*appearance defect) Where, α1, β1, γ1 are the first, second and third coefficients, respectively; Monthly path optimization suggestions (automatically avoid high-risk areas).

[0086] According to an embodiment of the application, based on the initial positioning in step two, when the robot operation is in an indoor scene, an indoor scene enhancement scheme is further adopted; specifically including: A pedestrian perception system is constructed. A 60GHz millimeter wave radar array is deployed to detect vital signs; a gait recognition algorithm is developed (to distinguish between maintenance personnel and visitors); and a voiceprint recognition is combined (a specific instruction "pause inspection" triggers a response).

[0087] Narrow space navigation is performed. A SLAM+RFID hybrid positioning system (accuracy ±5cm) is constructed; a flexible anti-collision structure is developed (automatically retracts when the contact force is greater than 10N); and social etiquette rules are added to the path planning (maintain a distance of 0.6m from people).

[0088] Intelligent lighting coordination is performed. The indoor lighting system is linked to illuminate the robot path in advance; the light intensity is dynamically adjusted according to the camera image (maintain 200lux illumination).

[0089] The application utilizes technologies such as two-dimensional codes, cameras, and 3D modeling to enable the robot to perform inspection within a specified area and timely feedback the shooting results and the robot status. The application can improve the inspection efficiency, reduce the labor input, and also ensure the inspection quality and safety, and can be applied to various scenes that require inspection (such as substations).

[0090] The application can help enterprises and institutions to manage equipment more comprehensively and precisely, thereby improving the service life and management efficiency of the equipment and reducing the loss caused by equipment failure. The application can also provide more convenient and efficient inspection and management methods for various industries, thereby improving work efficiency and safety.

[0091] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described method can refer to the corresponding process in the foregoing system embodiments, which will not be described here.

[0092] The above description is only the preferred embodiment of the application and the description of the technical principles used. Those skilled in the art should understand that the scope of the application involved in the application is not limited to the technical solutions formed by the specific combination of the above technical features, and also covers other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features can be replaced with technical features disclosed in the application (but not limited to) having similar functions to form technical solutions.

[0093] It should be understood that the size of the serial number of the steps in the summary of the application and the embodiments of the application does not absolutely mean the order of execution, the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

Claims

1. A remote camera and robot linked patrol system, characterized by, Comprise: Remote camera monitoring module, robot inspection module, collaborative control module, security module, database module, data analysis optimization module; The remote camera monitoring module is used for remote monitoring and inspection of robots and industrial facilities through remote cameras; The robot inspection module is used for inspecting the industrial facilities through the inspection robot; The collaborative control module is used for processing and analyzing the data obtained by the remote camera and the inspection robot, and controlling the remote camera monitoring module and the robot inspection module based on the analysis results; The security module is used for encrypting and decrypting the data transmitted in each module; when a fault is detected in the inspection system, the security module is also used to trigger the corresponding security protection strategy based on the fault type; The database module is used to store the operation data of each module and the inspection map generated based on the actual working scene; The data analysis optimization module is used to optimize the parameters of the inspection system based on the data stored in the database module.

2. The remote camera and robot linked surveillance system of claim 1, wherein: The remote camera monitoring module uses a YOLOv8 model with attention mechanism to detect multiple targets in the collected images, including robot pose, obstacles, and equipment status; An environment perception unit is provided on the inspection robot, which collects surrounding environment data of the inspection robot and transmits it to the robot inspection module, which uses a rain and fog compensation algorithm to judge the current operating environment.

3. The remote camera and robot linked surveillance system of claim 1, wherein, The inspection system also includes a dynamic safety identification module and a positioning module; The dynamic safety identification module includes intelligent signboards deployed at each industrial facility, which display relevant data of the corresponding industrial facility through dynamic QR codes; a QR code recognition unit is provided on the inspection robot to recognize the intelligent signboards; The positioning module includes a UWB ultra-wideband positioning base station network deployed in the monitoring area, a UWB tag and an IMU inertial unit deployed on the inspection robot; the positioning module obtains the positioning coordinates of the inspection robot through UWB positioning coordinates, robot vision positioning coordinates obtained by the remote camera, and inertial navigation correction provided by the IMU inertial unit.

4. The remote camera and robot linked surveillance system of claim 3, wherein, The collaborative control module includes an intelligent path planning unit and a control unit; The intelligent path planning unit generates a planned path for each inspection robot based on the inspection map, the location information of the industrial facilities to be inspected, and the location information of each inspection robot; The control unit controls the inspection robot to inspect the industrial facilities based on the planned path; the control unit also controls the corresponding remote camera to remotely monitor the inspection robot based on the planned path and the positioning coordinates of the inspection robot.

5. The remote camera and robot linked surveillance system of claim 4, wherein, Triggering the corresponding security protection strategy based on the fault type includes: When the positioning coordinates of the target patrol robot deviate from the preset planning path by more than a first preset distance threshold and less than a second preset distance threshold, an automatic deviation correction instruction is sent to the target patrol robot, and the target patrol robot automatically corrects its position; When the positioning coordinates of the target patrol robot deviate from the preset planning path by more than the second preset distance threshold, the intelligent path planning unit re-generates a planning path for the target patrol robot; When the target patrol robot encounters an obstacle on the corresponding planning path and cannot continue to patrol, the target patrol robot brakes urgently and sends an alarm prompt to the cooperative control module.

6. The remote camera and robot linked surveillance system of claim 3, wherein, Based on the fault type, the corresponding safety protection strategy is triggered, including: when it is detected that the target intelligent signboard is illegally disassembled or the environment is abnormal, the target intelligent signboard starts a self-destruction mechanism; Based on the operation data of the patrol robot, the LSTM model is used to predict the fault of the patrol robot, and an alarm prompt is sent to the cooperative control module at a preset time before the predicted time point of the fault of the patrol robot.

7. The remote camera and robot linked surveillance system of claim 3, wherein, The parameters of the patrol system are optimized, including: Based on the temperature data, vibration frequency and appearance defects of the patrol robot, the device health degree of the patrol robot is obtained, and the maintenance frequency of the patrol robot is planned according to the device health degree; The patrol map is divided into risk level areas, including low risk, medium risk and high risk; the intelligent path planning unit of the cooperative control module avoids high risk level areas and reduces the frequency of passing through medium risk level areas when generating a planning path.

8. The remote camera and robot linked surveillance system of claim 1, wherein: The patrol map is generated by using digital twin modeling technology; Before the patrol robot patrols according to the planning path, the cooperative control module simulates each patrol robot according to the patrol plan through the digital twin system, and when the simulation result confirms that the plan is risk-free, the patrol robot patrols according to the planning path.

9. A method for remote camera and robot linkage patrol, using the remote camera and robot linkage patrol system according to any one of claims 1-8, characterized in that, The method includes: Load the patrol map generated based on digital twin modeling technology, deploy the UWB ultra-wideband positioning base station network and intelligent signboard, start the self-checking program of each module and synchronize the initial parameters; Obtain the positioning coordinates of each patrol robot and generate an initial planning path for each patrol robot; Verify the initial planning path through virtual simulation by the digital twin system to obtain a planning path; Each patrol robot patrols the industrial facilities based on its corresponding planning path; the remote camera monitoring module controls the remote camera to cooperate with collecting information of the patrol robot and the industrial facilities according to the planning path sent by the cooperative control module; The cooperative control module obtains the patrol result based on the information collected by the remote camera and the information collected by the patrol robot.

10. The remote camera and robot linked patrol method of claim 9, wherein: When the cooperative control module determines that the patrol site of the patrol robot is located in an indoor scene, the patrol system starts an indoor scene enhancement scheme; The indoor scene enhancement scheme includes: The patrol robot performs pedestrian perception based on radar detection, gait recognition and voiceprint recognition; When the patrol robot senses the pedestrian, a distance between the patrol robot and the pedestrian is kept not less than a third preset distance; The patrol system comprises an indoor lighting module, the indoor lighting module controls indoor light to illuminate the patrol path of the patrol robot in advance, and ensures that the illumination intensity of the patrol path reaches an illumination intensity threshold.

Images