Pipe gallery inspection robot system based on virtual reality and control method

Abstract

The invention provides a pipe gallery inspection robot system based on virtual reality and a control method, and belongs to the technical field of robot control. According to the pipe gallery inspection robot system based on virtual reality and the control method, a control module is connected with an environment sensing module, a virtual reality construction module, a display module and a fault recognition module; the environment sensing module is used for collecting environment information of the pipe gallery; the fault identification module is used for identifying fault information in the pipe gallery; the control module is used for determining target display data based on the environment information and the fault information; the virtual reality construction module is connected with the environment sensing module and the display module; the virtual reality construction module is used for constructing a virtual environment based on the environment information and the target display data; and the display module is used for displaying the virtual environment constructed by the virtual reality construction module. The problem that an existing AI inspection robot system is difficult to completely meet the increasing complex inspection requirements can be solved.

Description

Technical Field

[0001] The present disclosure relates to the field of robot control technology, and in particular to a pipe gallery inspection robot system and a control method based on virtual reality. Background Art

[0002] At present, with the acceleration of urbanization, the inspection demand for key infrastructure such as underground pipe corridors, tunnels and power facilities is increasing. Traditional manual inspection methods have many limitations, especially in complex environments and high-risk areas, such as underground pipe networks, power transmission lines or hazardous chemical storage areas. These environments are not only complex, but also potentially dangerous. Manual inspections are time-consuming and have safety hazards. In addition, manual inspections are insufficient in terms of data acquisition accuracy, real-time performance and comprehensive data analysis capabilities, making it difficult to effectively respond to the multi-dimensional data monitoring needs of complex equipment.

[0003] In recent years, with the rapid development of artificial intelligence (AI) and robotics, AI inspection robots have gradually become an effective solution to replace manual inspections. Robots have the ability to conduct autonomous inspections, can realize remote control, multi-sensor fusion monitoring, and synchronize inspection data to the control system in real time through wireless data transmission technology (such as 5G, industrial Wi-Fi). However, the existing AI inspection robot system still has some shortcomings. For example, it is difficult for existing robots to combine multi-dimensional environmental data in real time to build high-precision virtual scenes, resulting in inspection results that are not intuitive enough, and it is difficult to fully meet the growing complex inspection needs. Summary of the invention

[0004] The disclosed embodiments provide a pipe gallery inspection robot system and control method based on virtual reality to solve the problem that the existing AI inspection robot system is difficult to fully meet the growing complex inspection needs.

[0005] In a first aspect, an embodiment of the present disclosure provides a pipe gallery inspection robot system based on virtual reality, comprising: An environment perception module, a virtual reality building module, a display module, a control module and a fault identification module; the control module is connected to the environment perception module, the virtual reality building module, the display module and the fault identification module respectively; The environment perception module is used to collect the environment information of the pipe gallery; the fault identification module is used to identify the fault information in the pipe gallery; the control module is used to determine the target display data based on the environment information and the fault information; The virtual reality construction module is connected to the environment perception module and the display module respectively; the virtual reality construction module is used to construct a virtual environment based on the environment information and the target display data; the display module is used to display the virtual environment constructed by the virtual reality construction module.

[0006] In an exemplary embodiment of the present disclosure, the environmental information includes sensor data and image data; The environment perception module comprises: A sensor unit and a camera unit; The sensor unit and the camera unit are both connected to the control module; The sensor unit is used to collect sensor data of the pipeline gallery, and the camera unit is used to collect image data of the pipeline gallery.

[0007] In an exemplary embodiment of the present disclosure, the fault identification module includes: Environmental fault identification unit and appearance fault detection unit; The environmental fault identification unit and the appearance fault detection unit are both connected to the control module; The environmental fault identification unit is used to detect environmental abnormality information in the pipe gallery, and the appearance fault detection unit is used to detect appearance fault information of equipment and buildings in the pipe gallery.

[0008] In an exemplary embodiment disclosed herein, the pipe gallery inspection robot system based on virtual reality further includes: an alarm module; The alarm module is connected to the control module; The alarm module is used for generating an alarm based on the control instruction of the control module.

[0009] In an exemplary embodiment disclosed herein, the pipe gallery inspection robot system based on virtual reality further includes: a behavior recognition module; The behavior recognition module is connected to the control module; The behavior recognition module is used to detect the behavior information of people in the tunnel.

[0010] In an exemplary embodiment of the present disclosure, the behavior recognition module includes: Face recognition unit and behavior recognition unit; The face recognition unit is connected to the control module and the behavior recognition unit respectively; The behavior recognition unit is connected to the fault recognition module; The face recognition unit is used to recognize the identity information of people in the tunnel, and the behavior recognition unit is used to recognize the behavior information of people in the tunnel.

[0011] In an exemplary embodiment disclosed this time, the pipe gallery inspection robot system based on virtual reality further includes: a virtual touch interaction module; The virtual touch interaction module is connected to the control module; The virtual tactile interaction module is used to collect contact information between the inspection robot and the pipe gallery, and transmit the contact information to the tactile interaction device through the control module.

[0012] In an exemplary embodiment disclosed herein, the pipe gallery inspection robot system based on virtual reality further includes: an equipment inspection module; The equipment inspection module is connected to the control module; The equipment inspection module is used to detect the working status and health status of the equipment in the pipe gallery.

[0013] In an exemplary embodiment of the present disclosure, determining target display data based on the environment information and the fault information includes: determining a first area and a second area based on the environmental information; determining a third area based on the fault information; determining an environmental feature and a first data volume based on the environmental information of the first area; determining first display data based on the environmental feature and the first data amount; determining a second data amount based on the environmental information of the second area; determining second display data based on the environment information of the second area and the second data amount; determining a third data volume based on the environmental information of the third area; determining third display data based on the environment information of the third area and the third data amount; The second data amount is greater than the first data amount, and the second data amount is less than or equal to the third data amount; Target display data is determined based on the first display data, the second display data, and the third display data.

[0014] In a second aspect, an embodiment of the present disclosure provides a control method for a pipe gallery inspection robot system based on virtual reality, which is applied to the pipe gallery inspection robot system based on virtual reality in the first aspect above. The control method includes: determining a target operating mode based on an external command signal; The working states of the environment perception module, the virtual reality building module, the display module and the fault identification module are determined based on the target working mode.

[0015] The beneficial effects of the pipe gallery inspection robot system and control method based on virtual reality provided by the embodiments of the present disclosure are: The multi-dimensional environmental information collected by the environmental perception module is combined with the fault information of the fault identification module, and after being processed by the control module, accurate data is provided to the virtual reality construction module. This allows the constructed virtual environment to highly realistically reflect the actual situation of the pipeline corridor, whether it is the operating status of the equipment, changes in environmental parameters or potential fault hazards, all can be intuitively presented on the display module. Operators can observe every detail in detail, effectively solving the problem of the difficulty of comprehensive monitoring of the complex pipeline corridor environment, and greatly improving the accuracy of the grasp of the pipeline corridor status.

[0016] The modules work together to cope with the increasingly diverse and complex inspection needs in the tunnel. Through the real-time updated virtual environment, relevant areas and information can be displayed according to different inspection tasks. At the same time, the intelligent analysis of the fault identification module can quickly process large amounts of data, discover potential problems in a timely manner, and adapt to the higher inspection requirements brought about by the continuous development of the tunnel. Based on rich and accurate information, operation and maintenance personnel can also make more scientific and reasonable decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 It is a structural schematic diagram of a pipe gallery inspection robot system and a control method based on virtual reality provided in an embodiment of the present disclosure; Figure 2 It is a structural schematic diagram of another pipe gallery inspection robot system and control method based on virtual reality provided in an embodiment of the present disclosure; Figure 3 It is a structural schematic diagram of another virtual reality-based pipeline corridor inspection robot system and control method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] In order to enable people in the technical field to better understand the present solution, the technical solution in the embodiment of the present solution will be clearly described below in conjunction with the drawings in the embodiment of the present solution. Obviously, the described embodiment is an embodiment of a part of the present solution, not all of the embodiments. Based on the embodiments in the present solution, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present solution.

[0020] The term "including" and any other variations in the specification and claims of this solution and the above drawings mean "including but not limited to", and is intended to cover non-exclusive inclusions and is not limited to the examples listed in the text. In addition, the terms "first" and "second" are used to distinguish different objects, not to describe a specific order.

[0021] The following is a detailed description of the implementation of the present disclosure in conjunction with the specific drawings: Figure 1 A schematic structural diagram of a pipe gallery inspection robot system and control method based on virtual reality provided in an embodiment of the present disclosure.

[0022] Reference Figure 1 The pipeline corridor inspection robot system and control method based on virtual reality include: an environment perception module, a virtual reality construction module, a display module, a control module and a fault identification module. The control module is connected to the environment perception module, the virtual reality construction module, the display module and the fault identification module respectively.

[0023] The environment perception module is used to collect the environment information of the pipe gallery. The fault identification module is used to identify the fault information in the pipe gallery. The control module is used to determine the target display data based on the environment information and fault information.

[0024] The virtual reality building module is connected to the environment perception module and the display module respectively. The virtual reality building module is used to build a virtual environment based on environmental information and target display data. The display module is used to display the virtual environment built by the virtual reality building module.

[0025] In this embodiment, the environment perception module may include multiple types of acquisition devices, including multiple sensors and camera devices. Specifically, multiple sensors can collect environmental information such as temperature, humidity, and gas composition in the tunnel, and the camera device can collect the appearance of the equipment. The environment perception module can transmit the collected information to the control module.

[0026] The fault identification module can receive the environmental information sent by the control module and perform in-depth analysis on the environmental data. For example, the environmental information can be input into a pre-established pipe gallery fault identification model to identify fault information such as pipeline leakage and equipment failure in the pipe gallery. After the identification is completed, the fault information can be sent to the control module. In this embodiment, a pipe gallery fault identification model can be constructed using a deep learning algorithm.

[0027] The control module integrates the environmental information and the fault information to determine the target display data. The target display data may be a focus area, for example, a fault area or an abnormal environmental parameter area.

[0028] The virtual reality construction module obtains environmental information and target display data, and uses 3D modeling technology to build a virtual pipe gallery environment. In the virtual pipe gallery environment, different areas and equipment can be presented differently based on data, such as marking faulty equipment with special colors and displaying abnormal environmental parameter changes with dynamic effects.

[0029] The display module can present the constructed virtual environment to the operator, who can then control the inspection robot for further inspection and operation.

[0030] For example, during the inspection of a petrochemical pipeline corridor, the environmental perception module detected that the temperature in a certain section of the corridor was elevated and there were signs of flammable gas leakage. At the same time, the fault identification module determined that there might be a leakage fault at a certain pipeline connection. After the control module determines the target display data, the virtual reality construction module marks the pipeline in the area with red flashing in the virtual environment, and displays the real-time temperature and gas concentration data. After seeing the display, the operator controls the inspection robot to approach for detailed inspection, further determine the fault situation, and take timely measures, such as closing relevant valves, starting ventilation equipment, etc., to prevent accidents and ensure the safe operation of the corridor and the safety of the surrounding environment.

[0031] In this embodiment, determining the target display data based on the environment information and the fault information includes: The first area and the second area are determined based on the environmental information.

[0032] The third area is determined based on the fault information.

[0033] An environmental feature and a first data volume are determined based on the environmental information of the first area.

[0034] The first display data is determined based on the environmental characteristic and the first data amount.

[0035] The second data amount is determined based on the environmental information of the second area.

[0036] The second display data is determined based on the environmental information of the second area and the second data amount.

[0037] A third data amount is determined based on environmental information of the third area.

[0038] The third display data is determined based on the environmental information of the third area and the third data amount.

[0039] The second data amount is greater than the first data amount, and the second data amount is less than or equal to the third data amount.

[0040] Target display data is determined based on the first display data, the second display data, and the third display data.

[0041] In this embodiment, the priority of the third area is higher than that of the second area, and the priority of the second area is higher than that of the first area. The priority is determined based on the importance of different areas. For areas with high priority, more data can be collected and transmitted to the virtual reality construction module for virtual environment construction. For areas with low priority, a small amount of data can be transmitted to the virtual reality construction module to meet basic display requirements, or the data can be processed by screening, compression, feature extraction, etc., and only the processed data is transmitted to reduce resource usage.

[0042] In this embodiment, the environmental information may include physical parameters such as temperature, humidity, gas concentration, light intensity, equipment operating status (such as vibration frequency, current, etc.) in the tunnel, as well as visual information such as the internal structure of the tunnel and equipment appearance, which is used to fully understand the environmental conditions of the tunnel and provide a data basis for the construction of the virtual environment.

[0043] Fault information can include parameters such as the type of equipment fault in the corridor, the coordinates of the fault location, and the severity of the fault, to clarify the abnormal conditions in the corridor.

[0044] The first area, the second area, and the third area are different pipe gallery areas divided according to environmental information and fault information. The first area can be a background environment, such as an area without key structures and devices and where the frequency of environmental information changes is less than the first information change rate. The second area may include a key inspection area where there are key structures and devices and the frequency of environmental information changes is greater than or equal to the first information change rate; the third area may include a key area where a fault exists.

[0045] The first data volume, the second data volume and the third data volume are data transmission volumes required for building a virtual environment. Different area types correspond to different data transmission volumes.

[0046] The first display data, the second display data and the third display data are virtual environment data to be finally presented in the first area, the second area and the third area respectively.

[0047] The target display data includes first display data, second display data, and third display data.

[0048] The priority can be determined based on the importance of the area to the overall operational safety and functionality of the tunnel, reflecting the order of different areas in data processing and display and the resource allocation weight.

[0049] For example, in the inspection scenario of the urban underground integrated pipe corridor, the environmental perception module detects relevant data. In one section of the pipe corridor, the temperature is within the preset temperature range, the humidity is within the preset humidity range, and there are no key structures and devices. The frequency of environmental information changes is low, and this area is divided into the first area (background environment). In another section of the pipe corridor, there are key structures such as important power equipment, and the frequency of environmental information changes reaches the first information change rate, which is divided into the second area (key inspection area). There is a pipeline leakage fault in the pipe corridor, and the fault area is the third area.

[0050] The control module determines the corresponding data volume and display data according to the situation of each area. For the first area, the basic data of the first data volume is processed and determined to be transmitted as the first display data, and a control instruction is generated, such as controlling the display module to display a normal background in light gray based on the control instruction. For the second area, the filtered and compressed data is transmitted as the second display data. Since the third area has the highest priority, a large amount of detailed data is collected as the third display data to build a virtual environment. At the same time, the control module can generate additional information (such as flashing red and warning icons), and the additional information is used together to build the virtual environment of the third area to focus on displaying the fault situation. All display data are integrated into target display data to build a virtual environment, and then presented to the inspection personnel through the display module, so that they can quickly locate key areas and fault points and carry out inspection work efficiently.

[0051] In this embodiment, determining the environmental feature and the first data amount based on the environmental information of the first area may include: A first standard data amount is determined based on the first area.

[0052] A first data amount is determined based on the environmental information of the first area and a first standard data amount.

[0053] An environmental characteristic is determined based on the environmental information of the first area.

[0054] In this embodiment, the first standard data volume is a data processing volume standard set based on the conventional or expected demand of the first area, which is used to guide the data collection, processing and transmission of the area. The first standard data volume can be determined based on historical data. The first data volume is the actual data processing volume adjusted according to the environmental information of the first area and the first standard data volume to ensure effective allocation of resources. The environmental characteristics are key elements extracted from the environmental information of the first area, which are used to optimize the data processing strategy and serve as the data basis for building a virtual environment.

[0055] In this embodiment, determining the first data amount based on the environment information of the first area and the first standard data amount may include: An environmental information change rate is calculated based on the environmental information of the first area.

[0056] If the environmental information change rate is greater than or equal to the first change rate, a first adjustment step is determined based on the environmental information change rate.

[0057] The first standard data volume is adjusted based on the first adjustment step to obtain the first data volume.

[0058] In this embodiment, the environmental information change rate refers to the degree of change of environmental information (such as temperature, humidity, gas concentration, etc.) in the first area within a unit time, which can be obtained by calculating the ratio of the difference between the environmental information at adjacent time points to the environmental information at the previous time point. The environmental information change rate reflects the dynamic change characteristics of the environmental information.

[0059] The first change rate is a preset threshold value used to determine whether the environmental information has changed significantly. When the environmental information change rate is greater than or equal to the first change rate, it is considered that the environmental information has changed significantly.

[0060] The first adjustment step is an adjustment amount determined according to the rate of change of the environmental information, and is used to dynamically adjust the first standard data amount. The size of the adjustment step may be proportional to the rate of change of the environmental information.

[0061] For example, in the actual application of pipeline corridor inspection, a first standard data volume is set according to the routine inspection requirements in the pipeline corridor as the data collection benchmark for daily inspection. The environmental perception module monitors the environmental information in the pipeline corridor in real time, such as temperature, humidity and gas concentration, and calculates the change rate of these environmental information.

[0062] When it is detected that the temperature in a certain section of the pipe gallery rises sharply and its change rate exceeds the preset first change rate, the first adjustment step can be determined based on the environmental information change rate, and the first standard data volume can be increased based on the first adjustment step, so as to collect and transmit the environmental information in the section of the pipe gallery more frequently. In this way, the data collection volume can be dynamically adjusted according to actual inspection needs and environmental changes, which can improve inspection efficiency and resource utilization while ensuring data accuracy.

[0063] From the above, it can be concluded that this embodiment can enable inspection personnel to quickly focus on key areas and fault points through precise regional division and differentiated processing of data volume and display data, greatly improving inspection efficiency and reducing troubleshooting time. This embodiment allocates data processing and transmission resources based on priority, which can effectively avoid waste of system resources, ensure priority processing and display of important information, and improve overall operating performance.

[0064] This embodiment uses virtual reality to build a virtual environment, which allows inspection personnel to remotely and intuitively understand the status of the pipeline corridor, reducing the risk and cost of on-site inspections. When troubleshooting, it can provide operators with detailed and intuitive information in a timely manner, facilitate rapid decision-making and take measures, effectively ensure the safe operation of the pipeline corridor, reduce the probability of accidents and the environmental damage and economic losses they may cause, enhance the reliability and stability of pipeline corridor operation and maintenance, and improve the inspection robot's ability to cope with various complex situations in the pipeline corridor, meeting various complex inspection needs.

[0065] Figure 2 FIG. 1 is a schematic diagram of another virtual reality-based pipeline inspection robot system and control method provided by an embodiment of the present disclosure. Figure 2 As shown, in one embodiment disclosed herein, the environmental information includes sensor data and image data.

[0066] Environmental perception module, including: Sensor unit and camera unit.

[0067] The sensor unit and the camera unit are both connected to the control module.

[0068] The sensor unit is used to collect sensor data of the pipeline corridor, and the camera unit is used to collect image data of the pipeline corridor.

[0069] In this embodiment, the sensor unit may include a temperature sensor, a humidity sensor, a gas detection sensor, a pressure sensor, etc., to continuously collect corresponding physical parameter data in the pipe gallery. The camera unit may include an infrared camera to collect image data in the pipe gallery.

[0070] Exemplarily, when the inspection task begins, the inspection robot collects information about the tunnel environment through the environmental perception module, and transmits the data collected by the sensor unit and the camera unit to the control module to establish a digital model of the tunnel.

[0071] For example, temperature and humidity data collection:

[0072] in, is the measured value of temperature, is the measured value of humidity, and is the measurement error, is the sampling time, is the actual temperature value at time t, is the actual humidity value at time t.

[0073] Gas concentration data collection:

[0074] is the measured value of gas concentration, is the measurement error, is the actual gas concentration value at time t. The data collection of gas concentration is used to monitor whether there is harmful gas in the pipeline corridor.

[0075] The image data acquisition process includes infrared image and visible light images The collected image is denoised using the following formula:

[0076] in, is the initial image data collected, is the Gaussian noise of the initial image data, is the noise intensity, is the image data after denoising.

[0077] This embodiment can accurately collect the environment data of the pipe gallery and provide a comprehensive basis for the assessment of the pipe gallery status. The data collection formula can ensure the accuracy of information, improve the quality of data, and help to timely discover hidden dangers and ensure the safe and efficient operation of the pipe gallery.

[0078] like Figure 2 As shown, in one embodiment of the present disclosure, the fault identification module includes: Environmental fault identification unit and appearance fault detection unit.

[0079] The environmental fault identification unit and the appearance fault detection unit are both connected to the control module.

[0080] The environmental fault identification unit is used to detect environmental abnormality information in the pipe gallery, and the appearance fault detection unit is used to detect appearance fault information of equipment and buildings in the pipe gallery.

[0081] In this embodiment, the fault identification module uses a deep learning model to perform real-time analysis on the collected data. The environmental fault identification unit can receive the data collected by various sensors in the environmental perception module and input these data into the deep learning model to identify abnormal patterns in the data, such as abnormal temperature rise, abnormal changes in the concentration of harmful gases, etc., so as to determine whether there are hidden faults in the environment of the pipe gallery.

[0082] The appearance fault detection unit can receive the images or video data taken by the camera unit in the environmental perception module and processed by the control module, and input these images or video data into the deep learning model. The deep learning model extracts and analyzes the features of the equipment appearance and building structure in the image, and determines whether there is an appearance fault after comparing them with the normal appearance model.

[0083] For example, as shown in Table 1, Table 1 is a list of fault identification information. Among them, the environmental fault identification unit may include water accumulation detection, foreign body intrusion detection, smoke / fire detection, and the appearance fault detection unit may include wall crack detection, rust on the electric control cabinet, etc.

[0084] In terms of wall crack detection, once cracks are found in the wall, an alarm will be triggered immediately, and the specific location and length of the cracks will be accurately marked in the virtual reality (VR) environment. In addition, targeted reinforcement plans can be matched and generated, and the development and change data of the cracks can be continuously recorded to continuously monitor and evaluate the wall condition.

[0085] The water accumulation detection function can immediately sound an alarm when water accumulates in the pipe gallery, and clearly display the water accumulation area and its size in VR. The water accumulation detection function has extremely high detection sensitivity, and can promptly detect and sound an alarm when water droplets begin to form. At the same time, when water accumulation is detected, it will remind you to start the drainage device, but the execution of drainage measures still needs manual confirmation to ensure the accuracy and rationality of the drainage operation.

[0086] For foreign body intrusion detection, when unknown objects or organisms appear in the tunnel, a real-time alarm can be quickly issued, and relevant information about the intruder can be displayed in VR. The surrounding environment data can be comprehensively analyzed to accurately assess the risks that may be caused by foreign body intrusion, providing solid data support for taking corresponding countermeasures.

[0087] In terms of smoke and fire detection, an alarm can be issued based on the temperature matching calculation results, and the area where the smoke or fire occurs can be intuitively displayed on the VR interface. At the same time, a fire risk rating can be generated to help staff quickly understand the severity of the fire and start the fire extinguishing device urgently, but manual confirmation is also required to avoid unnecessary losses caused by misoperation.

[0088] For the rust problem of the electric control cabinet, when rust appears on the control cabinet, a rust expansion warning will be issued, and the rusted area will be clearly marked in VR. At the same time, the corresponding rust rating will be generated to prompt maintenance personnel to perform anti-rust treatment in time to ensure the normal operation and service life of the electric control cabinet.

[0089] Table 1 Fault identification information list

[0090] For example, regarding wall crack detection, the collected images can be processed based on a method combining image processing and deep learning. Analysis to detect the width and length of wall cracks:

[0091] in, is the width of the wall crack, is the length of the wall crack, It is an image recognition model.

[0092] Crack width alarm threshold: ≥5mm.

[0093] The prediction of crack growth trend can include linear regression based on historical data, defining the crack growth rate as .

[0094]

[0095] Among them, when When the crack size is ≥1mm / day, t is the time point of data measurement, and an expansion warning is issued. The crack location is marked in the VR scene, and a crack expansion alarm is issued to the maintenance team.

[0096] Crack propagation prediction can also include combining historical crack data and current images to predict crack propagation using a time series prediction model based on a long short-term memory network (LSTM):

[0097] in, is the crack characteristics at time t, is the historical data state, t is the time point of data measurement, is the predicted value of crack expansion at the next time point, LSTM It is a time series prediction model used to predict cracks.

[0098] For example, regarding the identification and early warning of water accumulation in pipe corridors, a water accumulation monitoring model can be constructed to monitor in real time whether there is water accumulation in the pipe corridor and issue an early warning before water accumulation forms.

[0099] This embodiment can accurately identify the environment and appearance faults of the pipe gallery, and the deep learning model improves the accuracy and efficiency of detection. Various detection functions will issue timely alarms and present details in VR, which is conducive to rapid positioning and processing. This embodiment can also generate response strategies for corresponding fault conditions, comprehensively assess risks, and assist manual decision-making, which can effectively ensure the safe and stable operation of the pipe gallery and reduce the hazards of faults and operation and maintenance costs.

[0100] In one embodiment disclosed herein, the virtual reality-based pipeline corridor inspection robot system also includes: an alarm module.

[0101] The alarm module is connected with the control module.

[0102] The alarm module is used to generate an alarm based on the control instruction of the control module.

[0103] In this embodiment, when the control module determines that there is an abnormal situation in the tunnel according to the information transmitted by the environment perception module, the fault identification module, etc., it will send a control instruction to the alarm module. After receiving the instruction, the alarm module starts the corresponding alarm action according to the preset alarm method. The alarm method may include sending a loud alarm and flashing warning lights at the tunnel site and the control center, sending text messages containing fault information to the mobile phones of relevant operation and maintenance personnel, and popping up a striking alarm window on the monitoring system interface to display detailed fault content, etc., to ensure that relevant personnel can know the abnormal situation in the tunnel in time.

[0104] For example, if the control module determines that the smoke concentration in the tunnel is increased and the temperature is abnormal based on the smoke / fire monitoring data collected by the environmental perception module, the control module determines that a fire may occur and immediately sends a command to the alarm module. The alarm module activates the sound and light alarm at the tunnel site, emitting a high-decibel alarm and strong flashing lights to remind people in the tunnel to evacuate quickly. At the same time, a text message notification is sent to the fire department and operation and maintenance management personnel to inform them of the location of the fire and the preliminary judgment of the fire situation so that fire-fighting measures can be taken in time.

[0105] The alarm module of this embodiment can ensure that personnel in the tunnel, the fire department and operation and maintenance personnel can know the abnormality in the first time through various means (sound and light alarm, SMS notification, pop-up alarm), improve the emergency response speed, and ensure the safety of personnel and tunnel facilities.

[0106] like Figure 2 As shown, in one embodiment disclosed this time, the pipeline corridor inspection robot system based on virtual reality also includes: a behavior recognition module.

[0107] The behavior recognition module is connected with the control module.

[0108] The behavior recognition module is used to detect the behavior information of people in the tunnel.

[0109] In this embodiment, the behavior recognition module includes: Face recognition unit and behavior recognition unit.

[0110] The face recognition unit is connected to the control module and the behavior recognition unit respectively.

[0111] The behavior recognition unit is connected to the fault recognition module.

[0112] The face recognition unit is used to identify the identity information of people in the tunnel, and the behavior recognition unit is used to identify the behavior information of people in the tunnel.

[0113] As shown in Table 2, Table 2 is a list of behavior recognition information. Table 2 details the relevant processing mechanisms of the behavior recognition module in the virtual reality-based corridor inspection robot system for different behavior recognition types.

[0114] Table 2 Behavior recognition information list

[0115] Among them, fall detection: when a fall is detected in the tunnel, the real-time alarm function is immediately activated, and the specific situation of the fall is intuitively presented in the virtual reality (VR) scene. At the same time, through comprehensive analysis of factors such as the personnel's past behavior patterns and the environment in which they are located, early warning of areas where there may be a risk of falling can be given so that corresponding preventive measures can be taken, such as improving lighting, clearing ground obstacles, etc., to ensure the safety of personnel walking.

[0116] Face detection: Use facial key point matching technology to identify the face information of people in the tunnel. When a stranger's face that is not successfully matched is detected, a real-time alarm is immediately issued and the stranger's face is highlighted in the VR display interface. According to the preset access permission rules, corresponding permission restriction measures are implemented on strangers, such as prohibiting them from entering specific areas or triggering further identity verification processes, thereby effectively maintaining the security and confidentiality of the tunnel area.

[0117] Dress code detection: Use image recognition technology to detect the wearing of helmets and other labor protection products by personnel in the tunnel. If it is found that a person is not wearing a standard, such as not wearing a helmet or not wearing labor protection products correctly, an alarm will be triggered in real time, and the relevant information of the person wearing the standard will be clearly displayed in the VR display interface. At the same time, the alarm module will also send a reminder to the person who is not wearing it in real time, prompting him to correct the dress problem in time to meet the safety regulations of the tunnel operation and reduce the probability of safety accidents.

[0118] Behavior norm detection: Monitor and identify prohibited behaviors in the pipe gallery, such as smoking and unprotected operation. When a person is detected to have violated such behavior norms, a real-time alarm is issued, and the person with irregular behavior is presented on the VR display interface. At the same time, the alarm module is used to promptly issue reminders to the person with irregular behavior, requiring him to immediately stop the illegal behavior, ensuring the safety and compliance of the pipe gallery working environment, and preventing safety accidents such as fire and casualties caused by illegal behavior.

[0119] For example, regarding fall detection, a deep learning model (YOLOv7) can be used to analyze a person's posture to detect whether someone has fallen in the tunnel.

[0120] Fall detection algorithm: Based on human posture estimation, define a set of posture features :

[0121] in, Represents the position sequence of multiple key points of the human body.

[0122] Through the posture estimation network, the posture features The data is mapped to a fall probability:

[0123] in, is the probability of falling, ) is the fall probability mapping function.

[0124] The fall alarm threshold can be set to ≥0.7, that is, when the probability of falling exceeds 0.7, an early warning is triggered, such as sounding an alarm and notifying the operator.

[0125] This embodiment can accurately identify the identity and behavior of personnel and ensure the safety of the pipe gallery area. This embodiment corrects violations in a timely manner through broadcasting and alarm reminders, effectively reduces safety accidents caused by human factors, improves the safety and standardization of pipe gallery operation and maintenance, and protects the life and health of personnel and the normal operation of pipe gallery facilities.

[0126] Figure 3 Schematic diagram of another virtual reality-based pipeline inspection robot system and control method provided by the embodiment of the present disclosure. Figure 3 As shown, in one embodiment disclosed this time, the pipeline corridor inspection robot system based on virtual reality also includes: a virtual touch interaction module.

[0127] The virtual touch interaction module is connected to the control module.

[0128] The virtual tactile interaction module is used to collect the contact information between the inspection robot and the pipeline corridor, and transmit the contact information to the tactile interaction device through the control module.

[0129] In this embodiment, the tactile interaction device may include a tactile feedback glove or the like.

[0130] For example, when an operator uses a virtual reality-based tunnel inspection robot system, the tactile feedback gloves can simulate the touch of equipment in the tunnel environment and provide a more realistic operating experience. For example, when an operator touches a device in a virtual environment, the tactile gloves will feedback the corresponding tactile information.

[0131] In this embodiment, during the interaction between the inspection robot and the tunnel environment, the virtual touch interaction module senses the contact force, contact position, contact area and other information between the robot and various objects in the tunnel (such as equipment surfaces, pipes, walls, etc.) through sensors, and transmits this contact information to the control module. After processing and converting the data, the control module transmits it to the touch interaction device (such as tactile feedback gloves). Based on the received information, the tactile feedback gloves use built-in micro motors, pressure sensors and other components to simulate the corresponding touch by generating vibrations and pressure changes of different intensities and frequencies, so that the operator can feel as if they are actually touching the objects in the tunnel.

[0132] For example, an operator wearing tactile feedback gloves operates a patrol robot in virtual reality to approach an electric control cabinet with a simulated fault. When the robot's mechanical arm contacts the electric control cabinet in the virtual environment, the virtual tactile interaction module collects and transmits the contact information. Based on this information, the tactile feedback gloves will produce a slight sense of resistance and texture touch on the operator's fingers, just like actually touching the metal shell and operation buttons of the electric control cabinet, so that the operator can more accurately judge the operation force and position, and improve the effect and realism of maintenance skills training.

[0133] This embodiment can accurately collect and transmit the contact information between the robot and the pipe gallery, so that the tactile feedback gloves can simulate the touch of the equipment and improve the authenticity of the operator's perception of the pipe gallery environment. This helps the operator to judge the robot's operating actions more accurately. For example, when repairing equipment, the force and position can be accurately grasped to reduce the probability of misoperation. At the same time, the enhanced sense of reality allows operators to respond to various situations more efficiently, such as quickly identifying the material and state of abnormal objects, thereby improving the accuracy and timeliness of inspections and ensuring the safety and stability of pipe gallery operations.

[0134] In one embodiment disclosed herein, the virtual reality-based pipeline corridor inspection robot system also includes: an equipment inspection module.

[0135] The equipment inspection module is connected to the control module.

[0136] The equipment inspection module is used to detect the working status and health of the equipment in the tunnel.

[0137] In this embodiment, the equipment inspection module includes: an RFID reading unit and an equipment detection unit.

[0138] The RFID reading unit is connected to the equipment detection unit and the control module respectively.

[0139] The RFID reading unit is used to identify the device QR code, and the device detection unit is used to detect the working status and health status of the device.

[0140] In this embodiment, when the inspection robot moves in the tunnel, the Radio Frequency Identification (RFID) reading unit uses visual technology to scan the QR code of the equipment in the tunnel. When the QR code is found, its image information is transmitted to the control module, which parses the QR code through image processing and recognition algorithms to obtain the unique identification information of the equipment, such as the equipment number and model. The identification information is fed back to the RFID reading unit, so that it is associated with the equipment detection unit and is ready to detect the equipment.

[0141] After receiving the command signal from the RFID reading unit, the equipment detection unit starts to detect the working status and health status of the equipment. The equipment detection unit can perform detection in a variety of ways, such as communicating with the equipment's own sensors to obtain operating data such as temperature, pressure, and flow, or using its own vibration sensors, infrared thermal imagers, etc. to detect the vibration, temperature distribution, etc. of the equipment. After the detection is completed, the equipment detection unit transmits the detection results to the control module, which associates these data with the unique identification information of the equipment and updates the status record of the equipment in the asset database to achieve real-time inventory of the equipment status.

[0142] For example, in the power pipeline corridor, the inspection robot inspects transformers, switch cabinets and other equipment. The RFID reading unit identifies the QR code on the transformer casing and obtains its equipment information and related working parameters (such as related parameter indicators and parameter thresholds). The equipment detection unit detects the oil temperature, winding temperature, and operating sound of the transformer by communicating with the transformer oil temperature sensor. When the oil temperature is found to be greater than the preset temperature threshold, the equipment detection unit transmits the data to the control module. While updating the status of the transformer in the asset database, the control module can also highlight the abnormal status of the transformer in the virtual environment through the virtual reality construction module, reminding the operation and maintenance personnel to deal with it in time to prevent failures.

[0143] This embodiment realizes the full life cycle management of the tunnel assets through automatic identification and status recording of the equipment ID, which helps to improve the efficiency of inspection and refine the asset management.

[0144] In one embodiment of the present disclosure, a method applied to a pipe gallery inspection robot system based on virtual reality includes: determining a target operating mode based on an external command signal; The working states of the environment perception module, virtual reality building module, display module and fault identification module are determined based on the target working mode.

[0145] In this embodiment, the external command signal is a specific command code or signal issued by an operator or a remote monitoring center. The external command signal may include parameters such as command type (such as inspection, fault detection, emergency handling, etc.), priority (high, medium, low), execution area range (specific section identification of the corridor), etc.

[0146] The target working mode is the overall operating mode determined according to external instructions. The target working mode may include daily inspection mode, deep troubleshooting mode, energy-saving standby mode, etc. The working status of each module is different in each mode.

[0147] The working status can include parameters such as the start and stop of each module, data collection frequency, data processing depth, information transmission bandwidth, etc. The collection frequency of the environmental perception module can range from several times per second to dozens of times per second in different modes; for example, in the daily inspection mode, the environmental perception module can collect temperature, humidity and gas data based on a sampling frequency of 5Hz, while the virtual reality construction module starts to build a high-resolution three-dimensional scene. The fault identification module can be a low-intensity screening in the daily inspection mode, and a high-intensity in-depth analysis when troubleshooting.

[0148] Exemplarily, after receiving the external command signal, the control module analyzes the command content to determine the target working mode. According to the preset corresponding relationship, the number of sensor activations and sampling frequency of the environment perception module, the scene construction precision and data source emphasis of the virtual reality construction module, the display layout and information refresh frequency of the display module, the algorithm activation level and data processing volume of the fault identification module and other working state parameters are set for each target working mode, so as to coordinate the collaborative work of each module.

[0149] For example, in the operation and maintenance of the pipe corridor, if the remote monitoring center issues an "emergency detection of suspected leakage points in a certain section of the pipe corridor" command signal. The control module determines that the target working mode is the emergency fault detection mode. The environmental perception module starts all gas sensors and collects data at the highest frequency, while the high-resolution camera focuses on the suspected area; the fault identification module enables a high-precision leak detection algorithm to deeply analyze a large amount of real-time data; the virtual reality construction module constructs a virtual scene with high details containing detailed structure and environmental information of the suspected leakage area; the display module highlights the real-time data and fault analysis results of the area, which facilitates the operation and maintenance personnel to quickly judge and take measures, such as arranging robots to go for precise detection or notifying maintenance personnel to prepare for emergency repairs.

[0150] This embodiment can accurately determine the working mode according to external instructions, so that each module can cooperate as needed, quickly and reasonably allocate resources, and help operation and maintenance personnel quickly locate problems and make decisions. Overall, it improves the efficiency of pipeline corridor inspection and fault handling, ensures the safe and stable operation of the pipeline corridor, and reduces potential risks and losses.

[0151] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A pipe gallery inspection robot system based on virtual reality, characterized in that: include: An environment perception module, a virtual reality building module, a display module, a control module and a fault identification module; the control module is connected to the environment perception module, the virtual reality building module, the display module and the fault identification module respectively; The environmental perception module is used to collect environmental information of the pipeline corridor; The fault identification module is used to identify fault information in the pipe gallery; the control module is used to determine target display data based on the environmental information and the fault information; The virtual reality construction module is connected to the environment perception module and the display module respectively; the virtual reality construction module is used to construct a virtual environment based on the environment information and the target display data; the display module is used to display the virtual environment constructed by the virtual reality construction module.

2. The virtual reality-based pipeline corridor inspection robot system according to claim 1, characterized in that: The environmental information includes sensor data and image data; The environment perception module comprises: A sensor unit and a camera unit; The sensor unit and the camera unit are both connected to the control module; The sensor unit is used to collect sensor data of the pipeline gallery, and the camera unit is used to collect image data of the pipeline gallery.

3. The virtual reality-based pipeline corridor inspection robot system according to claim 1, characterized in that: The fault identification module comprises: Environmental fault identification unit and appearance fault detection unit; The environmental fault identification unit and the appearance fault detection unit are both connected to the control module; The environmental fault identification unit is used to detect environmental abnormality information in the pipe gallery, and the appearance fault detection unit is used to detect appearance fault information of equipment and buildings in the pipe gallery.

4. The virtual reality-based pipeline corridor inspection robot system according to claim 1, characterized in that: Also includes: Alarm module; The alarm module is connected to the control module; The alarm module is used for generating an alarm based on the control instruction of the control module.

5. The virtual reality-based pipeline corridor inspection robot system according to claim 1, characterized in that: Also includes: Behavior recognition module; The behavior recognition module is connected to the control module; The behavior recognition module is used to detect the behavior information of people in the tunnel.

6. The virtual reality-based pipeline corridor inspection robot system according to claim 5, characterized in that: The behavior recognition module includes: Face recognition unit and behavior recognition unit; The face recognition unit is connected to the control module and the behavior recognition unit respectively; The behavior recognition unit is connected to the fault recognition module; The face recognition unit is used to recognize the identity information of people in the tunnel, and the behavior recognition unit is used to recognize the behavior information of people in the tunnel.

7. The virtual reality-based pipeline corridor inspection robot system according to claim 1, characterized in that: Also includes: Virtual touch interaction module; The virtual touch interaction module is connected to the control module; The virtual tactile interaction module is used to collect contact information between the inspection robot and the pipe gallery, and transmit the contact information to the tactile interaction device through the control module.

8. The virtual reality-based pipeline corridor inspection robot system according to claim 1, characterized in that: Also includes: Equipment inspection module; The equipment inspection module is connected to the control module; The equipment inspection module is used to detect the working status and health status of the equipment in the pipe gallery.

9. The virtual reality-based pipeline corridor inspection robot system according to claim 1, characterized in that: The determining target display data based on the environmental information and the fault information comprises: determining a first area and a second area based on the environmental information; determining a third area based on the fault information; determining an environmental feature and a first data volume based on the environmental information of the first area; determining first display data based on the environmental feature and the first data amount; determining a second data amount based on the environmental information of the second area; determining second display data based on the environment information of the second area and the second data amount; determining a third data volume based on the environmental information of the third area; determining third display data based on the environment information of the third area and the third data amount; The second data amount is greater than the first data amount, and the second data amount is less than or equal to the third data amount; Target display data is determined based on the first display data, the second display data, and the third display data.

10. A control method for a pipe gallery inspection robot system based on virtual reality, applied to the pipe gallery inspection robot system based on virtual reality as claimed in any one of claims 1 to 9, characterized in that: include: determining a target operating mode based on an external command signal; The working states of the environment perception module, the virtual reality building module, the display module and the fault identification module are determined based on the target working mode.

Images