Converter station valve hall unmanned inspection method, device and equipment and storage medium

Abstract

The invention discloses a converter station valve hall unmanned inspection method, device and equipment and a storage medium, and relates to the technical field of converter station equipment inspection, and the converter station valve hall unmanned inspection method comprises the steps: obtaining to-be-inspected equipment and the priority of the to-be-inspected equipment; establishing a valve hall three-dimensional model containing equipment information by using a laser radar; planning an inspection path according to the equipment, the priority and the model, and determining current inspection equipment and information; acquiring an equipment image through an infrared camera and a high-definition camera, and inputting the equipment image into the fault detection model to obtain a preliminary result; and sending the preliminary result, the equipment information and the temperature, humidity, ultrasonic wave and gas monitoring data to a monitoring system or a cloud, and carrying out deep fault analysis to complete routing inspection. According to the method, the equipment state can be evaluated from multiple dimensions, the equipment fault in the converter station valve hall can be identified timely and accurately, the reliability and safety of the equipment are improved, and the stable operation of the converter station valve hall is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of converter station equipment inspection, and in particular to a converter station valve hall unmanned inspection method, device, equipment and storage medium. Background Art

[0002] The converter station is a core facility connecting the power generation end and the transmission and distribution end. The valve hall of the converter station is the heart of the converter station. The valve towers are numerous, the spatial structure is complex, the electromagnetic environment is complex, and there is electromagnetic shielding from the outside world. It is very important to ensure its safe and stable operation. The equipment in the valve hall mainly includes suspended quadruple valves and corresponding valve cooling systems. Auxiliary equipment includes station power systems, fire protection systems, paging systems, and monitoring systems. With the rapid development of power technology, the demand for inspection of the valve hall of the converter station is increasing to ensure the normal operation of the equipment and timely detection of potential faults.

[0003] At present, the inspection of the valve hall of the converter station mainly relies on manual inspection and real-time monitoring at fixed positions. The manual inspection method has the disadvantages of high labor intensity, low work efficiency, scattered detection quality, and single means. The inspection rate and timeliness cannot be guaranteed. The fixed-position monitoring system can only monitor the local position of some equipment due to objective conditions such as the overall height and stratification of the valve tower. There are large monitoring blind spots, which makes it difficult to truly meet the requirements of full coverage of the inspection range.

[0004] There are many problems with existing manual inspection and fixed monitoring systems, including but not limited to: high labor intensity, low work efficiency, scattered detection quality, single means, large monitoring blind spots, and inability to achieve full coverage. These problems make it difficult to ensure the inspection rate and timeliness, and it is difficult to meet the requirements of large-scale detection. In complex environments such as ultra-strong electromagnetic fields, manual inspections pose safety risks. Therefore, how to timely and accurately identify equipment failures in the converter station valve hall has become an urgent problem to be solved.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The purpose of the present application is to provide an unmanned inspection method, device, equipment and storage medium for a converter station valve hall, aiming to solve the technical problem of how to timely and accurately identify faults of equipment in the converter station valve hall.

[0007] To achieve the above objectives, the present application proposes an unmanned inspection method for a converter station valve hall, the method comprising:

[0008] Get the equipment to be inspected and its priority;

[0009] Establishing a three-dimensional model of the valve hall through the three-dimensional point cloud data of the valve hall of the converter station collected by the laser radar, wherein the three-dimensional model of the valve hall includes valve hall equipment information;

[0010] Obtaining an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected, and the three-dimensional model of the valve hall, and determining the current inspection equipment and current inspection equipment information according to the inspection path and the valve hall equipment information;

[0011] Input the images of the current inspection equipment collected by the infrared camera and the high-definition camera into an equipment fault detection model to obtain a preliminary fault detection result, wherein the equipment fault detection model is trained based on cracked equipment images, damaged equipment images, corroded equipment images, deformed equipment images, and faded equipment images;

[0012] The preliminary fault detection result, the current inspection equipment information and the target data of the current inspection equipment obtained by the temperature sensor, the humidity sensor, the ultrasonic sensor and the gas monitoring sensor are sent to the monitoring system or the cloud, so that the monitoring system or the cloud can analyze the target data in combination with the preliminary fault detection result to identify the deep fault and complete the inspection.

[0013] In one embodiment, the step of establishing a three-dimensional model of the valve hall by using the three-dimensional point cloud data of the valve hall of the converter station collected by the laser radar comprises:

[0014] Scanning various objects and channels in the valve hall of the converter station by the laser radar to obtain three-dimensional point cloud data of the valve hall;

[0015] Registering the three-dimensional point cloud data of the valve hall to obtain a three-dimensional point cloud set of the valve hall;

[0016] Preprocessing the valve hall three-dimensional point cloud set to obtain target point cloud data;

[0017] Constructing an initial three-dimensional model of the converter station valve hall according to the target point cloud data;

[0018] Acquiring image information and depth information of each object in the valve hall of the converter station by means of the high-definition camera;

[0019] Acquire valve hall equipment information, wherein the valve hall equipment information includes equipment images;

[0020] The valve hall three-dimensional model is obtained by identifying the device image and matching it with the image information, and mapping the image information, the depth information and the valve hall device information to the initial three-dimensional model.

[0021] In one embodiment, the step of constructing the initial three-dimensional model of the converter station valve hall according to the target point cloud data comprises:

[0022] Construct an octree according to the target point cloud data;

[0023] By setting a spatial function on each node of the octree, the target point cloud data is converted into a continuous function space;

[0024] Creating a vector field according to the gradient of the target point cloud data by using a cubic spline interpolation strategy;

[0025] Iteratively solving the Poisson equation according to the vector field and the Laplace matrix to obtain an implicit function of the target point cloud data;

[0026] Extracting the three-dimensional surface mesh of the valve hall from the implicit function by using an isosurface extraction strategy;

[0027] The three-dimensional surface mesh of the valve hall is smoothed, denoised and refined to obtain an initial three-dimensional model of the converter station valve hall.

[0028] In one embodiment, the valve hall three-dimensional model includes valve hall channel information, the valve hall equipment information includes equipment location information, and the step of obtaining an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected, and the valve hall three-dimensional model includes:

[0029] Calculate the distance matrix between the devices to be inspected according to the valve hall channel information and the device location information;

[0030] Calculate the path segment weight according to the distance matrix;

[0031] Setting a priority weight according to the priority of the equipment to be inspected;

[0032] The ratio of the priority weight to the path segment weight is used as a weight ratio, wherein the weight ratio corresponds to the path segment between each device;

[0033] Sort the path segments between the devices according to the weight ratios to obtain a path segment sequence;

[0034] Determine the inspection order of each device according to the path segment sequence;

[0035] An inspection path is obtained according to the inspection sequence and the device location information.

[0036] In one embodiment, the step of inputting the image of the current inspection device collected by the infrared camera and the high-definition camera into the device fault detection model to obtain a preliminary fault detection result includes:

[0037] Acquire a high-definition image and an infrared image of the current detection device through the infrared camera and the high-definition camera;

[0038] Performing a weighted average on the brightness values ​​of the high-definition image and the infrared image to obtain a grayscale image;

[0039] Performing weighted averaging on the pixels of the grayscale image according to a preset discrete Gaussian matrix to obtain a target image;

[0040] The target image is input into a device fault detection model to obtain a fault detection result.

[0041] In one embodiment, the device fault detection model includes an input layer, a residual network, a region proposal network, an interest region pooling network, and a fully connected layer, the residual network includes a convolutional layer, an activation layer, and a pooling layer, and the step of inputting the target image into the device fault detection model to obtain a fault detection result includes:

[0042] Normalizing the target image through the input layer to obtain a standard image;

[0043] Performing sliding filtering on the standard image through the convolution layer to obtain a linear feature map;

[0044] Performing a nonlinear transformation on the linear feature map through the activation layer to obtain a nonlinear feature map;

[0045] Performing spatial dimension reduction on the nonlinear feature map through the pooling layer to obtain a pooling feature map;

[0046] Anchoring and pooling are performed through the region proposal network and the region of interest pooling network to obtain a region aggregation feature map;

[0047] Flattening the regional aggregation feature map into a one-dimensional vector through the fully connected layer;

[0048] Classifying and regressing the one-dimensional vector through the fully connected layer to obtain a fault location, a category label, and a confidence level;

[0049] A fault detection result is obtained according to the fault location, the category label and the confidence level.

[0050] In one embodiment, the step of performing anchoring and pooling through the region proposal network and the region of interest pooling network to obtain a region aggregation feature map includes:

[0051] Generate multiple anchor boxes on the pooled feature map through the region proposal network, and calculate the foreground probability of each anchor box;

[0052] Regress each of the anchor boxes through the region proposal network to obtain bounding box regression parameters;

[0053] The position and size of each anchor frame are adjusted according to the bounding box regression parameters by the region proposal network to obtain a target anchor frame;

[0054] By using the region proposal network, a candidate region is obtained by screening from the target anchor frame according to the foreground probability;

[0055] Dividing the candidate region into target regions according to preset sizes by the region of interest pooling network;

[0056] The target region is subjected to maximum pooling through the region of interest pooling network to obtain a regional aggregation feature map.

[0057] In addition, to achieve the above-mentioned purpose, the present application also proposes an unmanned inspection device for a converter station valve hall, the device comprising:

[0058] A data acquisition module is used to obtain the equipment to be inspected and the priority of the equipment to be inspected;

[0059] A valve hall model building module, used to build a valve hall three-dimensional model through the three-dimensional point cloud data of the converter station valve hall collected by the laser radar, wherein the valve hall three-dimensional model includes valve hall equipment information;

[0060] A path planning module, used to obtain an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected and the three-dimensional model of the valve hall, and determine the current inspection equipment and current inspection equipment information according to the inspection path and the valve hall equipment information;

[0061] A preliminary fault detection module, used for inputting the images of the current inspection equipment collected by the infrared camera and the high-definition camera into an equipment fault detection model to obtain a preliminary fault detection result, wherein the equipment fault detection model is trained based on cracked equipment images, damaged equipment images, corroded equipment images, deformed equipment images, and faded equipment images;

[0062] A deep fault detection module is used to send the preliminary fault detection result, the current inspection equipment information and the target data of the current inspection equipment obtained by the temperature sensor, the humidity sensor, the ultrasonic sensor and the gas monitoring sensor to the monitoring system or the cloud, so that the monitoring system or the cloud can analyze the target data in combination with the preliminary fault detection result to identify the deep fault and complete the inspection.

[0063] In addition, to achieve the above-mentioned purpose, the present application also proposes an unmanned inspection device for the valve hall of a converter station, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the unmanned inspection method for the valve hall of a converter station as described above.

[0064] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the unmanned inspection method of the converter station valve hall are implemented as described above.

[0065] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the unmanned inspection method of the converter station valve hall are implemented as described above.

[0066] One or more technical solutions proposed in this application have at least the following technical effects:

[0067] Obtain the equipment to be inspected and its priority; establish a three-dimensional model of the valve hall through the three-dimensional point cloud data of the converter station valve hall collected by the laser radar, wherein the three-dimensional model of the valve hall includes valve hall equipment information; obtain an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected and the three-dimensional model of the valve hall, and determine the current inspection equipment and current inspection equipment information according to the inspection path and the valve hall equipment information; input the image of the current inspection equipment collected by the infrared camera and the high-definition camera into the equipment fault detection model to obtain a preliminary fault detection result, wherein the equipment fault detection model is trained based on cracked equipment images, damaged equipment images, corroded equipment images, deformed equipment images and faded equipment images; send the preliminary fault detection result, the current inspection equipment information and the target data of the current inspection equipment obtained by the temperature sensor, the humidity sensor, the ultrasonic sensor and the gas monitoring sensor to the monitoring system or the cloud, so that the monitoring system or the cloud analyzes the target data in combination with the preliminary fault detection result to identify deep faults and complete the inspection. First, by accurately determining the equipment to be inspected and its priority, it can be ensured that the inspection work can be arranged according to the importance and potential risks of the equipment, so that the most critical equipment can be inspected first, improving the pertinence and efficiency of the inspection. Then, the three-dimensional point cloud data collected by the lidar is used to establish a three-dimensional model of the valve hall, which includes equipment information. This not only makes the inspection path planning more accurate and reduces the movement error of the inspection robot in a complex environment, but also facilitates equipment management and fault simulation in a virtual environment, improving the accuracy and efficiency of the inspection. Then, the inspection path is planned according to the equipment to be inspected, its priority, and the three-dimensional model of the valve hall, and the current inspection equipment and its information are determined. Such intelligent path planning optimizes the inspection process, reduces unnecessary movement and time waste, and ensures that the inspection work is carried out in the established order and plan. In addition, the images collected by the infrared camera and the high-definition camera are input into the equipment fault detection model, which can quickly identify the surface faults of the equipment, such as cracking and damage, to achieve early problem discovery and reduce the potential risk of equipment failure. Finally, the preliminary fault detection results, current inspection equipment information, and target data obtained through various sensors are sent to the monitoring system or cloud. Combined with the preliminary fault detection results, these target data are deeply analyzed to identify the deep faults of the equipment and complete the inspection. This application can evaluate the equipment status from multiple dimensions, timely and accurately identify the faults of the equipment in the converter station valve hall, improve the reliability and safety of the equipment, and ensure the stable operation of the converter station valve hall. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0069] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0070] Figure 1 A flow chart of the first embodiment of the unmanned inspection method for the converter station valve hall of the present application;

[0071] Figure 2 A schematic diagram of the inspection path provided in the first embodiment of the unmanned inspection method for the valve hall of a converter station of the present application;

[0072] Figure 3 A flow chart of the second embodiment of the unmanned inspection method for the converter station valve hall of the present application;

[0073] Figure 4 A schematic diagram of a three-dimensional model of a converter station provided in Embodiment 2 of the unmanned inspection method for the valve hall of a converter station of the present application;

[0074] Figure 5 A schematic diagram of a three-dimensional model of a valve hall provided in Embodiment 2 of the unmanned inspection method for the valve hall of a converter station of the present application;

[0075] Figure 6 This is a schematic diagram of the module structure of the unmanned inspection device for the converter station valve hall according to an embodiment of the present application;

[0076] Figure 7 Schematic diagram of the equipment structure of the hardware operating environment involved in the unmanned inspection method of the converter station valve hall in the embodiment of the present application.

[0077] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0078] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0079] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0080] As the core facility connecting power generation and transmission and distribution, the converter station has a complex internal structure of the valve hall and a special electromagnetic environment, so safe and stable operation is crucial. The main equipment in the valve hall includes a suspended quadruple valve and a cooling system, and auxiliary equipment includes station power, fire protection, paging and monitoring systems. With the development of power technology, the demand for inspections in the valve hall has increased to ensure the normal operation of the equipment and timely detection of faults. At present, inspections mainly rely on manual and fixed monitoring. The former is labor-intensive, inefficient, and has inconsistent detection quality, while the latter has monitoring blind spots and cannot achieve full coverage. These problems lead to inadequate and untimely inspections, making it difficult to meet large-scale detection needs, and manual inspections pose safety risks in complex electromagnetic environments.

[0081] The main solutions of the embodiments of the present application are: first, determine the equipment to be inspected and its priority, ensure that key equipment is inspected first, and improve inspection efficiency. Next, use lidar to establish a three-dimensional model of the valve hall containing equipment information, accurately plan the inspection path, reduce movement errors, and facilitate equipment management and fault simulation. Then, plan the inspection path according to the equipment, priority and three-dimensional model, determine the current inspection equipment and its information, optimize the inspection process, and reduce time and movement waste. Use infrared and high-definition cameras to capture images, input fault detection models, and quickly identify surface faults such as cracking and damage to achieve early detection. Finally, send the preliminary fault detection results, equipment information and various sensor data to the monitoring system or cloud for in-depth analysis, identify deep faults, and complete the inspection.

[0082] It should be noted that the execution subject of the embodiment of the present application can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a drone, an inspection robot, etc. that can realize the above functions. The following takes the inspection robot as an example to illustrate this embodiment and the following embodiments.

[0083] Based on this, the embodiment of the present application provides an unmanned inspection method for the valve hall of a converter station, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the unmanned inspection method for the converter station valve hall of the present application.

[0084] In this embodiment, the method is applied to an inspection robot, which is equipped with a temperature sensor, a humidity sensor, an infrared camera, a high-definition camera, an ultrasonic sensor, a gas monitoring sensor and a laser radar. The unmanned inspection method of the converter station valve hall includes steps S10 to S50:

[0085] Step S10, obtaining the equipment to be inspected and the priority of the equipment to be inspected;

[0086] It should be noted that the inspection robot refers to an automated device used to perform inspection tasks in environments such as the valve hall of a converter station. It is usually equipped with a variety of sensors and cameras, and can automatically navigate and detect equipment status to reduce the need and risk of manual inspections. A temperature sensor refers to a sensor that can sense temperature changes and convert them into usable output signals. It can work based on different principles, such as resistance changes, thermoelectric effects, etc. Common types of temperature sensors include thermistors, thermocouples, and RTDs (resistance temperature detectors). A humidity sensor refers to a sensor that can sense the water vapor content in the atmosphere and convert it into a usable output signal. It can be divided into two categories: resistive and capacitive. Resistive humidity sensors measure humidity changes through a film made of a moisture-sensitive material, while capacitive sensors detect humidity through changes in dielectric constants. An infrared camera refers to a camera that can capture infrared radiation. It is usually used for monitoring and imaging at night or in low light conditions. Infrared cameras can detect the heat energy emitted by objects, allowing them to work in a completely dark environment. HD camera refers to a camera device that can capture high-resolution images. It provides clearer images and videos than standard cameras. HD cameras perform better in detail capture and image quality, and are suitable for occasions that require high-definition monitoring. Ultrasonic sensor refers to a sensor that uses ultrasonic signals for distance measurement or object detection. It determines the distance to the object by emitting ultrasonic waves and receiving their echoes. It is widely used in fields such as automation control, robot navigation, and obstacle detection. Gas monitoring sensor refers to a sensor that can detect specific gases or gas mixtures. It identifies and measures the presence and concentration of gases through chemical reactions or physical changes. It is often used in environmental monitoring, industrial safety, and air quality assessment. LiDAR refers to a remote sensing technology that uses lasers for distance measurement and target detection. It measures the distance and angle to the target object by emitting laser pulses and receiving the reflected light. It is often used in terrain mapping, autonomous vehicles, and air navigation. Equipment to be inspected refers to equipment that needs to be regularly inspected and maintained in the valve hall of the converter station. These equipment are crucial to ensuring the stable operation of the power system and require inspection robots for status monitoring and fault diagnosis. The priority of equipment to be inspected refers to a mechanism for sorting inspection tasks based on factors such as the equipment's operating status, importance, and failure risk. Equipment with higher priority requires more frequent or more urgent inspections to ensure safe and reliable operation of the power system.

[0087] It is understandable that first, the system will establish a comprehensive equipment list based on information such as the type, function, historical fault record, maintenance cycle and safety standards of the equipment. Then, by analyzing the equipment's operating data, historical failure rate, degree of impact on the entire system, and current operating status, the system will assign a priority to each device (the priority of the equipment to be inspected is obtained based on the importance level of the equipment to be inspected and the historical fault equipment information). This priority reflects the urgency and importance of the equipment in the inspection, ensuring that key equipment can be inspected and maintained in a timely manner. For example, for a device that has historically frequently failed and is critical to system operation, its priority will be set higher to ensure that it is inspected first in the inspection plan. Such priority settings help optimize the allocation of inspection resources, improve inspection efficiency, and reduce potential equipment failure risks, thereby ensuring the safe and stable operation of the converter station valve hall.

[0088] Step S20, establishing a three-dimensional model of the valve hall through the three-dimensional point cloud data of the converter station valve hall collected by the laser radar, wherein the three-dimensional model of the valve hall includes valve hall equipment information;

[0089] It should be noted that the 3D point cloud data is a data set of spatial points scanned by a 3D laser radar device. Each point cloud contains 3D coordinates (XYZ) and laser reflection intensity (Intensity). These data points can provide accurate spatial position information about objects or environments, and sometimes also include color information, reflection intensity information, echo frequency information, etc. Point cloud data can be used for spatial measurement, modeling and analysis, and they are the basis for building 3D models. The 3D model of the valve hall is a 3D model constructed by processing the collected 3D point cloud data. This model not only contains the geometric structure of the valve hall, but also includes the precise position and shape information of the internal equipment and structures, which can be used for digital management and visual display. The valve hall equipment information includes detailed data of various equipment in the valve hall, such as equipment model, location, operating status, maintenance records, etc. This information can be integrated into the 3D model of the valve hall to form an equipment data management library to realize the addition, modification and comprehensive management of equipment data. Through the 3D model, the specific data of the equipment can be intuitively displayed and edited, which is convenient for equipment management and maintenance.

[0090] It is understandable that, first of all, a high-precision 3D laser scanner is used to carefully scan the valve hall building and internal equipment and structures of the converter station. This step can collect a large amount of accurate point cloud data, which includes the geometric structure and spatial layout information of the valve hall. Secondly, the collected point cloud coordinate data is imported into the point cloud processing software for preprocessing, including filtering and denoising, data downsampling, feature extraction, etc., to obtain clear and complete overall scan point cloud data. This step is to ensure the data quality of subsequent modeling and improve the accuracy and efficiency of the model. Finally, a 3D model of the valve hall is constructed based on the processed point cloud data, and detailed information of the equipment, such as model, location and function, is integrated to form a valve hall equipment information library. This can intuitively display the internal structure and equipment status of the valve hall, facilitate digital management and intelligent inspection, and improve operation and maintenance efficiency and safety.

[0091] Step S30, obtaining an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected and the three-dimensional model of the valve hall, and determining the current inspection equipment and the current inspection equipment information according to the inspection path and the valve hall equipment information;

[0092] It should be noted that the inspection path is one or more optimal paths calculated by the inspection robot or inspection system based on the location, priority and three-dimensional model of the equipment to be inspected. This path planning takes into account the inspection order of the equipment and the efficiency of the path to ensure that the robot can complete the inspection tasks of all equipment in the least time and distance. The inspection path is determined to optimize the inspection process, reduce unnecessary movement and improve inspection efficiency. The current inspection device refers to the equipment that the inspection robot or system is inspecting or about to inspect based on the current location or progress of the inspection path. This equipment is determined based on the inspection path and the priority of the equipment to be inspected, ensuring that the robot can inspect the equipment in the established order. The current inspection equipment information refers to all data and parameters related to the current inspection equipment, including the model, location, historical maintenance records, operating status, recent inspection results, etc. of the equipment. This information is crucial for the inspection personnel or system because it provides a comprehensive status of the equipment, helps to determine whether the equipment is operating normally and whether maintenance or repair is required. Through this information, the equipment can be effectively managed and maintained.

[0093] It can be understood that, first of all, using the location information and priority of the equipment to be inspected, combined with the spatial layout and equipment distribution in the three-dimensional model of the valve hall, an efficient inspection path is calculated. This path is designed to cover all the equipment to be inspected in a shorter path according to the priority order of the equipment. Secondly, the inspection robot or system determines the equipment to be inspected corresponding to its current position in real time according to this preset inspection path, that is, the current inspection equipment, which is the equipment that the robot will inspect next on the path. Finally, the system will extract detailed information related to the current inspection equipment from the valve hall equipment information library, including the technical parameters of the equipment, historical maintenance records, the results of the last inspection, etc. This information is crucial for performing specific inspection tasks and making maintenance decisions, ensuring the accuracy and effectiveness of the inspection work. Through this process, the inspection work can be carried out in an orderly and efficient manner, while ensuring the timely inspection and maintenance of key equipment.

[0094] As an example, the three-dimensional model of the valve hall includes valve hall channel information, and the valve hall equipment information includes equipment location information. The step of obtaining the inspection path according to the equipment to be inspected, the priority of the equipment to be inspected and the three-dimensional model of the valve hall includes: calculating the distance matrix between the equipment to be inspected according to the valve hall channel information and the equipment location information; calculating the path segment weight according to the distance matrix; setting the priority weight according to the priority of the equipment to be inspected; taking the ratio of the priority weight to the path segment weight as the weight ratio, and the weight ratio corresponds to the path segment between each device; sorting the path segments between each device according to the weight ratio to obtain a path segment sequence; determining the inspection order of each device according to the path segment sequence; and obtaining the inspection path according to the inspection order and the equipment location information.

[0095] Valve hall channel information refers to the channel layout inside the valve hall of the converter station, including the direction, length, width of the channel and the connection relationship between the channels. This information is crucial for planning the inspection path because it determines the possible paths and channels for the inspection robot or system to move inside the valve hall. Equipment location information refers to the specific coordinate position of each device to be inspected in the valve hall of the converter station, including their X, Y, and Z coordinates in three-dimensional space. This information is used to determine the relative position between devices and is the basic data for calculating the inspection path. The distance matrix is ​​a mathematical matrix that contains the pairwise distances between all devices to be inspected. This matrix is ​​obtained by calculating the straight-line distance or actual passable path distance between the coordinate points of each device in the device location information. The path segment weight refers to the weight of each path segment (i.e., the path from one device to another) in the inspection path planning. This weight may be calculated based on factors such as the length, complexity, and passage conditions of the path, and is used to evaluate the importance or priority of the path segment. The priority weight is the weight assigned according to the priority of the device to be inspected. The device priority may be determined based on factors such as the importance of the device, the risk of failure, and the maintenance cycle. Priority weights are used to give certain devices higher priority in path planning. The weight ratio is the ratio of the priority weight to the path segment weight. This ratio is used to comprehensively consider the priority of the device and the weight of the path segment to determine the relative importance of each device in the inspection path. A path segment refers to a continuous path from one device to another in the inspection path. Each path segment has its corresponding weight and priority, which together determine the order of the path segments in the inspection path. The path segment sequence is a sequential list of path segments obtained by sorting according to the weight ratio. This sequence determines the order in which the inspection robot or system accesses the devices to ensure that high-priority devices can be inspected first. The inspection order is the inspection order of each device determined by the path segment sequence. This order combines the priority of the device and the efficiency of the path segment to optimize the entire inspection process. The inspection path is the path that the inspection robot or system moves in the valve hall, which is finally determined based on the inspection order and the device location information. This path is designed to cover all devices to be inspected in the most efficient way, while taking into account the priority of the device and the actual situation of the channel.

[0096] First, by analyzing the channel information and equipment location information in the three-dimensional model of the valve hall, the straight-line distance or actual travel distance between all the devices to be inspected is calculated, and a distance matrix is ​​constructed, which records the distance between each pair of devices in detail. Then, based on this distance matrix, considering the complexity and length of the channel, a path segment weight is assigned to each possible path segment, which reflects the difficulty and length of the path. Next, according to the priority of each device to be inspected, a priority weight is set for each device, which reflects the importance and urgency of the device. After that, the ratio of each device priority weight to its corresponding path segment weight is calculated to obtain the weight ratio, which combines the priority of the device and the difficulty of the path. Next, all path segments are sorted according to the weight ratio to obtain a path segment sequence, which determines the access order between each device during inspection. Subsequently, the inspection order of each device is determined according to the path segment sequence to ensure that high-priority devices can be inspected first. Finally, an optimal inspection route is planned based on the inspection sequence and equipment location information. This route can cover all devices to be inspected in the most efficient way, while taking into account the equipment priority and the actual situation of the channel, thereby achieving efficient and orderly equipment inspection.

[0097] Please refer to Figure 2 , Figure 2 The inspection path schematic diagram provided for the first embodiment of the unmanned inspection method of the converter station valve hall of this application, which depicts in detail the moving route of the inspection robot inside the valve hall. The starting point and end point in the figure indicate the start and end positions of the inspection, and the arrows on the path indicate the direction of the robot. The inspection path is carefully planned to ensure that the robot can cover all key equipment in the valve hall, including smoothing reactors, transformers, converter valves and filters. This path planning not only takes into account the inspection needs of the equipment, but also takes into account the robot's mobile efficiency and safety, avoiding collisions with equipment in the valve hall. By inspecting along this path, the robot can use its onboard sensors and cameras to collect image information and depth information of the equipment, and combine these data with the three-dimensional model of the valve hall to achieve comprehensive monitoring and fault diagnosis of the equipment status. This automated inspection method greatly improves the efficiency and accuracy of the inspection, reduces the risk and cost of manual inspection, and ensures the stable operation of the converter station.

[0098] Step S40, inputting the image of the current inspection device collected by the infrared camera and the high-definition camera into a device fault detection model to obtain a preliminary fault detection result, wherein the device fault detection model is trained based on cracked device images, damaged device images, corroded device images, deformed device images, and faded device images;

[0099] It should be noted that the equipment fault detection model refers to an intelligent model obtained by using deep learning technology and training through a large number of image data of cracked, damaged, corroded, deformed and faded faulty equipment. The model can automatically identify and diagnose the type of equipment fault and improve the efficiency and accuracy of fault detection. The preliminary fault detection result refers to the preliminary judgment result given by the equipment fault detection model after analyzing the input equipment image. These results may include information such as fault type, location, severity, etc., providing a basis for subsequent detailed inspection and maintenance. The cracked equipment image refers to an image of cracks or gaps in the valve hall equipment. These cracks may be the result of damage to the structural integrity of the equipment due to material fatigue, overload or other factors. The damaged equipment image refers to an image showing that the equipment is damaged. The damage may include damage to the function or structure of the equipment caused by impact, wear, aging and other reasons. The corroded equipment image refers to an image reflecting the corrosion of the equipment due to the action of chemical substances or environmental factors. Corrosion will cause the performance of the equipment material to deteriorate and affect the normal use of the equipment. The deformed equipment image refers to an image depicting the shape change of the equipment due to improper force or temperature changes. The deformation may affect the normal operation and performance of the equipment. Faded device images are images where the colors of the display device gradually fade or disappear due to long-term use or exposure to specific environments. The fading may be the result of material aging or chemical changes.

[0100] It can be understood that, first, the current inspection equipment is photographed using an infrared camera and a high-definition camera. The infrared camera can capture the heat distribution of the equipment, while the high-definition camera can capture the clear visual details of the equipment. Then, the collected image data is input into a pre-trained equipment fault detection model. This model is established by analyzing and learning a large number of equipment images in different fault states such as cracking, damage, corrosion, deformation, and fading, and can identify and understand these fault characteristics. The model then analyzes the input image and uses its learned feature recognition capabilities to detect whether the current equipment has similar fault characteristics and gives preliminary fault detection results, which may include the location, type, and possible severity of the fault. The purpose of this process is to quickly identify potential problems with the equipment so that timely measures can be taken to repair or replace it to ensure the stable operation and safety of the equipment.

[0101] As an example, the steps of inputting the image of the current inspection equipment collected by the infrared camera and the high-definition camera into the equipment fault detection model to obtain a preliminary fault detection result include: acquiring a high-definition image and an infrared image of the current inspection equipment through the infrared camera and the high-definition camera; performing weighted averaging on the brightness values ​​of the high-definition image and the infrared image to obtain a grayscale image; performing weighted averaging on the pixels of the grayscale image according to a preset discrete Gaussian matrix to obtain a target image; and inputting the target image into the equipment fault detection model to obtain a fault detection result.

[0102] The current inspection equipment refers to the equipment that the inspection robot or system is inspecting during the inspection process. This is determined based on the inspection path and inspection sequence, and is the specific equipment that currently needs to be fault-checked. HD images refer to images captured by HD cameras with high resolution and clarity. Such images can provide a clear view of the details of the equipment and help identify the physical state and surface features of the equipment. Infrared images refer to images captured by infrared cameras. Such images can show the temperature distribution on the surface of the equipment and help detect thermal anomalies of the equipment, such as overheating or uneven cooling. Weighted average is a mathematical method in which different values ​​are averaged according to certain weights. In image processing, weighted average can be used to combine HD images and infrared images into one image, in which the value of each pixel is the weighted average of the HD pixel value and the infrared pixel value. Grayscale images are single-channel images in which each pixel has only one value representing brightness, ranging from 0 (black) to 255 (white). Grayscale images are represented by converting the RGB values ​​of color images into a single brightness value. The preset discrete Gaussian matrix is ​​a filter commonly used in image processing. It is based on Gaussian distribution and is used to blur images, reduce noise, and enhance certain features of images. Each value in the matrix is ​​a weight, which is used to perform weighted averaging on pixels in grayscale images. Pixels are the basic unit of an image, representing a point in an image, and contain color and brightness information. In a digital image, a collection of pixels constitutes the entire image. The target image is an image obtained after a specific image processing step, which is used for specific analysis or detection tasks. In this step, the target image is a grayscale image obtained by synthesizing a high-definition image and an infrared image and applying Gaussian filtering. It is used to input the device fault detection model for fault analysis.

[0103] First, the current inspection equipment is photographed using an infrared camera and a high-definition camera to obtain high-definition images and infrared images of the equipment. The high-definition image provides detailed information about the equipment, while the infrared image reveals the temperature distribution of the equipment. Then, by calculating the weighted average of the brightness values ​​of the high-definition image and the infrared image, the two images are fused into a grayscale image, which contains both the visible light details of the equipment and the temperature information. Next, a preset discrete Gaussian matrix is ​​used to perform weighted averaging on each pixel of the grayscale image to smooth the image and reduce noise, so that the target image is clearer and the features are more prominent. Finally, this target image is input into the equipment fault detection model, which analyzes the target image based on its training data (including cracking, damage, corrosion, deformation, and fading fault images), identifies whether the equipment has a fault and the type of fault, and obtains preliminary fault detection results. This process combines image processing technology and machine learning to achieve rapid and accurate assessment of equipment status.

[0104] As an example, a device fault detection model includes an input layer, a residual network, a region proposal network, a region of interest pooling network and a fully connected layer, the residual network includes a convolution layer, an activation layer and a pooling layer, and the step of inputting the target image into the device fault detection model to obtain a fault detection result includes: normalizing the target image through the input layer to obtain a standard image; performing sliding filtering on the standard image through the convolution layer to obtain a linear feature map; performing nonlinear transformation on the linear feature map through the activation layer to obtain a nonlinear feature map; performing spatial dimension reduction on the nonlinear feature map through the pooling layer to obtain a pooling feature map; anchoring and pooling through the region proposal network and the region of interest pooling network to obtain a region aggregation feature map; flattening the region aggregation feature map into a one-dimensional vector through the fully connected layer; classifying and performing bounding box regression on the one-dimensional vector through the fully connected layer to obtain a fault location, a category label and a confidence level; and obtaining a fault detection result based on the fault location, the category label and the confidence level.

[0105] The input layer is the first layer of a neural network. It receives raw data input, such as the pixel values ​​of an image, and passes it to the next layer of the network. In the equipment fault detection model, the input layer is responsible for normalizing the target image, that is, converting the image data into a standard format that the model can handle, such as scaling the pixel values ​​to between 0 and 1 for easy processing by subsequent layers. Residual network (ResNet) is a deep learning architecture that solves the gradient vanishing and gradient exploding problems in deep neural network training by introducing the concept of "residual learning". The residual network contains convolutional layers, activation layers, and pooling layers. The convolutional layers are used to extract features, the activation layers (such as ReLU) introduce nonlinearity, and the pooling layers are used to reduce the spatial dimension of the feature map. The Region Proposal Network (RPN) is responsible for quickly generating regions in the image that may contain targets, namely candidate region proposals. The RPN generates object scores and bounding boxes for each window by sliding the window on the feature map and extracting features. These bounding boxes are then used for further target detection. Region of Interest Pooling (RoIPooling) is a technique used to adjust candidate regions of different sizes into fixed-size feature maps for subsequent classification and bounding box regression. This step is critical because it allows the network to uniformly process objects of different sizes. A fully connected layer is a layer in a neural network in which each neuron is connected to all neurons in the previous layer. In the equipment fault detection model, the fully connected layer is responsible for flattening the regional pooled feature map into a one-dimensional vector, and performing classification and bounding box regression on this basis, and finally outputting the fault location, category label, and confidence. A convolutional layer is a layer in a neural network used to extract local features. It processes the input image through a convolution operation (sliding filter) to generate a feature map. Each convolution kernel is responsible for detecting specific features, such as edges or textures. By stacking multiple convolutional layers, the network can learn more complex feature representations. Activation layers contain nonlinear activation functions, such as ReLU (Rectified Linear Unit), which are applied to the output of the convolutional layer to introduce nonlinearity, allowing the neural network to learn and simulate more complex functions and patterns. The pooling layer is used to reduce the spatial dimension of the feature map. It extracts key features by sliding a window on the feature map and performing a maximum or average operation, while reducing the number of parameters and the amount of calculation, and improving the generalization ability of the model. Normalization is the process of adjusting the numerical range of the input data to a specific interval (such as 0 to 1 or -1 to 1). In image processing, normalization can speed up training and improve the convergence and generalization ability of the model by scaling the pixel values ​​to a smaller range.Standard images are normalized images whose pixel values ​​are adjusted to a standard range, such as between 0 and 1, so that the network can process them more effectively. This preprocessing step helps improve the performance and stability of the model. Sliding filtering is a step in the convolution operation, in which the convolution kernel slides on the input image and performs a weighted summation on the covered area to generate a feature map. This process can capture local features of the image, such as edges and textures. Linear feature maps are the results of the convolution layer output. They represent the linear transformation of the input image and have not yet been processed by the nonlinear activation function. These feature maps contain preliminary feature information of the image, but have not yet been enhanced by the nonlinear transformation of the activation layer. Nonlinear transformation is to process linear feature maps through activation functions, introduce nonlinearity, and enhance the expression ability of the network. This transformation enables the network to capture more complex features and patterns, and is a key step in deep learning. Nonlinear feature maps are linear feature maps processed by activation layers. They contain the results of nonlinear transformations, which enable the network to capture more complex features and patterns, and provide richer feature representations for subsequent classification and bounding box regression. Spatial dimensionality reduction is a function of the pooling layer, which reduces the dimensionality of the data by reducing the spatial dimensions (i.e. width and height) of the feature map, while retaining important feature information and improving the generalization ability of the model. The pooled feature map is a feature map processed by the pooling layer. It provides a more compact feature representation for subsequent network layers through spatial dimensionality reduction and feature extraction, while reducing the risk of overfitting. Anchoring refers to the predefined bounding boxes of multiple shapes and scales for each sliding window position, which are used to predict the actual bounding box of the object. These anchor boxes help the network detect objects at different scales and aspect ratios. Pooling is the operation of the pooling layer, including maximum pooling and average pooling, which is used to reduce the spatial dimension of the feature map, extract key features, and improve the invariance of the model. The region pooling feature map is a feature map processed by the RPN network and the RoIPooling network. It converts regions of different sizes into fixed-size feature representations for subsequent detection tasks such as classification and bounding box regression. The one-dimensional vector is the input of the fully connected layer, which flattens the multi-dimensional feature map into a linear sequence for classification and regression analysis. This flattening operation enables the fully connected layer to further process and learn the features. Classification is a task of the fully connected layer. It classifies the input data by learning the relationship between the feature vector and the category label. In the equipment fault detection model, the classification task is to determine whether there is a fault in the image and the type of fault. Bounding box regression is another task of the fully connected layer. By adjusting the predicted bounding box to more accurately locate the target object, it uses the mapping relationship to make the mapped target box of the candidate target box infinitely close to the real target box, thereby improving the accuracy of detection. The fault location is one of the results of bounding box regression, which indicates the exact location of the fault area in the image, providing accurate information for maintenance and diagnosis.The category label is the result of the classification task. It indicates the category or category identifier to which the input data belongs. In the equipment fault detection model, the category label indicates the type of fault, such as cracking, damage, etc. Confidence is one of the results of the classification and bounding box regression tasks. It indicates the degree of certainty of the model in its prediction results. High confidence means that the model is very confident in its predicted fault location and category label.

[0106] First, the target image is normalized through the input layer, that is, the pixel values ​​of the image are standardized to the range of 0 to 1. This can speed up the convergence of the model, improve training efficiency, and reduce the differences between different images. Secondly, the convolution layer performs sliding filtering on the normalized standard image, extracts the local features of the image through the convolution kernel, and generates a linear feature map. This step can capture basic features such as edges and textures in the image. Then, the activation layer performs nonlinear transformation on the linear feature map, usually through activation functions such as ReLU, so that the model can learn and simulate more complex features and patterns to obtain a nonlinear feature map. Next, the pooling layer performs spatial dimensionality reduction on the nonlinear feature map, reduces the size of the feature map through maximum pooling or average pooling, and retains important information to obtain a pooled feature map. This step helps to reduce the amount of calculation and improve the generalization ability of the model. Next, the region proposal network (RPN) and the region of interest pooling network (RoIPooling) perform anchoring and pooling operations to generate a regional aggregation feature map. RPN is responsible for generating candidate regions on the feature map that may contain targets, while RoIPooling adjusts these candidate regions of different sizes to a fixed size for subsequent classification and bounding box regression. Then, the fully connected layer flattens the regional aggregation feature map into a one-dimensional vector. This step compresses the multi-dimensional feature information into one-dimensional data, providing input for classification and bounding box regression. Finally, the fully connected layer classifies and regresses the one-dimensional vector to obtain the fault location, category label, and confidence. The classification task determines the type of fault, the bounding box regression task accurately locates the fault location, and the confidence indicates the degree of confidence of the model in its prediction results. Finally, the fault detection results are obtained based on this information, providing accurate guidance for equipment maintenance and repair. This process combines image preprocessing, feature extraction, region proposal, feature aggregation, and final decision-making to achieve efficient detection of equipment faults.

[0107] As an example, the steps of anchoring and pooling by the region proposal network and the region of interest pooling network to obtain a region aggregation feature map include: generating multiple anchor boxes on the pooling feature map by the region proposal network, and calculating the foreground probability of each anchor box; regressing each anchor box by the region proposal network to obtain a bounding box regression parameter; adjusting the position and size of each anchor box according to the bounding box regression parameter by the region proposal network to obtain a target anchor box; screening candidate areas from the target anchor box according to the foreground probability by the region proposal network; dividing the candidate areas into target areas according to a preset size by the region of interest pooling network; performing maximum pooling on the target area by the region of interest pooling network to obtain a region aggregation feature map.

[0108] Anchor boxes are bounding boxes with different scaling ratios and aspect ratios generated with each pixel as the center. These predefined boxes are used to approximate the true bounding box of the target and help the model to locate and identify. Foreground probability refers to the probability of containing a foreground object calculated by the region proposal network (RPN) for each anchor box. This probability is evaluated based on the degree of match between the anchor box and the true target box and is used to distinguish the foreground from the background. Regression here refers to bounding box regression, which is a process of adjusting the position and size of a predefined anchor box (or reference box) to make it closer to the true target box. Bounding box regression parameters refer to the parameters used to adjust the anchor box to match the true target box during the bounding box regression process, including position offset and size change. These parameters are obtained through regression analysis and are used to optimize the anchor box to better locate the target. The target anchor box refers to the anchor box adjusted by the bounding box regression parameters, and their position and size are optimized to more accurately represent the position of the target object. Candidate regions refer to the target anchor boxes screened out according to the foreground probability. These regions are considered to be areas that may contain target objects and will be further processed for target detection. The preset size refers to the fixed output size to which the candidate regions are resized in the Region of Interest Pooling Network (RoIPooling). For example, a preset size may be 7x7, which means that no matter what the size of the candidate region is in the original image, it will be resized to a fixed size of 7x7 for subsequent processing. The target region refers to the region for further analysis after being divided according to the preset size. These regions are extracted from the candidate regions and have been resized to the preset size for feature extraction and classification. Max pooling is a pooling operation that reduces the spatial dimension of the feature map by taking the maximum value in the local area while retaining the most significant features. This operation helps reduce the amount of computation and improve the robustness of the features.

[0109] First, the region proposal network (RPN) generates multiple anchor boxes on the pooled feature map. These anchor boxes are predefined bounding boxes used to predict the location of the target object. This can reduce the search space and improve detection efficiency. Secondly, the RPN calculates the foreground probability of each anchor box, that is, the probability that these anchor boxes contain the target object. This is achieved through a binary classification problem to distinguish the foreground from the background. Then, the RPN regresses each anchor box to obtain the bounding box regression parameters. These parameters are used to fine-tune the position and size of the anchor box to make it closer to the bounding box of the real target object. This can improve the accuracy of target positioning. Next, the RPN adjusts the position and size of each anchor box according to the bounding box regression parameters to obtain the target anchor box. This step is to further optimize the anchor box so that it can locate the target object more accurately. After that, the RPN selects candidate regions from the target anchor box according to the foreground probability. These regions are considered to be regions that may contain target objects. This step helps to reduce the amount of data for subsequent processing. Next, the region of interest pooling network (RoIPooling) divides the candidate regions into target regions according to preset sizes. This is done so that candidate regions of different sizes can be processed by the network in a unified manner to improve processing speed. For example, the preset size may be 7x7, which means that no matter what the size of the candidate region is in the original image, it will be adjusted to a fixed size of 7x7 for subsequent processing. RoIPooling then performs a maximum pooling operation on the target region, which is achieved by dividing each region into smaller sections and taking the maximum value in each section. This can further reduce the spatial dimension of the feature while retaining key information to obtain a regional convergence feature map. Finally, the regional convergence feature map is used for subsequent classification and bounding box fine-tuning to determine the exact location and category of the target object, which can improve the accuracy and efficiency of target detection.

[0110] Step S50, sending the preliminary fault detection result, the current inspection equipment information and the target data of the current inspection equipment obtained by the temperature sensor, the humidity sensor, the ultrasonic sensor and the gas monitoring sensor to the monitoring system or the cloud, so that the monitoring system or the cloud analyzes the target data in combination with the preliminary fault detection result to identify the deep fault and complete the inspection.

[0111] It should be noted that the target data refers to the data related to the current inspection equipment obtained through devices such as temperature sensors, humidity sensors, ultrasonic sensors, and gas monitoring sensors. These data include information such as the temperature, humidity, ultrasonic reflection signal, and gas composition of the equipment. They are important bases for analyzing the equipment status and identifying faults. Collecting these data can help the monitoring system or the cloud to more accurately analyze the operation status of the equipment and discover potential problems in a timely manner. The monitoring system refers to a system used to monitor and control the operation status of the equipment in real time. It can be a combination of local hardware and software, or a remote monitoring platform connected through the network. The main functions of the monitoring system include data acquisition, status monitoring, abnormal alarm, and remote control. It can ensure the stable operation of the equipment and respond in time when problems occur. The cloud usually refers to an Internet-based computing service, which provides functions such as data storage, processing, and analysis. By sending data to the cloud, remote access to data, big data analysis, and intelligent decision support can be achieved. The advantages of cloud services are their scalability, flexibility, and cost-effectiveness, allowing users to use resources on demand without investing in expensive hardware and maintenance costs. Analyzing and identifying deep faults means using the computing power of a monitoring system or the cloud, combined with preliminary fault detection results and target data, to conduct in-depth analysis to identify deep-seated faults in equipment. This process involves complex data processing, pattern recognition, and machine learning algorithms. The purpose is to mine the characteristics and patterns of faults from large amounts of data, to achieve accurate diagnosis and prediction of faults, and thus improve the reliability and safety of equipment.

[0112] It can be understood that, first, the preliminary fault detection results, current inspection equipment information, and target data obtained through temperature sensors, humidity sensors, ultrasonic sensors, and gas monitoring sensors are integrated to form a comprehensive data set. This step is to ensure that all relevant information is centralized for in-depth analysis. Secondly, the data is preprocessed, including data cleaning, missing value processing, standardization, and normalization to improve the data quality and make it suitable for subsequent analysis. For example, StandardScaler can be used for standardization to convert the data into a distribution with a mean of 0 and a standard deviation of 1, which can speed up the convergence of the model and improve the generalization ability of the model. Then, the preprocessed data is subjected to feature extraction and pattern recognition using a deep learning model. For example, a CNN model can be constructed, which contains convolutional layers, pooling layers, fully connected layers, etc., through which complex feature representations in the data are automatically learned. The CNN part first extracts features from the input image and identifies the mode of fault occurrence, such as cracks, abnormal points, etc. Subsequently, the BiLSTM part can be used to analyze the trend of these features over time in order to classify and locate the fault. Finally, through the output of the deep learning model, combined with the preliminary fault detection results and equipment information, the monitoring system or cloud can identify the deep fault of the equipment, including the fault type, location and severity. Such in-depth analysis can provide more accurate fault diagnosis and predict potential fault risks, thereby improving the reliability and safety of the equipment and ultimately completing a comprehensive inspection of the equipment.

[0113] The present embodiment provides an unmanned inspection method for a converter station valve hall, which obtains equipment to be inspected and the priority of the equipment to be inspected; establishes a three-dimensional model of the valve hall through the three-dimensional point cloud data of the converter station valve hall collected by the laser radar, and the three-dimensional model of the valve hall includes valve hall equipment information; obtains an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected and the three-dimensional model of the valve hall, and determines the current inspection equipment and the current inspection equipment information according to the inspection path and the valve hall equipment information; inputs the image of the current inspection equipment collected by the infrared camera and the high-definition camera into a device fault detection model to obtain a preliminary fault detection result, and the device fault detection model is obtained by training based on cracked equipment images, damaged equipment images, corroded equipment images, deformed equipment images and faded equipment images; sends the preliminary fault detection result, the current inspection equipment information and the target data of the current inspection equipment obtained by the temperature sensor, the humidity sensor, the ultrasonic sensor and the gas monitoring sensor to a monitoring system or a cloud, so that the monitoring system or the cloud analyzes the target data in combination with the preliminary fault detection result to identify deep faults and complete the inspection. First, by accurately determining the equipment to be inspected and its priority, it can be ensured that the inspection work can be arranged according to the importance and potential risks of the equipment, so that the most critical equipment can be inspected first, improving the pertinence and efficiency of the inspection. Then, the three-dimensional point cloud data collected by the lidar is used to establish a three-dimensional model of the valve hall, which includes equipment information. This not only makes the inspection path planning more accurate and reduces the movement error of the inspection robot in a complex environment, but also facilitates equipment management and fault simulation in a virtual environment, improving the accuracy and efficiency of the inspection. Then, the inspection path is planned according to the equipment to be inspected, its priority, and the three-dimensional model of the valve hall, and the current inspection equipment and its information are determined. Such intelligent path planning optimizes the inspection process, reduces unnecessary movement and time waste, and ensures that the inspection work is carried out in the established order and plan. In addition, the images collected by the infrared camera and the high-definition camera are input into the equipment fault detection model, which can quickly identify the surface faults of the equipment, such as cracking and damage, to achieve early problem discovery and reduce the potential risk of equipment failure. Finally, the preliminary fault detection results, the current inspection equipment information, and the target data obtained by various sensors are sent to the monitoring system or the cloud. These target data are deeply analyzed in combination with the preliminary fault detection results to identify the deep faults of the equipment and complete the inspection. This embodiment can evaluate the equipment status from multiple dimensions, timely and accurately identify the faults of the equipment in the converter station valve hall, improve the reliability and safety of the equipment, and ensure the stable operation of the converter station valve hall.

[0114] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 3 , Figure 3 This is a flow chart of the second embodiment of the unmanned inspection method for the converter station valve hall of the present application. Step S20 of the unmanned inspection method for the converter station valve hall includes steps S21 to S27:

[0115] Step S21, scanning various objects and channels in the valve hall of the converter station by the laser radar to obtain three-dimensional point cloud data of the valve hall;

[0116] It should be noted that objects and channels in the valve hall of the converter station refer to various elements that constitute the valve hall structure, including but not limited to the rack structure composed of steel columns, firewalls, steel roof trusses, etc., and channels inside the valve hall. These channels can be walkways between equipment or laying paths for cables, pipes, etc. These objects and channels are the main objects of 3D scanning of the valve hall of the converter station. By scanning them with a high-precision 3D laser scanner, detailed spatial layout and structural information inside the valve hall can be obtained. The 3D point cloud data of the valve hall refers to a set of points in a 3D coordinate system obtained by scanning the valve hall of the converter station through a laser radar. Each point contains 3D coordinates and can carry other information about the attributes of the point, such as color, reflectivity, intensity, etc. Point cloud data is usually obtained by laser scanners, cameras, 3D scanners and other devices, and can be used in applications such as 3D modeling, scene reconstruction, robot navigation, virtual reality and augmented reality. In the application of the valve hall of the converter station, these point cloud data can provide accurate spatial information for establishing a three-dimensional model of the valve hall, thereby realizing functions such as digital management of equipment, optimal resource allocation, safety monitoring and environmental management. Through these data, a three-dimensional visualization platform for the valve hall of the converter station can be built to provide more accurate data support for management and decision-making.

[0117] It is understandable that by using LiDAR to scan the valve hall of the converter station, emitting laser pulses and capturing the reflected light, the distance between the laser and various objects and channels in the valve hall, such as steel columns, firewalls, steel roof trusses, and equipment walkways, is accurately measured, and these distance data are collected. These data points contain the X, Y, and Z coordinates of each scanned point, forming a large amount of three-dimensional spatial data. By summarizing these data points, the three-dimensional point cloud data of the valve hall is obtained. This data not only accurately reflects the spatial structure inside the valve hall, but can also be used for further three-dimensional modeling and analysis, providing accurate three-dimensional spatial information for automated inspection path planning, equipment status monitoring, and maintenance.

[0118] Step S22, registering the valve hall three-dimensional point cloud data to obtain a valve hall three-dimensional point cloud set;

[0119] It should be noted that registration in valve hall 3D point cloud data processing refers to the process of merging multiple point cloud data sets collected from different perspectives or different time points into a unified coordinate system. The purpose of this is to obtain a complete and continuous 3D model, because a single measurement can often only obtain part of the data of an object or scene. The registration process usually involves calculating a rotation matrix and translation vector to adjust the point cloud data in different coordinate systems to a unified coordinate system, thereby achieving data alignment and fusion. The valve hall 3D point cloud set refers to a complete point cloud model formed by combining all the collected valve hall 3D point cloud data sets after registration processing. This model contains the precise spatial information of all objects and channels inside the valve hall, which can be used for subsequent 3D modeling, analysis and visualization. Registration improves the integrity and consistency of the data, so that local data obtained from multiple perspectives can be merged into a global, seamless 3D model, which is crucial for subsequent analysis, simulation and decision support. Through registration, the accuracy and reliability of point cloud data in spatial position can be ensured, providing a solid data foundation for the digital management and intelligent inspection of the valve hall of the converter station.

[0120] It can be understood that, first of all, the process of point cloud registration begins with the collection of valve hall 3D point cloud data, which comes from LiDAR scans at different perspectives or different time points. Since each scan may be located in a different coordinate system, the purpose of registration is to merge these scattered data sets into a unified coordinate system. Doing so can ensure the consistency of all point cloud data in spatial position and provide a complete reference framework for subsequent analysis and modeling. Secondly, the specific operation of registration involves identifying and matching the same feature points in different data sets, and then calculating the rotation matrix and translation vector to align the data sets. This step usually requires the use of algorithms such as the Iterative Closest Point (ICP) algorithm or feature matching-based methods, which can process large amounts of data and find the optimal transformation parameters. Through these algorithms, the accuracy and robustness of the registration can be improved, ensuring that accurate alignment results can be obtained even in the presence of noise and partial occlusion. Finally, after the registration is completed, all aligned point cloud data sets are merged to form a valve hall 3D point cloud set. This integrated data set contains detailed spatial information of all objects and channels inside the valve hall, providing an accurate and continuous reference model for subsequent 3D modeling, analysis and visualization. This improves the integrity and consistency of the data, allowing local data obtained from multiple perspectives to be merged into a global, seamless 3D model, which is critical for subsequent analysis, simulation, and decision support. Registration can ensure the accuracy and reliability of point cloud data in spatial position, providing a solid data foundation for digital management and intelligent inspection of the converter station valve hall.

[0121] Step S23, preprocessing the valve hall three-dimensional point cloud set to obtain target point cloud data;

[0122] It should be noted that preprocessing refers to a series of processing steps for the valve hall 3D point cloud data to improve data quality and prepare for subsequent analysis. The specific steps include: (1) Removing noise, outliers and outliers: In the point cloud data, due to scanning errors or environmental factors, there will be some data points that do not conform to the actual surface of the object. These are called noise or outliers. Outliers are those points that are significantly different from the distribution of most point cloud data. It is necessary to remove these data points because they will affect the accuracy of subsequent analysis. Usually, statistical filtering combined with radius filtering and other methods are used to remove noise points based on their distance from the model body. (2) Estimating the normal of the point cloud: The normal of the point cloud refers to the vector perpendicular to each point on the point cloud surface. Estimating the normal of the point cloud is very important for understanding the geometric structure of the point cloud and performing subsequent 3D modeling, feature extraction and other operations. Normal estimation is usually achieved by fitting a plane or surface in the neighborhood of each point, and then calculating the normal vector of the plane or surface. The target point cloud data refers to the point cloud data obtained after the above preprocessing steps. These data remove noise and outliers, retain the characteristic information of the object surface, and contain the normal information of each point. Such point cloud data is cleaner and more accurate, and is suitable for subsequent 3D modeling, feature extraction and analysis tasks. Preprocessing improves the quality of point cloud data, reduces errors in subsequent processing, improves the accuracy of point cloud matching, and reduces time overhead.

[0123] It can be understood that, first of all, the valve hall 3D point cloud set is preprocessed, starting with the removal of noise and outliers. This step identifies and eliminates those data points that are obviously inconsistent with the surrounding point clouds by applying statistical filtering algorithms, such as using mean filtering or median filtering. This can reduce random errors in the data and outliers in the scanning process, and improve the accuracy of the data. Next, outliers are removed, usually using radius filtering or statistical methods to identify those points that are too far away from the neighboring point clouds and delete them to ensure that the point cloud data fits the actual object surface more closely. Then, the normal of the point cloud is estimated. This step involves fitting a plane or surface in the local neighborhood of each point and calculating the normal vector of the plane or surface. Normal estimation is crucial for understanding the geometric structure of the point cloud and performing subsequent 3D modeling, feature extraction and other operations. Finally, the point cloud data obtained after these preprocessing steps is called the target point cloud data. These data are more accurate and reliable, providing high-quality basic data for subsequent analysis and application, making subsequent processing such as modeling, feature extraction and analysis more accurate and efficient.

[0124] Step S24, constructing an initial three-dimensional model of the converter station valve hall according to the target point cloud data;

[0125] It should be noted that the initial 3D model refers to the preliminary 3D representation of the converter station valve hall constructed based on the target point cloud data. This model is generated directly from the point cloud data and usually contains the geometric shape and spatial position information of all objects and structures inside the valve hall, but the actual image has not yet been mapped.

[0126] It is understandable that the initial 3D model is the basis for subsequent work and can be used for a variety of applications, such as further model refinement, visualization, collision detection, spatial analysis, etc. This model is built to capture the overall structure and detailed features of the valve hall and provide accurate 3D spatial information for automated inspection, maintenance planning and equipment management. The construction of the initial model is an important step in digital and intelligent management, which enables complex on-site environments to be reproduced and analyzed in the digital world.

[0127] As an example, the steps of constructing an initial three-dimensional model of the valve hall of the converter station according to the target point cloud data include: constructing an octree according to the target point cloud data; converting the target point cloud data into a continuous function space by setting a spatial function on each node of the octree; creating a vector field according to the gradient of the target point cloud data by a cubic spline interpolation strategy; iteratively solving the Poisson equation according to the vector field and the Laplace matrix to obtain an implicit function of the target point cloud data; extracting a three-dimensional surface mesh of the valve hall from the implicit function by an isosurface extraction strategy; smoothing, denoising and refining the three-dimensional surface mesh of the valve hall to obtain an initial three-dimensional model of the valve hall of the converter station.

[0128] Octree is a data structure used for scene management in three-dimensional space. It organizes data by recursively dividing the space into eight equal-sized sub-cubes (eight sub-nodes), so that the position of an object in a 3D scene can be quickly determined or collisions with other objects and the visible range can be detected. Spatial function usually refers to a function defined in a certain space, while function space is a space that uses functions as a basis, which can be understood as a space composed of all possible functions. The cubic spline interpolation strategy is a mathematical method for data fitting. It approximates data points by piecewise cubic polynomials. The values ​​of these polynomials are continuous at the nodes and have continuous first-order and second-order derivatives, thereby achieving smooth curve fitting. The gradient is a vector that describes the direction and rate of the fastest growth of a scalar field at a certain point. It points to the direction in which the function grows fastest, and its modulus is the maximum rate of change in that direction. A vector field is a field composed of vectors, each of which is associated with a point in space and represents the vector value at that point, such as speed, force, etc. The Laplace matrix is ​​a concept in graph theory. It is a matrix obtained by mathematically processing the vertices and edges of a graph and is used to describe the structural characteristics of the graph. Poisson's equation is a common partial differential equation. It is widely used in fields such as electrostatics, mechanical engineering, and theoretical physics. It is usually used to describe gravitational fields, electric fields, and magnetic fields. An implicit function is a function implicitly defined by an implicit equation. Instead of explicitly representing one variable by another, it is defined by an equation containing multiple variables. The isosurface extraction strategy is a method of extracting isosurfaces of specific values ​​from a three-dimensional data field. It approximates isosurfaces through techniques such as linear interpolation and approximates them with triangular patches. The three-dimensional surface mesh of the valve hall refers to the three-dimensional geometric representation of the valve hall obtained through the above steps. It consists of a series of connected points, lines, and faces, forming the three-dimensional geometric structure of the valve hall. Smoothing, denoising and refinement are post-processing steps for the three-dimensional surface mesh of the valve hall. Smoothing is used to reduce noise and unevenness in the mesh, denoising is to remove outliers in the data, and refinement is to optimize the structure of the mesh, improve the quality and details of the mesh, and make the final model more accurate and beautiful.

[0129] First, the steps to construct the initial 3D model of the converter station valve hall start with the construction of an octree. An octree is a spatial segmentation data structure that groups point cloud data according to spatial coordinates for easy management and fast retrieval. This can improve the efficiency of data processing, especially when processing large-scale point cloud data. Secondly, by setting a spatial function on each node of the octree, the target point cloud data is converted into a continuous function space. This step involves defining a function that can mathematically describe the point cloud data in the spatial region corresponding to each node of the octree, thereby realizing the conversion from discrete data to continuous representation. Then, a cubic spline interpolation strategy is used to create a vector field based on the gradient of the target point cloud data. The gradient here refers to the rate of change of the point cloud data in space, and the vector field is composed of these gradient vectors, which provides directional information for subsequent surface reconstruction. Next, the Poisson equation is iteratively solved based on the vector field and the Laplace matrix to obtain the implicit function of the target point cloud data. The Poisson equation is a partial differential equation. By solving this equation, the geometric shape of the scene can be restored from the point cloud data. Afterwards, the valve hall 3D surface mesh is extracted from the implicit function through the isosurface extraction strategy. Isosurface extraction is the process of finding the geometric surface of a specific value in the implicit function. This step converts the implicit function into a specific 3D geometric model. Finally, the valve hall 3D surface mesh is smoothed, denoised, and refined to obtain the initial 3D model of the converter station valve hall. Smoothing is used to reduce the roughness of the mesh, denoising eliminates outliers, and refinement improves the resolution and details of the mesh. Finally, an accurate and beautiful initial 3D model is obtained, providing a solid data foundation for subsequent analysis, simulation, and decision-making.

[0130] Step S25, acquiring image information and depth information of each object in the valve hall of the converter station through the high-definition camera;

[0131] It should be noted that image information refers to the visual image data of each object in the valve hall of the converter station captured by a high-definition camera, including visual features such as the shape, color, and texture of the object. This information exists in the form of a pixel matrix, and each pixel contains color information (usually RGB values), which together constitute a two-dimensional image of the object. Depth information refers to the position information of each pixel in three-dimensional space relative to the camera that is matched with the image information, that is, the physical distance of each pixel from the camera. Depth information is usually obtained by a dedicated depth camera or a camera equipped with a depth sensor. It provides the three-dimensional position of the object in space, so that a three-dimensional model can be reconstructed from a plane image. In the application of the valve hall of the converter station, depth information is crucial for tasks such as understanding the spatial layout of objects, performing precise measurements, and navigation.

[0132] It can be understood that, first, the visual image inside the valve hall is captured by the camera, and the details such as the outline, color and texture of the object are recorded to form a high-resolution two-dimensional image. These image information provides the appearance characteristics of the object. Subsequently, the camera's depth sensor measures and records the precise distance of each pixel relative to the camera to generate a depth map. Each pixel value in the depth map corresponds to a distance value, reflecting the relative position of the object in three-dimensional space. Combining image information and depth information, a complete three-dimensional description of the object can be obtained, which is crucial for subsequent tasks such as three-dimensional modeling, fault analysis and equipment monitoring, because they provide precise spatial positioning and detailed visual details.

[0133] Step S26, obtaining valve hall equipment information, wherein the valve hall equipment information includes equipment images;

[0134] It should be noted that valve hall equipment information refers to the relevant data and parameters of the equipment composed of power electronic devices such as converter valves, DC circuit breakers, and DC starting resistors arranged in the valve hall of the converter station. These devices are essential to maintaining the normal operation of the converter station, and their working status and environmental conditions need to be strictly monitored and recorded. Equipment images refer to the visual image data of these valve hall equipment, which can be static photos or dynamic videos, used to record the physical status and appearance characteristics of the equipment.

[0135] It is understandable that the process of obtaining valve hall equipment information involves collecting detailed information on key equipment in the converter station valve hall, including basic information such as equipment specifications, models, locations, maintenance records, and equipment appearance images. Equipment images, i.e. static photos or dynamic videos of equipment, are used to record the actual appearance and status of the equipment. These images are reference images collected and stored in advance.

[0136] Step S27, obtaining a valve hall three-dimensional model by identifying the device image and matching it with the image information, and mapping the image information, the depth information and the valve hall device information to the initial three-dimensional model.

[0137] It should be noted that matching in this step refers to the process of using image recognition technology to compare and analyze the pre-acquired device image with the image information acquired in real time through the camera. The purpose of this process is to confirm whether the on-site device and the pre-recorded device image are the same device, that is, to verify the identity of the device. In the specific operation, the feature points of the device image are extracted and compared with the feature points in the image information captured by the camera to find the matching feature point pairs. This can ensure the accuracy of the identified device information, because only when the feature points are matched successfully can it be confirmed that the device information is the corresponding device. After the match is successful, these image information, depth information and valve hall equipment information are mapped to the initial three-dimensional model together, so as to obtain a richer and more accurate valve hall three-dimensional model. This model not only includes the geometric shape and spatial position of the equipment, but also integrates the detailed information of the equipment, providing strong support for the monitoring, maintenance and fault diagnosis of the equipment. The valve hall three-dimensional model refers to a detailed three-dimensional model containing geometric shape, spatial position and appearance features created by combining image information, depth information and valve hall equipment information. This model not only includes the physical location and appearance details of the equipment, but also reflects the spatial relationship between the equipment and the structural layout of the entire valve hall. The purpose of constructing the valve hall 3D model is to realize the digital management of the converter station valve hall, so that the management personnel can intuitively understand the actual situation of the valve hall and carry out effective monitoring and maintenance. Through this 3D model, the efficiency and accuracy of equipment management can be improved, and it also provides a powerful tool for fault diagnosis and emergency response.

[0138] It can be understood that, first, by identifying the equipment image, the image processing and pattern recognition algorithms are used to detect specific equipment image features from the real-time image information obtained by the high-definition camera. This step involves feature extraction and comparison, the purpose of which is to identify specific equipment in the image captured by the camera. Then, these identified equipment images are matched with the pre-stored equipment images. This process uses the algorithm to determine whether the equipment in the real-time image is consistent with the equipment image in the database, ensuring the accuracy and correspondence of the equipment information. After the match is successful, the image information, depth information and valve hall equipment information collected by the camera are integrated and mapped to the initial three-dimensional model. The image information provides the two-dimensional appearance characteristics of the equipment, the depth information provides the three-dimensional position and size of the equipment, and the valve hall equipment information contains the detailed parameters and properties of the equipment. Finally, this information together constructs a complete three-dimensional model of the valve hall, which accurately shows the geometric structure, spatial layout and detailed properties of the equipment inside the valve hall, providing an intuitive and detailed three-dimensional reference for subsequent analysis, monitoring and maintenance.

[0139] Please refer to Figure 4 , Figure 4A schematic diagram of the three-dimensional model of the converter station provided in Example 2 of the unmanned inspection method of the converter station valve hall of this application shows the three-dimensional models of the key equipment inside the converter station. These models are constructed through high-precision lidar scanning and point cloud data processing technology. The structure of core components such as converter valves, DC circuit breakers and DC starting resistors can be seen in the figure, which are the key to the normal operation of the converter station. The three-dimensional model of each device accurately reflects its layout and spatial relationship in the valve hall. These models are not only used for digital management of equipment, but also provide accurate navigation and fault detection information for automated inspection robots. These models also provide important data support for fault diagnosis and emergency response, ensuring the stable operation of the converter station and the reliability of power supply.

[0140] Please refer to Figure 5 , Figure 5 The schematic diagram of the three-dimensional model of the valve hall provided in the second embodiment of the unmanned inspection method of the valve hall of the converter station of this application shows a comprehensive and detailed digital representation, covering the layout of all key equipment and structures in the valve hall. The precise positions and relationships of equipment such as converter valves, DC circuit breakers, and DC starting resistors can be seen in the model, which are the core of the operation of the converter station. Through laser radar scanning and point cloud data processing technology, the shape and size of each device are accurately captured to form a continuous function space for subsequent analysis and simulation. In addition, the model also includes the passages, supporting structures and other auxiliary facilities of the valve hall, providing a reference for navigation and fault detection for the automated inspection robot. This three-dimensional model is not only used for digital management of equipment, but also provides important data support for fault diagnosis, emergency response and planning, ensuring the stable operation of the converter station and the reliability of power supply. In addition, through this model, operation and maintenance personnel can more intuitively understand the status of the equipment and other information, conduct effective monitoring and maintenance, and at the same time, provide accurate navigation and fault detection information for automated inspection.

[0141] In this embodiment, the laser radar is used to scan various objects and channels in the valve hall of the converter station to obtain the three-dimensional point cloud data of the valve hall; the three-dimensional point cloud data of the valve hall is registered to obtain the three-dimensional point cloud set of the valve hall; the three-dimensional point cloud set of the valve hall is preprocessed to obtain the target point cloud data; the initial three-dimensional model of the valve hall of the converter station is constructed according to the target point cloud data; the image information and depth information of each object in the valve hall of the converter station are obtained by the high-definition camera; the valve hall equipment information is obtained, and the valve hall equipment information includes the equipment image; the three-dimensional model of the valve hall is obtained by identifying the equipment image and matching it with the image information, and mapping the image information, the depth information and the valve hall equipment information to the initial three-dimensional model. First, the laser radar is used to scan various objects and channels in the valve hall of the converter station, accurately capture the spatial coordinates, and generate the three-dimensional point cloud data of the valve hall, which provides basic data for digital management and fault diagnosis. Subsequently, these point cloud data are registered, and the data sets of different perspectives or time points are merged into a unified coordinate system to ensure the consistency and integrity of the data, laying the foundation for building an accurate three-dimensional model. Next, the valve hall 3D point cloud set is preprocessed, including removing noise, outliers and outliers, and estimating the normal of the point cloud, which helps to improve data quality, reduce errors in subsequent processing, and improve the accuracy of point cloud matching. Then, the initial 3D model of the valve hall of the converter station is constructed based on the target point cloud data to provide a framework for detailed modeling and analysis. At the same time, the image information and depth information of each object in the valve hall are obtained through a high-definition camera to provide intuitive data for the identification and positioning of the equipment. In addition, the valve hall equipment information, including equipment images, is collected to provide a reference for the identification of the equipment and the accurate mapping of the model. Finally, by identifying the equipment image and matching it with the image information, the image information, depth information and valve hall equipment information are mapped to the initial 3D model to obtain the valve hall 3D model. This process realizes the precise correspondence between equipment information and 3D spatial coordinates, so that the position and status of each device in the 3D model can be accurately identified and displayed, improving the efficiency and accuracy of equipment management, and providing a powerful tool for fault diagnosis and emergency response.

[0142] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the unmanned inspection method of the converter station valve hall of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0143] This application also provides an unmanned inspection device for the valve hall of a converter station, please refer to Figure 6 The converter station valve hall unmanned inspection device comprises:

[0144] The data acquisition module 10 is used to acquire the equipment to be inspected and the priority of the equipment to be inspected;

[0145] A valve hall model building module 20 is used to build a three-dimensional model of the valve hall through the three-dimensional point cloud data of the converter station valve hall collected by the laser radar, and the three-dimensional model of the valve hall includes valve hall equipment information;

[0146] A path planning module 30, for obtaining an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected and the three-dimensional model of the valve hall, and determining the current inspection device and the current inspection device information according to the inspection path and the valve hall device information;

[0147] A preliminary fault detection module 40 is used to input the image of the current inspection device collected by the infrared camera and the high-definition camera into a device fault detection model to obtain a preliminary fault detection result, wherein the device fault detection model is trained based on cracked device images, damaged device images, corroded device images, deformed device images, and faded device images;

[0148] The deep fault detection module 50 is used to send the preliminary fault detection result, the current inspection equipment information and the target data of the current inspection equipment obtained by the temperature sensor, the humidity sensor, the ultrasonic sensor and the gas monitoring sensor to the monitoring system or the cloud, so that the monitoring system or the cloud can analyze the target data in combination with the preliminary fault detection result to identify the deep fault and complete the inspection.

[0149] In one embodiment, the valve hall model establishment module 20 is also used to scan various objects and channels in the valve hall of the converter station through the laser radar to obtain three-dimensional point cloud data of the valve hall; align the three-dimensional point cloud data of the valve hall to obtain a three-dimensional point cloud set of the valve hall; pre-process the three-dimensional point cloud set of the valve hall to obtain target point cloud data; construct an initial three-dimensional model of the valve hall of the converter station according to the target point cloud data; obtain image information and depth information of each object in the valve hall of the converter station through the high-definition camera; obtain valve hall equipment information, and the valve hall equipment information includes an equipment image; identify the equipment image and match it with the image information, and map the image information, the depth information and the valve hall equipment information to the initial three-dimensional model to obtain a three-dimensional model of the valve hall.

[0150] In one embodiment, the valve hall model establishment module 20 is also used to construct an octree based on the target point cloud data; by setting a spatial function on each node of the octree, the target point cloud data is converted into a continuous function space; a vector field is created according to the gradient of the target point cloud data through a cubic spline interpolation strategy; the Poisson equation is iteratively solved according to the vector field and the Laplace matrix to obtain an implicit function of the target point cloud data; the three-dimensional surface mesh of the valve hall is extracted from the implicit function through an isosurface extraction strategy; the three-dimensional surface mesh of the valve hall is smoothed, denoised and refined to obtain an initial three-dimensional model of the valve hall of the converter station.

[0151] In one embodiment, the path planning module 30 is also used to calculate the distance matrix between the devices to be inspected based on the valve hall channel information and the device location information; calculate the path segment weight based on the distance matrix; set the priority weight based on the priority of the device to be inspected; use the ratio of the priority weight to the path segment weight as the weight ratio, and the weight ratio corresponds to the path segment between each device; sort the path segments between each device according to the weight ratio to obtain a path segment sequence; determine the inspection order of each device based on the path segment sequence; and obtain the inspection path based on the inspection order and the device location information.

[0152] In one embodiment, the preliminary fault detection module 40 is also used to obtain a high-definition image and an infrared image of the current detection device through the infrared camera and the high-definition camera; perform weighted averaging on the brightness values ​​of the high-definition image and the infrared image to obtain a grayscale image; perform weighted averaging on the pixels of the grayscale image according to a preset discrete Gaussian matrix to obtain a target image; and input the target image into a device fault detection model to obtain a fault detection result.

[0153] In one embodiment, the preliminary fault detection module 40 is further used to normalize the target image through the input layer to obtain a standard image; perform sliding filtering on the standard image through the convolution layer to obtain a linear feature map; perform nonlinear transformation on the linear feature map through the activation layer to obtain a nonlinear feature map; perform spatial dimension reduction on the nonlinear feature map through the pooling layer to obtain a pooled feature map; perform anchoring and pooling through the region proposal network and the region of interest pooling network to obtain a region aggregation feature map; flatten the region aggregation feature map into a one-dimensional vector through the fully connected layer; perform classification and bounding box regression on the one-dimensional vector through the fully connected layer to obtain a fault location, a category label and a confidence level; and obtain a fault detection result based on the fault location, the category label and the confidence level.

[0154] In one embodiment, the preliminary fault detection module 40 is further used to generate multiple anchor boxes on the pooled feature map through the region proposal network, and calculate the foreground probability of each anchor box; regress each anchor box through the region proposal network to obtain bounding box regression parameters; adjust the position and size of each anchor box according to the bounding box regression parameters through the region proposal network to obtain a target anchor box; filter candidate areas from the target anchor frame according to the foreground probability through the region proposal network; divide the candidate areas into target areas according to a preset size through the region of interest pooling network; and perform maximum pooling on the target area through the region of interest pooling network to obtain a regional aggregation feature map.

[0155] The unmanned inspection device for the valve hall of a converter station provided by the present application adopts the unmanned inspection method for the valve hall of a converter station in the above-mentioned embodiment, and can solve the technical problem of how to timely and accurately identify the faults of the equipment in the valve hall of a converter station. Compared with the prior art, the beneficial effects of the unmanned inspection device for the valve hall of a converter station provided by the present application are the same as the beneficial effects of the unmanned inspection method for the valve hall of a converter station provided by the above-mentioned embodiment, and the other technical features of the unmanned inspection device for the valve hall of a converter station are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0156] The present application provides an unmanned inspection device for a converter station valve hall, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the unmanned inspection method for the converter station valve hall in the above-mentioned embodiment one.

[0157] Reference below Figure 7 , which shows a schematic diagram of the structure of the converter station valve hall unmanned inspection device suitable for implementing the embodiment of the present application. The converter station valve hall unmanned inspection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The unmanned inspection equipment for the converter station valve hall shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0158] like Figure 7As shown, the converter station valve hall unmanned inspection equipment may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the converter station valve hall unmanned inspection equipment are also stored in the RAM 1004. The processing device 1001, ROM 1002 and RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the converter station valve hall unmanned inspection device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows the converter station valve hall unmanned inspection device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0159] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0160] The converter station valve hall unmanned inspection device provided by the present application adopts the converter station valve hall unmanned inspection method in the above embodiment, which can solve the technical problem of how to timely and accurately identify the faults of the equipment in the converter station valve hall. Compared with the prior art, the beneficial effects of the converter station valve hall unmanned inspection device provided by the present application are the same as the beneficial effects of the converter station valve hall unmanned inspection method provided by the above embodiment, and the other technical features of the converter station valve hall unmanned inspection device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0161] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0162] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0163] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the unmanned inspection method of the converter station valve hall in the above-mentioned embodiment.

[0164] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0165] The above-mentioned computer-readable storage medium may be included in the unmanned inspection device of the converter station valve hall; or it may exist independently without being assembled into the unmanned inspection device of the converter station valve hall.

[0166] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the unmanned inspection device of the converter station valve hall, the unmanned inspection device of the converter station valve hall: obtains the equipment to be inspected and the priority of the equipment to be inspected; establishes a three-dimensional model of the valve hall through the three-dimensional point cloud data of the converter station valve hall collected by the laser radar, and the three-dimensional model of the valve hall includes valve hall equipment information; obtains an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected and the three-dimensional model of the valve hall, and determines the current inspection equipment and the current inspection equipment information according to the inspection path and the valve hall equipment information; and The image of the current inspection equipment in the set is input into a device fault detection model to obtain a preliminary fault detection result, wherein the device fault detection model is trained based on cracked equipment images, damaged equipment images, corroded equipment images, deformed equipment images, and faded equipment images; the preliminary fault detection result, the current inspection equipment information, and the target data of the current inspection equipment obtained by the temperature sensor, the humidity sensor, the ultrasonic sensor, and the gas monitoring sensor are sent to a monitoring system or a cloud, so that the monitoring system or the cloud analyzes the target data in combination with the preliminary fault detection result to identify deep faults and complete the inspection.

[0167] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0168] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0169] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0170] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned converter station valve hall unmanned inspection method, and can solve the technical problem of how to timely and accurately identify the faults of the equipment in the converter station valve hall. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the converter station valve hall unmanned inspection method provided by the above-mentioned embodiment, and will not be repeated here.

[0171] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned unmanned inspection method for the converter station valve hall.

[0172] The computer program product provided by the present application can solve the technical problem of how to timely and accurately identify the faults of the equipment in the converter station valve hall. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the unmanned inspection method of the converter station valve hall provided by the above embodiment, which will not be repeated here.

[0173] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An unmanned inspection method for a converter station valve hall, characterized in that: The method is applied to an inspection robot, which is equipped with a temperature sensor, a humidity sensor, an infrared camera, a high-definition camera, an ultrasonic sensor, a gas monitoring sensor, and a laser radar. The method includes: Get the equipment to be inspected and its priority; Establishing a three-dimensional model of the valve hall through the three-dimensional point cloud data of the valve hall of the converter station collected by the laser radar, wherein the three-dimensional model of the valve hall includes valve hall equipment information; Obtaining an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected, and the three-dimensional model of the valve hall, and determining the current inspection equipment and current inspection equipment information according to the inspection path and the valve hall equipment information; Input the images of the current inspection equipment collected by the infrared camera and the high-definition camera into an equipment fault detection model to obtain a preliminary fault detection result, wherein the equipment fault detection model is trained based on cracked equipment images, damaged equipment images, corroded equipment images, deformed equipment images, and faded equipment images; The preliminary fault detection result, the current inspection equipment information and the target data of the current inspection equipment obtained by the temperature sensor, the humidity sensor, the ultrasonic sensor and the gas monitoring sensor are sent to the monitoring system or the cloud, so that the monitoring system or the cloud can analyze the target data in combination with the preliminary fault detection result to identify the deep fault and complete the inspection.

2. The method according to claim 1, characterized in that The step of establishing a three-dimensional model of the valve hall by using the three-dimensional point cloud data of the valve hall of the converter station collected by the laser radar comprises: Scanning various objects and channels in the valve hall of the converter station by the laser radar to obtain three-dimensional point cloud data of the valve hall; Registering the three-dimensional point cloud data of the valve hall to obtain a three-dimensional point cloud set of the valve hall; Preprocessing the valve hall three-dimensional point cloud set to obtain target point cloud data; Constructing an initial three-dimensional model of the converter station valve hall according to the target point cloud data; Acquiring image information and depth information of each object in the valve hall of the converter station by means of the high-definition camera; Acquire valve hall equipment information, wherein the valve hall equipment information includes equipment images; The valve hall three-dimensional model is obtained by identifying the device image and matching it with the image information, and mapping the image information, the depth information and the valve hall device information to the initial three-dimensional model.

3. The method according to claim 2, characterized in that The step of constructing the initial three-dimensional model of the converter station valve hall according to the target point cloud data comprises: Construct an octree according to the target point cloud data; By setting a spatial function on each node of the octree, the target point cloud data is converted into a continuous function space; Creating a vector field according to the gradient of the target point cloud data by using a cubic spline interpolation strategy; Iteratively solving the Poisson equation according to the vector field and the Laplace matrix to obtain an implicit function of the target point cloud data; Extracting the three-dimensional surface mesh of the valve hall from the implicit function by using an isosurface extraction strategy; The three-dimensional surface mesh of the valve hall is smoothed, denoised and refined to obtain an initial three-dimensional model of the converter station valve hall.

4. The method according to claim 1, characterized in that The valve hall three-dimensional model includes valve hall channel information, the valve hall equipment information includes equipment location information, and the step of obtaining an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected and the valve hall three-dimensional model includes: Calculate the distance matrix between the devices to be inspected according to the valve hall channel information and the device location information; Calculate the path segment weight according to the distance matrix; Setting a priority weight according to the priority of the equipment to be inspected; The ratio of the priority weight to the path segment weight is used as a weight ratio, wherein the weight ratio corresponds to the path segment between each device; Sort the path segments between the devices according to the weight ratios to obtain a path segment sequence; Determine the inspection order of each device according to the path segment sequence; An inspection path is obtained according to the inspection sequence and the device location information.

5. The method according to claim 1, characterized in that The step of inputting the image of the current inspection device collected by the infrared camera and the high-definition camera into the device fault detection model to obtain a preliminary fault detection result comprises: Acquire a high-definition image and an infrared image of the current detection device through the infrared camera and the high-definition camera; Performing a weighted average on the brightness values ​​of the high-definition image and the infrared image to obtain a grayscale image; Performing weighted averaging on the pixels of the grayscale image according to a preset discrete Gaussian matrix to obtain a target image; The target image is input into a device fault detection model to obtain a fault detection result.

6. The method according to claim 5, characterized in that The device fault detection model includes an input layer, a residual network, a region proposal network, an interest region pooling network and a fully connected layer, the residual network includes a convolutional layer, an activation layer and a pooling layer, and the step of inputting the target image into the device fault detection model to obtain a fault detection result includes: Normalizing the target image through the input layer to obtain a standard image; Performing sliding filtering on the standard image through the convolution layer to obtain a linear feature map; Performing a nonlinear transformation on the linear feature map through the activation layer to obtain a nonlinear feature map; Performing spatial dimension reduction on the nonlinear feature map through the pooling layer to obtain a pooling feature map; Anchoring and pooling are performed through the region proposal network and the region of interest pooling network to obtain a region aggregation feature map; Flattening the regional aggregation feature map into a one-dimensional vector through the fully connected layer; Classifying and regressing the one-dimensional vector through the fully connected layer to obtain a fault location, a category label, and a confidence level; A fault detection result is obtained according to the fault location, the category label and the confidence level.

7. The method according to claim 6, characterized in that The step of anchoring and pooling by the region proposal network and the region of interest pooling network to obtain a region aggregation feature map comprises: Generate multiple anchor boxes on the pooled feature map through the region proposal network, and calculate the foreground probability of each anchor box; Regressing each of the anchor boxes through the region proposal network to obtain bounding box regression parameters; The position and size of each anchor frame are adjusted according to the bounding box regression parameters by the region proposal network to obtain a target anchor frame; By using the region proposal network, a candidate region is obtained by screening from the target anchor frame according to the foreground probability; Dividing the candidate region into target regions according to preset sizes by the region of interest pooling network; The target region is subjected to maximum pooling through the region of interest pooling network to obtain a regional aggregation feature map.

8. An unmanned inspection device for a converter station valve hall, characterized in that: The device comprises: A data acquisition module is used to obtain the equipment to be inspected and the priority of the equipment to be inspected; A valve hall model building module is used to build a three-dimensional model of the valve hall through the three-dimensional point cloud data of the converter station valve hall collected by the laser radar, and the three-dimensional model of the valve hall includes valve hall equipment information; A path planning module, used to obtain an inspection path according to the equipment to be inspected, the priority of the equipment to be inspected and the three-dimensional model of the valve hall, and determine the current inspection equipment and current inspection equipment information according to the inspection path and the valve hall equipment information; A preliminary fault detection module, used for inputting the image of the current inspection equipment collected by the infrared camera and the high-definition camera into the equipment fault detection model to obtain a preliminary fault detection result, wherein the equipment fault detection model is trained based on cracked equipment images, damaged equipment images, corroded equipment images, deformed equipment images, and faded equipment images; The deep fault detection module is used to send the preliminary fault detection results, the current inspection equipment information and the target data of the current inspection equipment obtained by the temperature sensor, humidity sensor, ultrasonic sensor and gas monitoring sensor to the monitoring system or the cloud, so that the monitoring system or the cloud can analyze the target data in combination with the preliminary fault detection results to identify the deep fault and complete the inspection.

9. An unmanned inspection device for the valve hall of a converter station, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the converter station valve hall unmanned inspection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the unmanned inspection method of the converter station valve hall according to any one of claims 1 to 7 are implemented.

Images