A method for operating a substation intelligent robot

Abstract

The present invention provides a method for operating an intelligent robot in a substation, which collects the operating status of each operating equipment in the substation and the corresponding equipment information; uploads the collected operating data of each equipment to a control center for analysis to determine equipment operating failures; and implements the robot's next mobile control strategy by establishing a semantic segmentation network, thereby realizing obstacle detection and obstacle avoidance in long and short distance areas.

Description

Technical Field

[0001] The present disclosure relates to the technical field of substation robot inspection, and in particular to a method for operating a substation intelligent robot. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] In existing open substations, substation inspection robots are generally used to conduct all-round and all-weather autonomous inspections of substation equipment, which can promptly discover equipment operating defects and ensure the safety of power grid operation. At present, in the image acquisition and analysis method of substation inspection robots, a fixed-point docking and then calling preset positions are adopted. However, the configuration of inspection points and preset positions is labor-intensive, and the personnel and time costs are high, which greatly increases the difficulty of on-site implementation of inspection robots. In relatively large substations, the navigation path between equipment and robots is required to reach the inspection point or maintenance point accurately and quickly, which requires extremely high requirements for path navigation and obstacle avoidance. However, when avoiding obstacles, the judgment accuracy of turning or stopping is not high, and it is difficult to achieve high efficiency of robot operation. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a substation intelligent robot operation method and system, which realizes the next movement control strategy of the robot by establishing a semantic segmentation network, and realizes obstacle detection and obstacle avoidance in long and short distance areas.

[0005] According to some embodiments, the present disclosure adopts the following technical solutions:

[0006] A method for operating a substation intelligent robot, comprising:

[0007] Collect the operating status of each operating equipment in the substation and the corresponding equipment information;

[0008] Upload the collected operation data of each device to the control center for analysis to determine the equipment operation failure;

[0009] If a fault occurs, the device number information and the location of the corresponding device are retrieved and sent to the pan-tilt platform. The pan-tilt platform plans a route based on the location of the device and the pre-uploaded station map, and moves along the path to perform inspection and maintenance.

[0010] Specifically, the inspection camera is used to collect real-time image information of the robot's surrounding environment and upload it to the gimbal; if there are obstacles in both far and near areas in the collection and analysis of the surrounding environment image, the gimbal analyzes the targets in the far-distance area and uses the targets in the far-distance area as the basis for the robot's next path planning, and determines whether the robot is currently feasible in the near-distance area.

[0011] If an insurmountable obstacle is detected in the near distance ahead, the robot's obstacle avoidance route is planned by comparing the road areas on the left and right sides of the obstacle, and a semantic segmentation network is used to output a mobile control strategy.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] The present invention adopts segmentation post-processing to ensure obstacle avoidance effect, improves the robot's obstacle avoidance performance, and can achieve accurate image acquisition. By using the cooperation of binocular cameras, inspection cameras and gimbals, accurate image and various information acquisition can be achieved, and standardized and accurate inspection image acquisition can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0015] Figure 1 It is a schematic diagram of the operating method of the present disclosure;

[0016] Figure 2 This is a schematic diagram of the network structure of the present disclosure;

[0017] Figure 3 This is a schematic diagram of the skeleton network module disclosed in the present invention;

[0018] Figure 4 This is a schematic diagram of the segmentation module disclosed in the present invention; DETAILED DESCRIPTION

[0019] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0021] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0022] Example 1

[0023] The present disclosure provides a substation intelligent robot operation method, which specifically includes:

[0024] Step 1: Collect the operating status of each operating equipment in the substation and the corresponding equipment information;

[0025] Step 2: Upload the collected operation data of each device to the control center for analysis to determine whether the device operation fails;

[0026] Step 3: If a fault is determined, the device number information and the location of the corresponding device are retrieved and sent to the gimbal. The gimbal plans a route based on the location of the device and the pre-uploaded station map, and moves along the route to perform inspection and maintenance.

[0027] In large substations, substation robots are wheel-driven and used in open substations. The substation robot components are visible light and infrared detection equipment, which can realize all-round and all-weather autonomous inspection of substation equipment. They can promptly detect defects in equipment operation and ensure the safety of power grid operation.

[0028] First, the operating status and equipment information of the equipment in the substation should be collected. The specific operating status of the equipment includes the equipment appearance, equipment temperature, display data of the instrument and the opening and closing status of the switch; the equipment information is the number information of each device. The equipment number is obtained and stored in the database to facilitate the rapid retrieval of equipment information later.

[0029] A remote control center is set up, and the control center is wirelessly connected to the pan-tilt head. The pan-tilt head is set on the intelligent robot, and the location distribution information of the equipment in the substation is collected by the laser sensor, uploaded to the pan-tilt head, and a three-dimensional map of the equipment distribution in the substation is created. The laser sensor is a three-dimensional space laser sensor, and the laser sensor can also feedback the robot's location information obtained in real time.

[0030] After the control center obtains the status information of the equipment operation, it uses the control center to determine whether there is any equipment failure. If fault information is detected, the equipment number information of the faulty equipment and the corresponding location information will be retrieved and sent to the gimbal. The gimbal plans the path according to the equipment location information and the created map, and uploads its own location to the control center in real time.

[0031] Send instructions to control the robot to go to the fault location. Specifically, during the movement, use the binocular camera to collect wide-angle equipment information in the substation in real time; use the inspection camera to collect real-time image information of the robot's surrounding environment and upload it to the pan-tilt. The binocular camera is fixedly installed in front of the robot body and uses a short-focus dual camera. The inspection camera is installed on the robot pan-tilt and uses a variable focal length camera. Through the movement of the pan-tilt and the change of focal length, accurate collection of surrounding environment images is achieved.

[0032] The substation robot autonomously collects data information of the moving process. Through the binocular camera fixedly installed in front of the robot body, it obtains various wide-angle equipment information in the substation in real time. The existing deep learning algorithm is used to realize real-time detection of target equipment in the left and right fields of view of the camera. According to the binocular stereo vision algorithm, the spatial position information of the target equipment camera coordinate system is obtained. The robot gimbal integrates the spatial information of the target equipment obtained by the laser sensor and the binocular camera, and then constructs a complete three-dimensional semantic map of the equipment location in the substation.

[0033] When the robot receives the fault confirmation command, it receives the target device information, then retrieves the location information of the target device, and then marks it in the pre-built map. After creating the forward path, it sends an execution command to the robot. The robot receives the command and executes it, and starts to move forward along the path.

[0034] While moving forward, the inspection camera installed on the robot collects information about the surrounding environment in real time. If there are obstacles in both far and near areas in the collected analysis of the surrounding environment image, the gimbal analyzes the targets in the far and near distances, and uses the targets in the far-distance area as the basis for the robot's next path planning, and determines whether the robot is currently feasible in the near-distance area.

[0035] If an insurmountable obstacle is detected in the near distance ahead, the robot's obstacle avoidance route is planned by comparing the road areas on the left and right sides of the obstacle, and a semantic segmentation network is used to output a mobile control strategy.

[0036] The specific control strategy is: when the outputs of the two branches in the semantic segmentation network are the same, the output direction is executed; when they are opposite, the execution direction is determined by the segmentation branch.

[0037] Specifically, when turning and going straight occur, if there is no obstacle, the classification output is executed, otherwise the split branch output is executed; when turning and stopping occur, turning is executed; when going straight and stopping occur, stopping is executed.

[0038] The above-mentioned semantic segmentation network is composed of a feature extraction structure, a classification structure, and a semantic segmentation structure, and is combined with post-processing to achieve robot obstacle avoidance. The feature extraction structure is mainly composed of a stack of basic network layers, which gradually extracts image feature information in depth. The classification structure further abstracts the extracted features into the robot's driving direction information through a fully connected operation. The semantic segmentation structure uses an upsampling operation to segment the area of ​​each target in the image from the feature information. Post-processing converts the semantic segmentation results into a driving direction signal understood by the robot. Finally, the post-processing and classification results are combined to jointly guide the robot's next action and achieve autonomous obstacle avoidance.

[0039] At present, in the deep learning algorithms used to solve computer vision tasks, the network construction basically adopts a mode that combines a backbone network with a specific task network. Image classification is relatively simple in computer vision tasks. It mainly consists of the backbone network extracting images effectively and then combining full connections to abstract the extracted information into categories to form a classification network. With the rapid development of deep learning, researchers have proposed many classic backbone networks, such as VGG1, Resnet2, Mo-eNet, Senet, etc. Since the method in this paper is mainly used in substation inspection robots, in order to ensure network efficiency, the proposed classification network mainly draws on lightweight networks and uses deep separable convolutions as basic convolution units to build the classification network. The detailed structure of the network is shown in Table 1. The network mainly extracts image features from shallow to deep by connecting multiple convolution block structures in series, and then classifies the objects through operations such as pooling and fully connected (FC). Each convolution block is stacked with multiple lightweight inverse residual modules and attention modules. Its basic structure is as follows Figure 3As shown. Among them, the lightweight inverse residual module is mainly improved on the basis of the traditional residual module. By first increasing the dimension of the feature map, and then using the deep separable convolution (Dwcony) to extract features and then reduce the dimension, it reduces the loss of feature diversity information caused by the traditional residual first reducing the dimension and then increasing the dimension; at the same time, this module uses deep separable convolution to replace the traditional convolution to extract features, which also effectively improves the network efficiency; finally, the residual connection of the traditional residual module is used to alleviate the problem of gradient disappearance during network training. The attention module is mainly composed of global pooling, full connection, and Sigmoid activation function. This module learns a weight for each feature channel through training to measure the importance of each feature channel to the current task, which can increase the weight of effective features and reduce the impact of invalid features on network accuracy. Compared with the current mainstream classification networks (Inception, Resnet, etc.), the entire classification structure is more streamlined and efficient, and has lower requirements on platform performance. Although the network loses some feature information at the expense of some feature information, for the classification of substation road scenes, its classification categories are relatively small and the target features are relatively simple. The proposed network is sufficient to meet the needs of inspection robots for road scene classification.

[0040] Table 1 Classification network structure

[0041]

[0042] The classification network mainly ensures the correct driving direction of the inspection robot from the perspective of global image information, but for robot obstacle avoidance, the classification network alone is difficult to meet its local dynamic obstacle avoidance requirements. Therefore, this paper adds a semantic segmentation branch on the basis of the classification structure to classify the image at the pixel level, and further improves the robot's obstacle avoidance effect by real-time recognition and segmentation of obstacles. Compared with classification, the semantic segmentation task is relatively more difficult and the network structure is more complex. Therefore, in order to ensure the segmentation accuracy and efficiency of the network in the actual substation scene, a variety of strategies are adopted to build the segmentation structure, such as Figure 4 To effectively obtain the spatial detail information of each target in the image, the network is upsampled layer by layer and fused with the feature map of the corresponding dimension in the backbone network, as shown in Figure 4 (a) is shown. The structure first performs the maximum pooling on the feature map after the last 5x5 convolution in the classification network and then upsamples it to the convolution scale for fusion (eltwise). Then the fused feature map is upsampled and fused again, and so on. The upsampling process uses the bilinear interpolation method to avoid introducing too many parameters and reducing the network efficiency. At the same time, since the shallow feature map contains more noise, the network only fuses the feature map with a scale less than 1 / 8 of the original image. That is, the convolution block n in the figure is the last layer in the convolution block 4, convolution block 6, and convolution block 7, respectively, and the upsampling i is the upsampling layer of the corresponding size of the convolution block n. Figure 4(b) The structure mainly further mines the target details and global information, adopts a feature extraction method similar to the image pyramid structure, and extracts information from the last layer of convolution block 4 in the backbone network through dilated convolution with different dilation rates and mean pooling operations. After dilated convolution, it is upsampled to convolution block 4 for concatenation. Figure 4 (a) and Figure 4 (b) The feature maps output by the two structures are balanced by batch normalization and then fused by point convolution. Finally, the fused feature maps are upsampled to the original image size to achieve semantic segmentation. The semantic segmentation branch obtains the full and edge information of the target from multiple angles to ensure that the substation inspection robot can accurately obtain the obstacle category and relative position during the real-time obstacle avoidance process, thereby guiding the robot to achieve efficient obstacle avoidance.

[0043] Post-processing is as follows: for the semantic segmentation results, the robot can only determine the category of each pixel in the image, and the obstacle avoidance information needs to be further converted and extracted by the segmented image. For the obstacle avoidance problem of the substation inspection robot, the semantic segmentation results are converted into information such as obstacle categories and feasible road areas. The obstacle category information mainly extracts the regional targets that affect the normal driving of the robot, that is, the target information directly in front of the robot. In order to effectively assist the robot in obstacle avoidance, two far and near areas are selected in front of the robot to identify the target. The size of the two areas is a mapping of the area occupied by the robot. Among them, the far-distance regional target can guide the robot to plan the path in advance; the close-distance regional target determines whether the robot is feasible. When an insurmountable obstacle appears in front of the robot, a relatively simple and intuitive method is used to select the obstacle avoidance area for the robot, that is, by comparing the road area on the left and right sides of the obstacle (the area can be represented by the number of road pixels) to plan the robot's obstacle avoidance route. Among them, the comparison area is similar to the way of judging obstacles, and two groups of far and near areas are also selected. When there are no obstacles or obstacles appear in the far target area, the far area is used to judge the result to guide the robot to drive; when obstacles appear in the near target area from far to near, the near area is selected to ensure that the robot accurately avoids obstacles. Finally, the obstacle avoidance information of the segmentation result and the classification result are combined to jointly control the robot's obstacle avoidance. The control strategy is as follows:

[0044] When the outputs of two branches in the semantic segmentation network are the same, the output direction is executed. When they are opposite, the execution direction is determined by the segmentation branch.

[0045] Specifically, when turning and going straight occur, if there is no obstacle, the classification output is executed, otherwise the split branch output is executed; when turning and stopping occur, turning is executed; when going straight and stopping occur, stopping is executed.

[0046] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0048] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0049] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0050] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

[0051] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A method for operating a substation intelligent robot, characterized in that: include: Collect the operating status of each operating equipment in the substation and the corresponding equipment information; Upload the collected operation data of each device to the control center for analysis to determine the equipment operation failure; If a fault occurs, the device number information and the location of the corresponding device are retrieved and sent to the PTZ. The PTZ plans a path based on the location of the device and a pre-built map, and moves along the path to perform inspection and maintenance. When the robot receives the fault confirmation command, it receives the target device information, then retrieves the location information of the target device, marks it in the pre-built map, creates a forward path, and sends an execution command to the robot. The robot receives the command and executes it, and starts to move forward along the path. While moving forward, the inspection camera installed on the robot collects information about the surrounding environment in real time. If there are obstacles in both far and near areas in the image collection and analysis of the surrounding environment, the gimbal analyzes the obstacles in the far and near areas, and uses the obstacles in the far-distance area as the basis for the robot's next path planning to determine whether the robot can overcome obstacles in the near-distance area. If an insurmountable obstacle is detected in front of the current near distance, the robot's obstacle avoidance path is planned by comparing the road area on the left and right sides of the obstacle, and a semantic segmentation network is used to output a mobile control strategy. The specific control strategy is: when the outputs of the two branches in the semantic segmentation network are the same, the output direction is executed; when they are different, the execution direction is determined by the segmentation branch; Specifically, when the output results of the two branches in the semantic segmentation network show turning and going straight, if there is no obstacle, the classification output is executed, otherwise the segmentation branch output is executed; When turning and stopping occur, the turning is executed; when going straight and stopping occur, the stopping is executed.

2. A substation intelligent robot operation method according to claim 1, characterized in that: The operating status of the equipment includes: equipment appearance, equipment temperature, display data of instruments and instruments, and the opening and closing status of the switch; the equipment information is the number information of each equipment.

3. A substation intelligent robot operation method according to claim 1, characterized in that: The control center is wirelessly connected to the pan-tilt platform, which is arranged on an intelligent robot. The location distribution information of equipment in the substation is collected by a laser sensor and uploaded to the pan-tilt platform to create a three-dimensional map of the equipment distribution in the substation.

4. A substation intelligent robot operation method according to claim 1, characterized in that: If the control center determines that a device failure has occurred, the control center will retrieve the device number information and the corresponding location information of the failed device and send them to the PTZ. The PTZ will plan a path based on the device location information and the created map, and upload its own location to the control center in real time.

5. A method for operating a substation intelligent robot according to claim 4, characterized in that: During the movement, the binocular camera is used to collect wide-angle equipment information in the substation in real time; the inspection camera is used to collect real-time image information of the robot's surrounding environment and upload it to the gimbal.

6. A method for operating a substation intelligent robot according to claim 5, characterized in that: The binocular camera is fixedly installed in front of the robot body and adopts a short-focus dual camera. The inspection camera is installed on the robot's pan-tilt head and adopts a variable-focus camera. Through the movement of the pan-tilt head and the change of focal length, accurate collection of the surrounding environment image is achieved.

Images