Key point target identification system for hot-line work robot

Abstract

The invention discloses a key point target recognition system for a hot-line work robot, and belongs to the technical field of hot-line work robot target recognition. The system further comprises an analysis model and a network management model. The target detection model and the analysis model are in bidirectional connection, and the output end of the analysis model is connected with the input end of the network management model; according to the method, the features extracted by the backbone network are fused; secondly, extracting more representative multi-scale features through a TUM (thin U-shaped module) and a FFM (feature fusion module); and finally, fusing the multi-level features through a scale feature aggregation module SFAM to obtain a multi-level feature pyramid for prediction of a final stage, identifying the operation key points of the robot through the system, effectively identifying problems and hidden dangers existing in rotation and movement of different operation tools and operation arms, further effectively coping with and avoiding in time, and improving the working efficiency. Hidden dangers are reduced, and the operation safety is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of target recognition for live-line working robots, and in particular, relates to a key point target recognition system for live-line working robots. Background Art

[0002] With the growth of the national economy, the power industry has developed rapidly. At present, the State Grid Corporation operates more than 400,000 distribution lines with a length of more than 3 million kilometers, of which overhead lines are still the main grid of the distribution network. When the distribution network is manually live-lined for maintenance, the maintenance personnel usually need to wear airtight insulating clothing, carry various equipment and tools, and work at high altitudes for a long time, which has high safety risks and low working efficiency. If it is under bad weather or geographical conditions, the operation is more difficult and risky. Distribution network hardware such as drainage wire clamps and grounding rings need to be replaced regularly. Therefore, it is very necessary to develop a distribution network live-line operation robot system that can be flexibly and safely operated, which can not only avoid unnecessary economic losses, but also play a vital role in the safety of national electricity use and the stable development of the economy; introducing live-line operation robots into the power industry to replace manual power maintenance and repair work can effectively avoid the occurrence of casualties during live-line operations, and can greatly improve the operating efficiency of the power industry.

[0003] A target recognition method based on a deep learning network and a live working robot were disclosed on 2017-08-18, with the publication number CN107053168A. A target recognition method based on a deep learning network and a live working robot, initialize the weights and biases of each layer in the convolutional neural network, input two-dimensional image data, perform convolution processing to obtain C1 layer data; perform maximum pooling processing on the C1 layer data to reduce the size of the data and obtain S2 layer data; perform convolution processing on the S2 layer data to obtain C3 layer data; perform maximum pooling processing on the C3 layer data to further reduce the data size and obtain S4 layer data; perform convolution processing on the S4 layer data to form C5 layer data; use the C5 layer data as input, fully connect it with the H6 layer, and the output layer is fully connected with the H6 layer, and output the learning results of the target features; use the back propagation algorithm to fine-tune the weights and biases to complete the network learning. The present invention has high target recognition rate and good robustness under complex backgrounds.

[0004] However, the above-mentioned identification method still has certain shortcomings in actual operation. During the lifting, clamping, positioning and operation of the robot arm, the robot arm may make false touches and unstable clamping due to unclear identification of its own operating key points. There may be certain safety hazards during the operation, and the operational safety needs to be further improved. Summary of the invention

[0005] To solve the above problems and technical defects, the embodiments of the present application adopt the following technical solutions: a key point target recognition system for a live working robot, including a target detection model;

[0006] Also includes analytical models and network management models;

[0007] The target detection model and the analysis model are bidirectionally connected, and the output end of the analysis model is connected to the input end of the network management model;

[0008] Object detection models are used to extract image features;

[0009] The analysis model is used to analyze and judge the extracted image features, and classify them after the analysis and judgment;

[0010] The network management model is used to receive the classified features and output the final detection and recognition results.

[0011] Preferably, the target detection model includes a feature fusion module FFM, a thinning U-shaped module TUM and a scale feature aggregation module SFAM; the steps of constructing the target detection model are as follows:

[0012] S1, extract shallow features and deep features in the backbone network;

[0013] S2 and FFMv1 fuse the shallow and deep features extracted by the backbone network to obtain the base feature;

[0014] S3. Stack multiple TUMs and FFMv2s. Each TUM generates multiple feature maps of different scales. Each FFMv2 fuses the base feature and the output of the previous TUM and gives it to the next TUM as level input.

[0015] S4 and SFAM aggregate multi-level and multi-scale features through scale-wise concatenation and channel-wise attention.

[0016] Furthermore, the output formula of the level is as follows:

[0017]

[0018] T is the unit time, X is the target feature, and l is the number of features.

[0019] Preferably, the feature fusion module FFM is used to fuse features of different levels in M2Det, and the specific method of the fusion is as follows: first compress the number of channels through 1x1 convolution, and then perform splicing.

[0020] Furthermore, FFMv1 uses feature maps of two different scales as input and adds an upsampling operation to adjust the size before the splicing operation; the two inputs of FFMv2 have the same scale and do not adjust the size.

[0021] Preferably, the output of each decoder in the refined U-type module TUM plus a 1x1 convolution after upsampling and element addition operations together constitutes the multi-scale output of the TUM; the output of each TUM together constitutes multi-level & multi-scale features, and TUM provides low-level features and high-level features respectively.

[0022] Preferably, the scale feature aggregation module SFAM is used to aggregate the multi-level multi-scale features generated by the refinement U-type module TUM to construct a multi-level feature pyramid.

[0023] Furthermore, the backbone network adopts VGG-16 and ResNet-101.

[0024] Furthermore, the target detection model includes 8 refined U-shaped modules TUM, each refined U-shaped module TUM includes 5 strided convolution kernels, and the output of each TUM includes 6 features of different scales.

[0025] Furthermore, the encoder of the refined U-shaped module TUM uses a convolutional layer of size 3×3 and stride 2 for feature extraction.

[0026] Compared with the prior art, the beneficial effects of the embodiments of the present application are:

[0027] (1) The present application fuses the features extracted by the backbone network; then extracts more representative multi-scale features through the refinement U-shaped module TUM and the feature fusion module FFM; finally, fuses the multi-level features through the scale feature aggregation module SFAM to obtain a multi-level feature pyramid for the final stage of prediction. Through the system, the key points of the robot operation can be identified, and the problems and hidden dangers of the rotation and movement of different working tools and working arms can be effectively identified, so that timely and effective response and avoidance can be achieved, reducing the occurrence of hidden dangers and improving operational safety.

[0028] (2) The present application extracts and detects key point target features by constructing a template detection model, and analyzes and judges the extracted image features by utilizing an analysis model, and finally identifies the final detection results through a network management model. The template detection model constructed in the recognition system can effectively eliminate the problem of unclear key point recognition during the operation process, thereby effectively eliminating the safety hazards in the operation process, and effectively improving operational safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In the attached picture:

[0030] Figure 1 A schematic diagram of the system structure of an embodiment of the present application;

[0031] Figure 2 The system of the feature fusion module FFM in the embodiment of the present application Figure 1 ;

[0032] Figure 3 The system of the feature fusion module FFM in the embodiment of the present application Figure 2 ;

[0033] Figure 4 A system diagram for refining the U-shaped module TUM in the embodiment of the present application;

[0034] Figure 5 This is a system diagram of the medium-scale feature aggregation module SFAM in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0036] Example

[0037] like Figure 1 As shown, a key point target recognition system for a live working robot includes a target detection model;

[0038] Also includes analytical models and network management models;

[0039] The target detection model and the analysis model are bidirectionally connected, and the output end of the analysis model is connected to the input end of the network management model;

[0040] Object detection models are used to extract image features;

[0041] The analysis model is used to analyze and judge the extracted image features, and classify them after the analysis and judgment;

[0042] The network management model is used to receive the classified features and output the final detection and recognition results.

[0043] Reference Figure 1-Figure 5, the target detection model includes a feature fusion module FFM, a refinement U-shaped module TUM and a scale feature aggregation module SFAM; the steps for building the target detection model are as follows:

[0044] S1, extract shallow features and deep features in the backbone network;

[0045] S2 and FFMv1 fuse the shallow and deep features extracted by the backbone network to obtain the base feature;

[0046] S3. Stack multiple TUMs and FFMv2s. Each TUM generates multiple feature maps of different scales. Each FFMv2 fuses the base feature and the output of the previous TUM and gives it to the next TUM as level input.

[0047] S4 and SFAM aggregate multi-level and multi-scale features through scale-wise concatenation and channel-wise attention.

[0048] The output formula of level is as follows:

[0049]

[0050] T is the unit time, X is the target feature, and l is the number of features.

[0051] The feature fusion module FFM is used to fuse features of different levels in M2Det. The specific method of fusion is as follows: first compress the number of channels through 1x1 convolution and then perform splicing.

[0052] FFMv1 uses two feature maps of different scales as input and adds an upsampling operation to adjust the size before the splicing operation; the two inputs of FFMv2 have the same scale and are not resized.

[0053] The refined U-type module TUM adds a 1x1 convolution after upsampling and element addition operations; the output of each decoder in the refined U-type module TUM together constitutes the multi-scale output of the TUM; the output of each TUM together constitutes the multi-level & multi-scale features, and TUM provides low-level features and high-level features respectively.

[0054] The scale feature aggregation module SFAM is used to aggregate the multi-level and multi-scale features generated by the refinement U-type module TUM to construct a multi-level feature pyramid.

[0055] The backbone network uses VGG-16 and ResNet-101.

[0056] The target detection model contains 8 refined U-shaped modules TUM, each of which contains 5 strided convolution kernels, and the output of each TUM contains 6 features of different scales.

[0057] The encoder of the Thinned U-Module TUM uses a 3×3 convolutional layer with a stride of 2 for feature extraction.

[0058] A more effective feature pyramid structure network method is adopted. First, the features extracted by the backbone network are fused; then more representative multi-scale features are extracted through the refinement U-shaped module TUM and the feature fusion module FFM; finally, multi-level features are fused through the scale feature aggregation module SFAM to obtain a multi-level feature pyramid for the final stage of prediction. Through this system, the key points of robot operations can be identified, and the problems and hidden dangers of the rotation and movement of different working tools and working arms can be effectively identified, so that timely and effective responses and avoidance can be made, reducing the occurrence of hidden dangers and improving operational safety.

[0059] By using a template detection model to extract and detect key point target features, and using an analysis model to analyze and judge the extracted image features, and finally identifying the final detection results through a network management model, the template detection model constructed in this recognition system can effectively eliminate the problem of unclear key point recognition in the operation process during operation, and thus can effectively eliminate the safety hazards in the operation process, thereby effectively improving operational safety.

[0060] The above-mentioned embodiments only express the preferred implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for ordinary technicians in this field, several modifications, improvements and substitutions can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A key point target recognition system for a live working robot, characterized in that: Includes object detection models; Also includes analytical models and network management models; The target detection model and the analysis model are bidirectionally connected, and the output end of the analysis model is connected to the input end of the network management model; Object detection models are used to extract image features; The analysis model is used to analyze and judge the extracted image features, and classify them after the analysis and judgment; The network management model is used to receive the classified features and output the final detection and recognition results.

2. A key point target recognition system for a live working robot according to claim 1, characterized in that: The target detection model includes a feature fusion module FFM, a thinning U-shaped module TUM and a scale feature aggregation module SFAM; the steps of constructing the target detection model are as follows: S1, extract shallow features and deep features in the backbone network; S2 and FFMv1 fuse the shallow and deep features extracted by the backbone network to obtain the base feature; S3. Stack multiple TUMs and FFMv2s. Each TUM generates multiple feature maps of different scales. Each FFMv2 fuses the base feature and the output of the previous TUM and gives it to the next TUM as level input. S4 and SFAM aggregate multi-level and multi-scale features through scale-wise concatenation and channel-wise attention.

3. A key point target recognition system for a live working robot according to claim 2, characterized in that: The output formula of the level is as follows: T is the unit time, X is the target feature, and l is the number of features.

4. A key point target recognition system for a live working robot according to claim 2, characterized in that: The feature fusion module FFM is used to fuse features of different levels in M2Det. The specific method of the fusion is as follows: first compress the number of channels through 1x1 convolution, and then perform splicing.

5. A key point target recognition system for a live working robot according to claim 4, characterized in that: FFMv1 uses two feature maps of different scales as input and adds an upsampling operation to adjust the size before the splicing operation; the two inputs of FFMv2 have the same scale and are not resized.

6. A key point target recognition system for a live working robot according to claim 2, characterized in that: The refined U-type module TUM adds a 1x1 convolution after upsampling and element addition operations; the output of each decoder in the refined U-type module TUM together constitutes the multi-scale output of the TUM; the output of each TUM together constitutes a multi-level & multi-scale feature, and the TUM provides low-level features and high-level features respectively.

7. A key point target recognition system for a live working robot according to claim 2, characterized in that: The scale feature aggregation module SFAM is used to aggregate the multi-level and multi-scale features generated by the refinement U-type module TUM to construct a multi-level feature pyramid.

8. A key point target recognition system for a live working robot according to claim 2, characterized in that: The backbone network adopts VGG-16 and ResNet-101.

9. A key point target recognition system for a live working robot according to claim 2, characterized in that: The target detection model includes 8 refined U-shaped modules TUM, each refined U-shaped module TUM includes 5 strided convolution kernels, and the output of each TUM includes 6 features of different scales.

10. A key point target recognition system for a live working robot according to claim 4, characterized in that: The encoder of the thinned U-shaped module TUM uses a convolutional layer of size 3×3 and stride 2 for feature extraction.

Images