Substation personnel safety distance measurement method based on multi-view analysis

Abstract

The invention discloses a transformer substation personnel safety distance measurement method based on multi-view analysis. Comprising the steps that multi-view images are collected and preprocessed, a standardized multi-view image set is obtained through preprocessing, the standardized multi-view image set is input into a constructed long-distance grouping Transform three-dimensional reconstruction network, a voxel-based three-dimensional reconstruction result is generated, and therefore multi-view depth information is obtained through calculation. Then inputting the multi-view depth information and the collected standardized multi-view image into a semantic segmentation network based on RGB-D three-flow coordinate attention to obtain a semantic segmentation result of the worker and the power equipment, and calculating distance information of the worker and the power equipment by combining the semantic segmentation result and the depth information; therefore, whether the worker keeps a safe distance from the power equipment is judged. The method aims at improving the safety in the construction process of the transformer substation, and safe distance measurement of workers and power equipment is achieved through accurate spatial data and semantic information.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method for measuring the safety distance of transformer substation personnel based on multi-view analysis. Background Art

[0002] As an important part of the power system, substations play an important role in transforming and distributing electric energy. In recent years, as the demand for electricity in society has increased, the operation and maintenance requirements of substations have also become higher and higher. In order to ensure the safe, stable and efficient operation of the power system, the maintenance and management of substation equipment has become particularly critical.

[0003] However, during the maintenance and management process, due to the complexity of high-voltage electrical equipment in the substation area and the potential for multiple risk factors, the staff's lack of grasp of the safe distance during management or construction may cause serious safety accidents. Therefore, it is very important to provide a safety warning for the distance between the workers and the high-voltage electrical equipment and large mechanical devices of the substation during the construction process. However, the existing substation identification lacks spatial information and has a low recognition accuracy, resulting in untimely safety warnings. Therefore, it is urgent to use the existing monitoring system of the substation to realize the intelligent identification and warning of substation construction safety, so as to ensure the construction safety of substation workers. Summary of the invention

[0004] Purpose of the invention: In response to the problems in the background technology, the present invention discloses a method for measuring the safe distance of substation personnel based on multi-view analysis, which uses a three-dimensional reconstruction network of a long-distance grouping Transformer to analyze the input multi-view image set, reconstructs the target area in three dimensions and obtains voxel data and depth information, and uses a three-stream coordinate attention network of RGB-D outdoor semantic segmentation to detect the target area based on depth information, obtains semantic segmentation results and distance information of people and power equipment, and realizes high-precision distance measurement and intelligent safety warning between people and equipment.

[0005] Technical solution: The present invention discloses a method for measuring the safety distance of substation personnel based on multi-view analysis, comprising the following steps:

[0006] Step S1: Collect and preprocess multi-view images of the target distribution area of ​​the substation to obtain a standardized multi-view image set C = {c1,...,c n};

[0007] Step S2: The preprocessed standardized multi-view image set C = {c1, ..., c nInput a 3D reconstruction network based on a long-distance grouping Transformer to generate a voxel-based 3D reconstruction result, thereby calculating the depth information of multiple views; the 3D reconstruction network based on the long-distance grouping Transformer includes an encoder based on a visual Transformer architecture, and the encoder is divided into two stages: an independent branch stage and an interleaved branch stage, in which N1 standard Transformer blocks are used to process all views in parallel in the independent branch stage; in the interleaved branch stage, a basic unit consisting of N2 Transformer blocks and Transformer blocks with long-distance grouping attention is used to establish a connection between multiple views, and inter-view feature markers are used to distinguish features of different views;

[0008] Step S3: Input the multi-view depth information and the collected standardized multi-view images into the RGB-D semantic segmentation network based on three-stream coordinate attention to obtain the semantic segmentation results of the staff and the power equipment, and combine the semantic segmentation results with the depth information to calculate the distance information between the staff and the power equipment, so as to judge whether the staff maintains a safe distance from the power equipment; the RGB-D semantic segmentation network based on three-stream coordinate attention adopts a three-stream encoder-decoder architecture, the encoder is used to extract RGB-D features, the decoder decodes step by step, and uses the multimodal feature fusion module based on coordinate attention to fuse the depth image and the visible light image.

[0009] Furthermore, the specific process of calculating the depth information of the multiple views in step 2 is as follows:

[0010] Step S21: constructing the encoder based on the visual Transformer architecture, and calculating the differential features X of different views;

[0011] Step S22: The calculated differential features X of different views are passed through a progressively upsampled decoder to generate a high-resolution reconstruction result and depth information;

[0012] Step S23: Use Dice loss as the loss function to handle the imbalance of voxel occupancy.

[0013] Furthermore, the long-distance grouping attention is used to establish the correlation between different views, specifically:

[0014] Given a specific i-th view feature Where P is the number of tokens, D e is the token dimension, based on X i The length and width of X are uniformly and regularly sampled to obtain g groups, and the tokens in each group are not adjacent in the original view features. i Expressed as in represents the token from the i-th view of the j-th group, and N represents the number of input views. Tokens with the same color belong to a group, and each group includes tokens within the view and inter-view tokens at corresponding positions. After grouping, the self-attention operation will process the tokens in each group independently.

[0015] The inter-view feature tagging consists of a convolutional layer to enhance the distinction between tokens of different views as follows:

[0016] Given the jth group of series tokens for the i-th view The token features are obtained through a trainable linear projection φ, which is used to predict V in the Transformer block. j The linear projection of shares: where g, P, D e Represent the number of groups, the number of tokens and the token dimension respectively, and the feature marker of the i-th view of the j-th group Obtained as follows:

[0017]

[0018] Define the inter-view feature markers as Where N is the number of views, for i Each view feature The long-range group attention with inter-view feature labeling is formulated as:

[0019]

[0020] Where Q j , K j , V j Respectively from The query, key and value of d are, d is the number of channels of the query, and the obtained attention maps are concatenated in the group dimension to complete an inter-view association calculation.

[0021] Furthermore, the generation of high-resolution reconstruction results in step S22 is specifically as follows:

[0022] The 4 high-resolution basic units extract the 4 3 The low-resolution feature map of size is gradually upsampled to 323. Each high-resolution basic unit contains 1 deconvolution layer and 2 Transformer blocks. The convolution and multi-head self-attention processing are complementary. At the same time, a skip connection is added to enhance the voxel correlation of the attention layer to the same token assignment through the convolution layer of local features.

[0023] Within the high-resolution basic unit, neighboring voxels are evenly grouped and combined in order to compress the number of tokens, given the voxel feature where v t is the t-th voxel token, T, D r denote the number of tokens and the token dimension respectively;

[0024] In the case of a resolution of S×S×S, define T=S×S×S, group each I×I×I adjacent voxel token to obtain the voxel feature in D' r =I 3 ×D r , thereby grouping the voxel features V into a size that can be processed by self-attention while retaining the original feature information at a relatively high resolution. Finally, the voxel features V are taken as input, and multi-head self-attention is applied to explore the voxel position relationship, outputting the voxel-based 3D reconstruction results and depth information.

[0025] Furthermore, Dice loss is used as a loss function to deal with the imbalance of voxel occupancy. The specific Dice loss is:

[0026]

[0027] Among them, p i , g i They represent the confidence of the i-th voxel grid in the reconstructed volume and the true value, respectively.

[0028] Furthermore, the encoder part of the three-stream encoder-decoder architecture in S3 uses ResNet-50 as the backbone structure, removes the average pooling layer and the fully connected layer, performs feature extraction on the visible light image and the depth image respectively, and extracts five levels of feature output respectively as the input of the multimodal feature fusion module.

[0029] Furthermore, the multimodal feature fusion module based on coordinate attention is used to fuse the depth image and the visible light image. The specific process is as follows:

[0030] S31: The visible light image RGB and the depth image Depth are respectively sent to the ResNet-50 backbone network for feature extraction to obtain multi-scale features {R i |i=1,2,3,4,5},{D i |i=1,2,3,4,5}, as the input of the coordinate attention-based multimodal feature fusion module;

[0031] S32: The input of the first feature fusion module is R1 and D1, both of which have a scale of C×H×W. First, R1 and D1 are concatenated to obtain feature X1, which has a scale of 2C×H×W. Then the output feature X1 is sent to two different CA modules CA1 and CA2 respectively. CA1 uses global average pooling and CA2 uses global maximum pooling. Each CA module outputs 2 tensors. The output of CA1 is The output of CA2 is (i,j) is the final output result of the i-th output attention tensor corresponding to the j-th CA module;

[0032] S33: According to the dimension Group them and add them pixel by pixel to get T h , T w ;

[0033] S34: Apply two sigmoid functions to activate T h , T w , the output is represented by G h , G w , the scales are C×H×1 and C×1×W respectively;

[0034] S35: The final output G h , G w , respectively fused with the initial visible light image feature R1 and the depth image feature D1, and then the fusion results are added pixel by pixel to obtain the final output OUT1;

[0035] S36: Repeat S32 to S32 to obtain the output OUT2 of the second feature fusion module whose input is R2, D2, and OUT1. At the end of the second feature fusion module, OUT1 and OUT2 are added element by element to obtain the final

[0036] S37: Repeat S32 to S36 to obtain The output OUT1 of the first four fusion modules, Through Conv 1×1 (·) The convolution operation with convolution kernel 1 performs channel alignment to obtain outputs OUT'1, OUT'2, OUT'3, OUT'4, and the output of the last module After the atrous spatial pyramid pooling module, the output OUT'5 is obtained;

[0037] S38: For the output OUT'5 of the atrous spatial pyramid pooling module, an upsampling operation is performed and fused pixel by pixel with the first four layers of features in sequence to achieve multi-scale feature fusion. The formula is as follows:

[0038] F4=Up(OUT'5)+OUT'4

[0039] F3=Up(F4)+OUT'3

[0040] F2=Up(F3)+OUT'2

[0041] F1=Up(F2)+OUT'1

[0042] Among them, Up(·) is an upsampling operation to adjust the size of the feature;

[0043] S39: Finally, the fused high-resolution feature map is converted into the final prediction output OUT = Conv(F1), where the number of output channels is the number of predicted categories. Conv(·) is used to integrate feature information and map it to specific category predictions to ensure the precision of the segmentation results.

[0044] Furthermore, for the CA1 module, two spatial pooling kernels of size (H, 1) and (1, W) are first used to perform global average pooling operations on X1 in the horizontal and vertical directions to obtain z h and z w Two features; z h and z w The features are connected and convolution operations are performed, and the BN function and Non-liner activation function are applied to obtain the feature f with a scale of Then f is split into two new independent tensors f along the spatial dimension h 、f w , the scales are Perform two more 1×1 convolution operations on the two tensors to get the final result The scales are C×H×1 and C×1×W respectively. For the CA1 module, the global maximum pooling operation is used, and the other operations are the same as the CA1 module.

[0045] Beneficial effects:

[0046] 1. The present invention analyzes the input multi-view image set through the 3D reconstruction network of long-distance grouping Transformer. In the stage of designing the 3D reconstruction network, the long-distance grouping attention mechanism and the progressive upsampling decoder are introduced to realize the efficient extraction of features of different scales in complex scenes and improve the accuracy of obtaining depth information.

[0047] 2. The present invention uses a three-stream coordinate attention network for RGB-D outdoor semantic segmentation to detect the target area based on depth information. In the stage of designing the semantic segmentation network for people and power equipment, a parallel three-stream structure, a multimodal feature fusion module and an embedded void space pyramid pool module are constructed, which realizes the full utilization and fusion of the characteristics of different modal data, and improves the network processing efficiency and the accuracy of target segmentation.

[0048] 3. In the implementation stage of substation personnel safety distance measurement, the present invention combines the three-dimensional reconstruction network of the long-distance grouping Transformer and the three-stream coordinate attention network of RGB-D outdoor semantic segmentation to achieve accurate ranging in outdoor changeable scenes, effectively improving the accuracy of ranging in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of a method for measuring the safety distance of substation personnel based on multi-view analysis proposed in the invention;

[0050] Figure 2 The 3D reconstruction network architecture of the long-distance grouping Transformer designed in the present invention;

[0051] Figure 3 This is the structure diagram of the long-distance group attention Transformer module designed in the present invention;

[0052] Figure 4 A high-resolution basic unit structure diagram designed in the present invention;

[0053] Figure 5 The three-stream coordinate attention network architecture for RGB-D semantic segmentation designed in this paper. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0055] The present invention discloses a method for measuring the safety distance of transformer substation personnel based on multi-view analysis, which specifically comprises the following steps:

[0056] Step 1: Use existing monitoring equipment, specifically a set of multi-view cameras, to collect and preprocess multi-view images of the distribution area of ​​the target substation to obtain a standardized multi-view image set C = {c1,...,c n}.

[0057] Step 2: The preprocessed standardized multi-view image set C = {c1,...,c n Input the 3D reconstruction network based on the long-distance grouping Transformer, and generate the voxel-based 3D reconstruction result after being processed by the encoder and decoder. The specific steps are as follows:

[0058] 21) Multi-view image set C = {c1,...,c n}The differentiated features X based on different views are obtained through an encoder based on the visual transformer architecture.

[0059] Combination Figure 2 , the preprocessed multi-view image set C = {c1,...,c n The input is an encoder based on the visual Transformer architecture, which is divided into two stages: an independent branch stage and an interleaved branch stage. The first stage consists of N1 standard Transformer blocks, which process all views in parallel using standard Transformer blocks. The second stage contains N2 basic units, which consist of Transformer blocks and Transformer blocks with long-range group attention to establish connections between views and use the inter-view feature labeling structure to help distinguish features from different views.

[0060] Given a specific i-th view feature Where P is the number of tokens, D e is the token dimension.

[0061] X-based i The length and width of X are uniformly and regularly sampled to obtain g groups, and the tokens in each group are not adjacent in the original view features. i Expressed as in represents the token from the i-th view of the j-th group, and N represents the number of input views. Tokens with the same color belong to a group, and each group contains not only the tokens within the view, but also the inter-view tokens at the corresponding positions. After grouping, the self-attention operation will process the tokens in each group independently, using the divide-and-conquer principle to reduce the difficulty of model training.

[0062] The inter-view feature tagging structure for enhancing the distinction between tokens of different views consists of a convolutional layer, specifically:

[0063] Given the jth group of series tokens for the i-th view The token features are obtained through a trainable linear projection φ. φ is used to predict V in this Transformer block. j The linear projection of is shared: where g, P, D e Represents the number of groups, the number of tokens, and the token dimension respectively. The feature markers of the i-th view of the j-th group It can be obtained as follows:

[0064]

[0065] Define the inter-view feature markers as Where N is the number of views. i Each view feature The long-range grouping attention with inter-view feature labeling structure can be formulated as:

[0066]

[0067] Where Q j , K j , V j Respectively from The query, key and value of d are, and d is the number of channels of the query. The obtained attention maps are concatenated in the group dimension to complete an inter-view association calculation.

[0068] 22) Combine Figure 3 The low-resolution differential feature X is used to generate a high-resolution voxel-based 3D reconstruction result through a progressively upsampled Transformer decoder. The 4 high-resolution basic units extracted by the encoder are 3 The low-resolution feature maps of size are gradually upsampled to 323. Each high-resolution basic unit contains 1 deconvolution layer and 2 Transformer blocks. The convolution and multi-head self-attention processing are complementary. At the same time, a skip connection is added to enhance the voxel correlation of the attention layer to the same token assignment through the convolution layer of the local features.

[0069] In this method, within the high-resolution basic unit, neighboring voxels are evenly grouped and combined in order to compress the number of tokens. where v t is the t-th voxel token, T, D r denote the number of tokens and the token dimension respectively.

[0070] In the case of a resolution of S×S×S, define T=S×S×S. Group each I×I×I adjacent voxel token to obtain the voxel feature in D' r =I 3 ×D r , thereby grouping the voxel features V into a size that can be processed by self-attention. Finally, the voxel features V are taken as input, and multi-head self-attention is applied to explore the voxel position relationship and output the voxel-based 3D reconstruction result.

[0071] 23) Use Dice loss as the loss function to handle highly imbalanced voxel occupancy. Dice loss not only takes into account the intersection (the correct part of the prediction) and the union (all predicted parts), but also includes the processing of negative examples (the case where the prediction is empty and the actual is empty) to ensure that the model can balance the prediction of dense and sparse voxel areas, and further learn how to perform effective voxel segmentation and reconstruction in unbalanced data.

[0072]

[0073] where p i , g i They represent the confidence of the i-th voxel grid in the reconstructed volume and the true value, respectively.

[0074] Step 3: Combine Figure 2 , the standardized multi-view image set C = {c1,...,c n The RGB-D segmentation network based on three-stream coordinate attention is used to input the depth information and semantic segmentation results of people and power equipment and the distance information.

[0075] 31) The standardized multi-view image set C = {c1,...,c n The network adopts a three-stream encoder-decoder architecture, where the encoder is used to extract RGB-D features and the decoder decodes them step by step. In the encoder part, ResNet-50 is used as the backbone structure, and the average pooling layer and the fully connected layer are removed. The visible light image RGB and the depth image Depth are respectively sent to the ResNet-50 backbone network for feature extraction to obtain the multi-scale feature {R i |i=1,2,3,4,5},{D i |i=1,2,3,4,5}, as the input of the coordinate attention based multimodal feature fusion module.

[0076] 32) The coordinate attention-based multimodal feature fusion module is used to fuse the information of the depth image and the visible light image. The input of the last four fusion modules includes the output of the previous fusion branch in addition to the visible light image features and the depth image features. The output of the first four fusion modules is directly fed into the decoder, and the output of the last module is fed into the decoder after passing through the atrous spatial pyramid pooling module.

[0077] 321) The input of the first feature fusion module is R1, D1, both of which have a scale of C×H×W. First, the channel concatenation operation is performed on R1 and D1 to obtain feature X1, with a scale of 2C×H×W. Then the output feature X1 is sent to two different CA modules CA1 and CA2 respectively. CA1 uses global average pooling and CA2 uses global maximum pooling. Each CA module outputs 2 tensors. The output of CA1 is The output of CA2 is (i,j) is the final output result corresponding to the i-th output attention tensor of the j-th CA module.

[0078] For the CA1 module, two spatial pooling kernels of size (H, 1) and (1, W) are first used to perform global average pooling operations on X1 in the horizontal and vertical directions respectively. The calculation formula is:

[0079] z h =GAP (H,1) (X1)

[0080] z w =GAP (1,W) (X1)

[0081] Among them, GAP (H,1) (·) indicates global average pooling along the height direction, GAP (1,W) (·) is global average pooling along the width direction, at this time z h The scale is 2C×H×1,z w The dimensions are 2C×1×W.

[0082] z h and z w The features are connected and convolution operations are performed, and BatchNormalization normalization operations and Non-liner activation function operations are applied to obtain feature f, with a scale of The scale is calculated as follows:

[0083] f=φ(F(Concat((z h ) T ,z w )))

[0084] Where r is the reduction ratio used to control the block size, (·) T represents the transformation function, Concat(·) represents the connection operation, F(·) represents the 1×1 convolution operation, and φ(·) represents the BN function and Non-liner activation function operation.

[0085] Then f is split into two new independent tensors f along the spatial dimension h , f w , the scales are Perform two more 1×1 convolution operations F on the two tensors h and F w , so the final result y h and w , the scales are C×H×1 and C×1×W respectively, and the calculation formula is as follows:

[0086] y h =F h ((f h ) T )

[0087] y w =F w ((f w ) T )

[0088] 322) According to the dimension Group them and add them pixel by pixel to get T h , T w , the calculation formula is:

[0089]

[0090] Where + is a pixel-by-pixel addition operation.

[0091] 323) Apply two sigmoid functions to activate feature T h and T w , the output can be expressed as G h , the scale is C×H×1, G w , the scale is C×1×W, and the calculation formula is as follows

[0092] G h =σ(T h )

[0093] G w =σ(T w )

[0094] Where σ represents the sigmoid function activation operation.

[0095] 324) Finally, the output G h , G w , respectively fused with the initial visible light image feature R1 and the depth image feature D1, and then the fusion results are added pixel by pixel to obtain the final output OUT1. The calculation formula is as follows:

[0096]

[0097] OUT1=RGB CA +Depth CA

[0098] in Represents pixel-by-pixel multiplication.

[0099] 325) According to steps 321) to 324), the output OUT2 of the second feature fusion module with inputs R2, D2, and OUT1 can be obtained. Different from the first feature fusion module, the second feature fusion module needs to add OUT1 and OUT2 element by element at the end to obtain the final The formula is:

[0100]

[0101] in The stride of the 1×1 convolution operation is 2, the number of OUT1 channels is increased from C to 2C to match the number of channels of OUT2, and the spatial size of the feature map is downsampled from H×W to To match the space size of OUT2,

[0102] 327) According to steps 321) to 325), it can be obtained

[0103] 328) The output of the last module After the atrous spatial pyramid pooling module, the output OUT'5 is obtained, and the calculation formula is as follows:

[0104] s1=Conv 1×1 (OUT4)

[0105]

[0106] s5=GlobalPool(OUT4)

[0107] OUT'5=Concat(s1,s2,s3,s4,s5)

[0108] Among them, Conv 1×1 (·) is a standard 1×1 convolution, They represent dilated convolution operations with a convolution kernel of 3 and dilation rates of 6, 12, and 18 respectively. GlobalPool(·) indicates a global pooling operation on the input feature map. Concat(·) indicates concatenating multiple feature maps in the channel dimension.

[0109] S33) The outputs of the first four fusion modules are aligned through 1×1 convolution to ensure that the number of channels of features at different levels is consistent, which is convenient for subsequent pixel-by-pixel addition. The output of the dilated space pyramid pool module is upsampled and fused with the first four layers of features pixel by pixel in turn to achieve multi-scale feature fusion. Finally, the convolution layer and activation function are applied to convert the fused high-resolution feature map into the final prediction output.

[0110] In one embodiment of the present invention, step S33 may specifically include:

[0111] 331) The output OUT1 of the first four fusion modules, Through Conv 1×1 (·) The convolution operation with a convolution kernel of 1 performs channel alignment so that the channels are all adjusted to C. The formula is as follows:

[0112] OUT'1=Conv1×1 (OUT1)

[0113]

[0114] 332) For the output OUT'5 of the atrous spatial pyramid pooling module, an upsampling operation is used to fuse it pixel by pixel with the features of the first four layers in turn to achieve multi-scale feature fusion. The formula is as follows:

[0115] F4=Up(OUT'5)+OUT'4

[0116] F3=Up(F4)+OUT'3

[0117] F2=Up(F3)+OUT'2

[0118] F1=Up(F2)+OUT'1

[0119] Among them, Up(·) is an upsampling operation to adjust the size of the feature.

[0120] 333) Finally, the fused high-resolution feature map is converted into the final prediction output, and the formula is as follows:

[0121] OUT=Conv(F1)

[0122] The number of output channels is the number of predicted categories, and Conv(·) is used to integrate feature information and map it to specific category predictions to ensure the precision of the segmentation results.

[0123] Among them, F1 is the multi-scale fusion feature, OUT is the semantic segmentation output result, and Conv(·) is used to integrate feature information and map it to specific category predictions to ensure the precision of the segmentation result.

[0124] 334) The final semantic segmentation result obtained in step 333) is combined with the depth information to calculate the distance between the operator and the power equipment, and determine whether the operator is in a dangerous state from the power equipment based on the safety distance threshold.

[0125] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for measuring the safety distance of substation personnel based on multi-view analysis, characterized in that: The following steps are involved: Step S1: Collect and preprocess multi-view images of the target distribution area of ​​the substation to obtain a standardized multi-view image set C = {c1,...,c n }; Step S2: The preprocessed standardized multi-view image set C = {c1, ..., c n Input a 3D reconstruction network based on a long-distance grouping Transformer to generate a voxel-based 3D reconstruction result, thereby calculating the depth information of multiple views; the 3D reconstruction network based on the long-distance grouping Transformer includes an encoder based on a visual Transformer architecture, and the encoder is divided into two stages: an independent branch stage and an interleaved branch stage, in which N1 standard Transformer blocks are used to process all views in parallel in the independent branch stage; in the interleaved branch stage, a basic unit consisting of N2 Transformer blocks and Transformer blocks with long-distance grouping attention is used to establish a connection between multiple views, and inter-view feature markers are used to distinguish features of different views; Step S3: Input the multi-view depth information and the collected standardized multi-view images into the RGB-D semantic segmentation network based on three-stream coordinate attention to obtain the semantic segmentation results of the staff and the power equipment, and combine the semantic segmentation results with the depth information to calculate the distance information between the staff and the power equipment, so as to judge whether the staff maintains a safe distance from the power equipment; the RGB-D semantic segmentation network based on three-stream coordinate attention adopts a three-stream encoder-decoder architecture, the encoder is used to extract RGB-D features, the decoder decodes step by step, and uses the multimodal feature fusion module based on coordinate attention to fuse the depth image and the visible light image.

2. The method for measuring the safety distance of substation personnel based on multi-view analysis according to claim 1 is characterized in that: The specific process of calculating the depth information of multiple views in step 2 is as follows: Step S21: constructing the encoder based on the visual Transformer architecture, and calculating the differential features X of different views; Step S22: The calculated differential features X of different views are passed through a progressively upsampled decoder to generate a high-resolution reconstruction result and depth information; Step S23: Use Dice loss as the loss function to handle the imbalance of voxel occupancy.

3. The method for measuring the safety distance of substation personnel based on multi-view analysis according to claim 2 is characterized in that: The long-distance group attention is used to establish the correlation between different views, specifically: Given a specific i-th view feature Where P is the number of tokens, D e is the token dimension, based on X i The length and width of X are uniformly and regularly sampled to obtain g groups, and the tokens in each group are not adjacent in the original view features. i Expressed as i=1,...,N,where represents the token from the i-th view of the j-th group, and N represents the number of input views; Tokens with the same color belong to a group. Each group includes tokens within the view and inter-view tokens at corresponding positions. After grouping, the self-attention operation will process the tokens in each group independently. The inter-view feature tagging consists of a convolutional layer to enhance the distinction between tokens of different views as follows: Given the jth group of series tokens for the i-th view The token features are obtained through a trainable linear projection φ, which is used to predict V in the Transformer block. j The linear projection of is shared: where g, P, D e Represent the number of groups, the number of tokens and the token dimension respectively, and the feature marker of the i-th view of the j-th group Obtained as follows: Define the inter-view feature markers as j = 1, ..., g, where N is the number of views. i Each view feature The long-range group attention with inter-view feature labeling is formulated as: Where Q j , K j , V j Respectively from The query, key and value of d are, d is the number of channels of the query, and the obtained attention maps are concatenated in the group dimension to complete an inter-view association calculation.

4. The method for measuring the safety distance of substation personnel based on multi-view analysis according to claim 2 is characterized in that: The generation of high-resolution reconstruction results in step S22 is specifically as follows: The 4 high-resolution basic units extract the 4 3 The low-resolution feature map of size is gradually upsampled to 323. Each high-resolution basic unit contains 1 deconvolution layer and 2 Transformer blocks. The convolution and multi-head self-attention processing are complementary. At the same time, a skip connection is added to enhance the voxel correlation of the attention layer to the same token assignment through the convolution layer of local features. Within the high-resolution basic unit, neighboring voxels are evenly grouped and combined in order to compress the number of tokens, given the voxel feature where v t is the t-th voxel token, T, D r denote the number of tokens and the token dimension respectively; In the case of a resolution of S×S×S, define T=S×S×S, group each I×I×I adjacent voxel token to obtain the voxel feature in D' r =I 3 ×D r , thereby grouping the voxel features V into a size that can be processed by self-attention while retaining the original feature information at a relatively high resolution. Finally, the voxel features V are taken as input, and multi-head self-attention is applied to explore the voxel position relationship, outputting the voxel-based 3D reconstruction results and depth information.

5. The method for measuring the safety distance of substation personnel based on multi-view analysis according to claim 4 is characterized in that: Dice loss is used as a loss function to deal with the imbalance of voxel occupancy. The specific Dice loss is: Among them, p i , g i They represent the confidence of the i-th voxel grid in the reconstructed volume and the true value, respectively.

6. The method for measuring the safety distance of substation personnel based on multi-view analysis according to claim 1 is characterized in that: The encoder part of the three-stream encoder-decoder architecture in S3 uses ResNet-50 as the backbone structure, removes the average pooling layer and the fully connected layer, performs feature extraction on the visible light image and the depth image respectively, and extracts five levels of feature outputs respectively as the input of the multimodal feature fusion module.

7. The method for measuring the safety distance of substation personnel based on multi-view analysis according to claim 6 is characterized in that: The multimodal feature fusion module based on coordinate attention is used to fuse the depth image and the visible light image. The specific process is as follows: S31: The visible light image RGB and the depth image Depth are respectively sent to the ResNet-50 backbone network for feature extraction to obtain multi-scale features {R i |i=1,2,3,4,5},{D i |i=1,2,3,4,5}, as the input of the coordinate attention-based multimodal feature fusion module; S32: The input of the first feature fusion module is R1 and D1, both of which have a scale of C×H×W. First, R1 and D1 are concatenated to obtain feature X1, which has a scale of 2C×H×W. Then the output feature X1 is sent to two different CA modules CA1 and CA2 respectively. CA1 uses global average pooling and CA2 uses global maximum pooling. Each CA module outputs 2 tensors. The output of CA1 is The output of CA2 is (i,j) is the final output result of the i-th output attention tensor corresponding to the j-th CA module; S33: According to the dimension Group them and add them pixel by pixel to get T h , T w ; S34: Apply two sigmoid functions to activate T h , T w , the output is represented by G h , G w , the scales are C×H×1 and C×1×W respectively; S35: The final output G h , G w , respectively fused with the initial visible light image feature R1 and the depth image feature D1, and then the fusion results are added pixel by pixel to obtain the final output OUT1; S36: Repeat S32 to S32 to obtain the output OUT2 of the second feature fusion module whose input is R2, D2, and OUT1. At the end of the second feature fusion module, OUT1 and OUT2 are added element by element to obtain the final S37: Repeat S32 to S36 to obtain The output OUT1 of the first four fusion modules, Through Conv 1×1 (·) The convolution operation with convolution kernel 1 performs channel alignment to obtain outputs OUT′1, OUT′2, OUT′3, OUT′4. The output of the last module After the atrous spatial pyramid pooling module, the output OUT′5 is obtained; S38: For the output OUT5' of the dilated spatial pyramid pooling module, an upsampling operation is performed and fused pixel by pixel with the features of the first four layers in sequence to achieve multi-scale feature fusion. The formula is as follows: F4=Up(OUT′5)+OUT′4 F3=Up(F4)+OUT′3 F2=Up(F3)+OUT′2 F1=Up(F2)+OUT′1 Among them, Up(·) is an upsampling operation to adjust the size of the feature; S39: Finally, the fused high-resolution feature map is converted into the final prediction output OUT = Conv(F1), where the number of output channels is the number of predicted categories. Conv(·) is used to integrate feature information and map it to specific category predictions to ensure the precision of the segmentation results.

8. The method for measuring the safety distance of substation personnel based on multi-view analysis according to claim 7 is characterized in that: For the CA1 module, we first use two spatial pooling kernels of size (H, 1) and (1, W) to perform global average pooling operations on X1 in the horizontal and vertical directions to obtain z h and z w Two features; z h and z w The features are connected and convolution operations are performed, and the BN function and Non-liner activation function are applied to obtain the feature f with a scale of Then f is split into two new independent tensors f along the spatial dimension h 、f w , the scales are Perform two more 1×1 convolution operations on the two tensors to get the final result The scales are C×H×1 and C×1×W respectively. For the CA1 module, the global maximum pooling operation is used, and the other operations are the same as the CA1 module.

Images