10kV uninterruptible power operation safety distance detection method fusing AMF and EGC
By integrating the adaptive multi-scale feature network and optoelectrode geometric constraints, the high-precision and real-time problems of safe distance detection for 10kV non-off operation are solved, and high-precision and fast safe distance detection are achieved.
Patent Information
- Application Number
- CN202410876248.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-02
- Publication Date
- 2024-11-15
AI Technical Summary
The existing 10kV non-power-off operation safety distance detection method is difficult to achieve high-precision and real-time safety distance detection due to problems such as strong human subjectivity, large distance measurement error, large environmental changes, and complex operations.
Using a method of fusion adaptive multi-scale feature network (AMF) and opposing geometric constraints (EGC), images are acquired through industrial-grade binocular cameras, and segmented images of personnel and live bodies are extracted by example segmentation networks, and EGC is used to constrain feature matching points. Search range, AMF network extracts multi-scale features and calculates safe distances through parallax regression.
It realizes high-precision (94.6%) and real-time (28.9 frames/s) safe distance detection, meets the detection needs of 10kV non-off operation sites, and reduces errors and environmental impacts.
Smart Images

Figure CN118968395A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of electric power detection, and in particular relates to a 10kV non-stop operation safety distance detection method integrating AMF and EGC. Background Art
[0002] 10kV non-stop operation is widely used in power production due to its high efficiency and practicality. In order to ensure the life safety of operators, the State Grid has formulated strict standards for the safe distance of 10kV voltage level. The traditional distance measurement method is generally for safety inspectors to observe the distance between the operator's head, hands, arms, face and other key parts and live equipment with the naked eye and combine experience to judge whether it meets safety regulations. However, this method is difficult to detect the safe distance in real time and accurately due to its strong human subjectivity, thereby increasing the risk of electric shock accidents. Therefore, it is of great significance to study the safe distance detection method for 10kV non-stop operation.
[0003] At present, the research on this issue at home and abroad mainly focuses on the electric field strength ranging method, positioning technology ranging method and laser ranging method. The electric field strength ranging method determines the safe distance of the operator by evaluating the intensity distribution change of the electric field around the 10kV power equipment. However, this method is limited by the measurement accuracy of the electric field strength, resulting in a large ranging error in practical applications. The positioning technology ranging method uses Beidou and ultra-wideband wireless communication technology to monitor the position of the operator and equipment in real time and calculate the distance between the operator and the live equipment. This method cannot be widely used in actual production because it is greatly affected by environmental changes and the cost of initial construction and later maintenance is high. The laser ranging method has the advantage of high precision of point-to-point measurement, but it is difficult to promote in complex work sites due to its high operational complexity. With the rapid development of computer vision, the safe distance detection method based on image ranging technology has gradually become a research hotspot. It mainly includes the method of fusing monocular camera and laser radar and the method based on binocular stereo vision. The method of fusing camera and radar is greatly affected by changes in environmental factors and has a small ranging range. The binocular stereo vision-based method has shown significant advantages in the field of safe distance detection due to its rich semantic information, good ranging performance and low cost.
[0004] The most critical problem of binocular stereo vision-based methods is how to accurately match the corresponding pixels in the left and right images. However, in the outdoor large field of view environment of 10kV non-stop operation, the existing binocular stereo vision methods face many problems. For example, the depth of field of the scene is large and there are more complex elements, which leads to unclear pixel relationship information and blurred edge textures of key parts. This brings great difficulties to high-precision real-time distance detection.
[0005] In view of the above problems, the present invention provides a safe distance detection method integrating adaptive multi-scale feature network (AMF) and epipolar geometry constraint (EGC). This method enhances the accuracy and robustness of feature point matching through adaptive multi-scale feature network, optimizes the matching range by using epipolar constraint and introduces lightweight network structure to improve real-time performance, thus meeting the requirements of safe distance detection for 10kV non-stop operation. Summary of the invention
[0006] In order to solve the above technical problems, the present invention provides a 10kV non-stop operation safety distance detection method integrating AMF and EGC, comprising the following steps:
[0007] S1. Using an industrial binocular camera as the data input source, lock the working range of the operation, and use the instance segmentation network to obtain the segmented images and edge contours of the detected personnel and charged objects;
[0008] S2. The segmented image is subjected to epipolar geometry constraints to obtain the area where the left image pixel and its matching point on the right image may appear;
[0009] S3. These matching information are then fed into the AMF network to extract rich and powerful feature representations;
[0010] S4. This network uses a cascade method to effectively integrate the features of the left and right images, construct a joint cost volume, and aggregate these costs through encoding, and then generate an accurate disparity map through disparity regression and calculation;
[0011] S5. Finally, based on the generated disparity map, the operator’s safe proximity distance to electricity is accurately predicted, and the result is displayed in real time on the detection terminal device.
[0012] Preferably, the feature points based on IS and EGC are pre-selected as follows:
[0013] The RGB images of the left and right cameras segmented from the instance are passed to the epipolar geometry constraint module to constrain the area where the left and right image matching points appear. The internal and external parameter matrices, rotation matrices, and translation matrices of the camera obtained when the binocular camera is calibrated in the laboratory through MATLAB are used to calculate the essential matrix through these matrices to obtain the basic matrix; according to the calibration method, the internal and external parameters of this stereo camera are obtained; point p is a point in the real scene, p1 and p2 are a pair of feature matching points of point p on the imaging plane, O1 and O2 are the centers of the left and right binocular cameras, K1 and K2 are the imaging planes of the cameras, l1 and l2 are their corresponding epipolar lines, and the epipolar line solution process is as follows:
[0014] E=TR (1)
[0015]
[0016] From (1) and (2), we can get
[0017]
[0018] F and the coordinates of the pixel in the right or left image can be obtained by the following formula:
[0019]
[0020] By solving (4), we can get:
[0021] (f 11 U L +f 21 V L +f 31 ) R +(f 12 U L +f 22 V L +f 32 )V R +(f 13 U L +f 23 V L +f 33 )=0
[0022] Combined with the equation of the line: aU R +bV R +c=0
[0023] Solve to obtain the corresponding polar lines of the points in the left image in the right image, and obtain the linear geometric constraint areas of the left and right RGB images; after obtaining the corresponding constraint areas, send them to the feature extraction network of the AMF network.
[0024] Preferably, the AMF network includes a CSM attention module, which uses the inter-layer attention to obtain multi-scale fusion features of the input binocular image, hierarchically pays attention to different features and then inputs them into the feature aggregation module;
[0025] Take two feature maps T from one layer of the network d T p ∈R C×H×W , where C, H, and W represent the number of channels, spatial height, and width of the feature map, respectively; where T d is obtained by downsampling the input features at each level, T P It is the result of the residual block of the input network of each layer; there is a convolution kernel pooling operation in the residual block, T P It has a large receptive field; first get the fusion feature: T = T d +T p, where T obtained is a simple fusion between low-level features and higher-level features.
[0026] Preferably, the CSM attention module comprises a joint attention module and a spatial feature fusion module; the joint attention module is composed of local feature attention and global feature attention; in the local feature attention component, point-by-point convolution is used as a mechanism for local channel context aggregation, and each spatial position is used for channel-by-channel interaction; the local feature K(T) is calculated:
[0027] K(T)=EW2 Conv (a(B(EW1 Conv (T))))) (2)
[0028] a refers to the activation function sigmoid, and B(x) is a batch normalization function;
[0029] The channel statistics p are generated by reducing the spatial dimension H×W of each channel in T;
[0030]
[0031] Where F(T) is the global average pooling function, which calculates the global feature P(T):
[0032] P(T)=B(EW2 Conv (a(B(EW1 Conv (p))))) (4)
[0033] In order to make the AMF network as lightweight as possible, it is necessary to fuse features of different scales in the joint attention module to combine local features with global features;
[0034]
[0035] The spatial attention features are generated by exploiting the spatial relationship between features in different layers; pooling operations are performed along the channel dimension and connected to generate a valid feature map; then, the connected feature maps are convolutionally operated to generate a spatial feature map N s ∈R 1×H×W Where N s It is a spatial feature fusion module; two feature maps are generated through two pooling operations, and then they are spliced and convolved along the channel size, and the sigmoid function is used to convert N s (X) is scaled to 0-1 and multiplied by N accordingly s (X) obtains the spatial weight; where the weight W∈R 1×H×W Obtained through calculation;
[0036]
[0037] Finally, the obtained weights W and 1-W are multiplied by the features respectively, so that the output feature map has both overall dependencies and global information, aggregates high-level and low-level spatial features, and obtains multi-scale fusion features.
[0038] Preferably, the AMF network further includes an MGA module and a 3D aggregation module;
[0039] The MGA module is used to improve the distinguishability of features in the multi-scale cost volume generation stage;
[0040] The 3D aggregation module is used to obtain high-quality feature information in the cost aggregation stage to reduce the problem of object blur, occlusion or unclear matching in the input image by generating an attention mask M∈R 1×H×W , perform element-by-element product of all channel dimensions and input features; first, perform average pooling on the input feature T in the channel dimension to obtain the global channel attention tensor B∈R 1×H×W , then B will B'∈R 1×H×W , converted and input into the 3D convolution layer, the output is converted back to the original dimension and the channel attention mask is generated through the Sigmoid function.
[0041] Preferably, the disparity is obtained by using a disparity regression method, the feature map of the binocular image is used as input, and the key point safety distance is output in real time; the specific steps are as follows:
[0042] Disparity calculation: Since there may be some noise in the cost volume, first output the feature map, and then use the softarg max function to predict the probability of each disparity d;
[0043]
[0044] Among them, d max is the candidate range of the maximum disparity of this pixel, softmax(-c d ) is the probability of disparity d; combined with the predicted operating scene disparity map D output by the network, the depth value Z of the left image of the 10kV operating scene is calculated based on the known designed camera parameters,
[0045]
[0046] That is, the distance between the key point pixel and the target point in the three-dimensional world and the optical center of the camera imaging; the actual accurate value of the real distance obtained by the high-precision distance measuring instrument is used to train the model;
[0047] By establishing a spatial coordinate system to obtain the spatial coordinates of the key feature points, the straight line equation of the wire is calculated, and the shortest distance D between these key points and the straight line is calculated to obtain the real-time safety distance;
[0048]
[0049] The proposed AMF network is a trainable network, and in the final disparity prediction step, SmoothL1 is used as the loss function to train the network, and the loss function is defined as:
[0050]
[0051] Where d is the real parallax, To predict disparity, N is the number of pixels in all key locations.
[0052] Detecting whether the distance between the operator and the charged object exceeds the limit is the key to ensuring the safety of 10kV non-stop operation. In view of the problem of insufficient accuracy and real-time performance of traditional stereoscopic vision ranging methods, the present invention provides a safe distance detection method that integrates adaptive multi-scale features (AMF) network and epipolar geometry constraints (EGC). First, the contours of the key parts of the personnel and the charged object in the binocular image are accurately extracted by instance segmentation. Secondly, the search range of the feature matching point is constrained by EGC. Then, AMF is used to highlight the relevant features of the matching points and assign adaptive weights to the multi-scale features to obtain multi-scale fusion features. AMF improves the feature point representation capability and matching accuracy through its joint attention module, and its global-local fusion module provides a more robust feature representation for multi-scale cost aggregation by combining the left and right image features. In addition, the 3D cost aggregation is used to reduce the number of parameters and thus improve the detection speed. Finally, the safe distance evaluation is performed by key point disparity calculation to achieve the distance detection between the operator and the charged object. Experimental results show that the distance measurement accuracy of the proposed method reaches 94.6%, and the frame rate reaches 28.9 frames / s, which meets the accuracy and real-time requirements of safety distance detection in 10kV non-stop operation sites. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 This is the overall framework diagram of the 10kV non-stop operation safety distance stereoscopic vision detection method system integrating AMF and EGC.
[0054] Figure 2 is an epipolar geometry constraint.
[0055] Figure 3 It is a 10kV non-stop operation safety distance stereoscopic vision detection network architecture that integrates AMF and ECR.
[0056] Figure 4 are inter-layer and spatial attention modules.
[0057] Figure 5 It is a 3D cost aggregation module.
[0058] Figure 6 It is a diagram of the equipment of the present invention.
[0059] Figure 7 It is the distance detection performance of the method of the present invention.
[0060] Figure 8 It is a schematic diagram of the job site test. DETAILED DESCRIPTION
[0061] 1. Safety distance evaluation architecture integrating AMF and EGC
[0062] Taking into account the uncertainty of the working area and the complexity of the working environment during the actual 10kV non-stop operation, as well as the instantaneous situation of workers crossing the safety distance, real-time monitoring of the distance between the key parts of the workers and the charged objects poses a severe challenge. The present invention uses an industrial-grade binocular camera as the data input source, locks the working range of the operation, and uses an instance segmentation network to obtain the segmented image and edge contour of the detected personnel and the charged object. The segmented image is subjected to epipolar geometric constraints to obtain the area where the pixels of the left image and their matching points on the right image may appear. These matching information are then input into the AMF network to extract rich and powerful feature representations. This network uses a cascade method to effectively integrate the features of the left and right images, construct a joint cost body, and aggregate these costs through encoding, and then generate an accurate disparity map through disparity regression and calculation. Finally, based on the generated disparity map, an accurate prediction of the worker's safety distance near electricity is obtained, and the result is displayed in real time on the detection terminal device. Overall architecture Figure 1 shown.
[0063] 1.1 Feature point preselection based on IS and EGC
[0064] The RGB images of the left and right cameras segmented from the instance are passed to the epipolar geometry constraint module to constrain the area where the left and right image matching points appear. The camera's internal and external parameter matrices, rotation matrix, and translation matrix obtained when the binocular camera was calibrated in the laboratory using MATLAB are used to calculate the essential matrix (Essential Matrix) and then obtain the fundamental matrix (Fundamental Matrix). Figure 2 As shown in the figure, according to the calibration method, the internal and external parameters of this stereo camera are obtained. Point p is a point in the real scene, p1 and p2 are a pair of feature matching points of point p on the imaging plane, O1 and O2 are the centers of the left and right binocular cameras, K1 and K2 are the imaging planes of the cameras, l1 and l2 are their corresponding epipolar lines, and the epipolar line solution process is shown below.
[0065] E=TR (1)
[0066]
[0067] From (1) and (2), we can get
[0068]
[0069] F and the coordinates of the pixel in the right or left image can be obtained by the following formula:
[0070]
[0071] By solving (4), we can get:
[0072] (f 11 U L +f 21 V L +f 31 ) R +(f 12 U L +f 22 V L +f 32 )V R +(f 13 U L +f 23 V L +f 33 )=0
[0073] Combined with the equation of the line: αU R +bV R +c=0
[0074] Solve the corresponding epipolar lines of the points in the left image in the right image, and obtain the linear geometric constraint areas of the left and right RGB images, reduce the range of finding matching points, and improve the accuracy of matching. Compared with the traditional semi-global stereo matching (SGBM), this geometric constraint method can better adapt to the outdoor large field of view environment in outdoor scenes. Reduce the influence of light on the feature matching area. After obtaining the corresponding constraint area, it is sent to the feature extraction network of the AMF network.
[0075] The epipolar constraint module can constrain the range of feature point matching to improve robustness. It can reduce the number of pixels in key areas during subsequent feature extraction, avoid the problem of too many elements in the entire photo, and improve the accuracy of feature point matching.
[0076] 1.2AMF Network
[0077] In the 10kV non-stop power supply operation site, the scene elements are complex, and the obtained image pixel relationship information and key parts edge texture are not clear. This often makes it difficult to find accurate matching points when matching feature points, which can easily lead to errors in the calculation of parallax of the worker's contour details and the edge area of the human body. To address this type of problem, in order to accurately realize the real-time safe distance detection between workers and live objects, the present invention uses an adaptive multi-scale feature (AMF) network to realize feature extraction, uses multi-threaded data processing, and quickly extracts features from the image data after polar constraints, and obtains feature information after feature fusion, which is then passed into the module for subsequent operations. The overall architecture of the AMF network is as follows: Figure 3 shown.
[0078] CSM Joint Attention Module
[0079] The present invention introduces a joint attention mechanism CSM in the output stage of the feature extraction network, which effectively improves the expressive power of feature extraction without affecting the feature extraction speed of the network. Although deeper networks can extract rich information, the detailed features of the texture extracted by the shallow network will be lost during multiple downsampling processes, which has a great impact on the detection of the safe distance of the human body edge contour during 10kV non-stop operation. Therefore, the inter-layer CSM attention module proposed in the present invention uses this inter-layer attention to obtain the multi-scale fusion features of the incoming binocular image, pays attention to different features in layers, and then passes them into the feature aggregation module.
[0080] Depend on Figure 3 From the AMF network structure diagram in the figure, we can see that two feature maps T are taken from one layer of the network. d T p ∈R C×H×W , which represent the number of channels, spatial height and width of the feature map respectively. d is obtained by downsampling the input features at each level, T P It is the result of the residual block of the input network of each layer. There is a convolution kernel pooling operation in the residual block, T P It has a large receptive field. First, we get the fusion feature: T = T d +T p , where T obtained is a simple fusion between low-level features and higher-level features.
[0081] In the CSM module, there are joint attention modules and spatial feature fusion modules. The joint attention module is composed of local feature attention and global feature attention. Figure 4As shown in the figure, the features extracted by local feature attention can be retained and the underlying fine features of the image contour edge can be highlighted, which is helpful for subsequent feature point matching. In the local feature attention component, point-by-point convolution is used as the mechanism for local channel context aggregation, and each spatial position is used for channel-by-channel interaction. Calculate the local feature K(T):
[0082] K(T)=EW2 Conv (a(B(EW1 Conv (T))))) (2)
[0083] In formula (2), by controlling the channel reduction ratio, more feature details can be aggregated, and the details of the low-level features of the image can be retained and highlighted. a refers to the activation function sigmoid, and B(x) is a batch normalization processing function.
[0084] The global attention component can aggregate spatial information and reduce model parameters, compressing the spatial dimension of the input features. The channel statistics p are generated by reducing the spatial dimension H×W of each channel in T.
[0085]
[0086] Where F(T) is the global average pooling function, which calculates the global feature P(T):
[0087] P(T)=B(EW2 Conv (a(B(EW1 Conv (p))))) (4)
[0088] In order to make it as lightweight as possible, it is necessary to fuse features of different scales in the joint attention module to combine local features with global features.
[0089]
[0090] The spatial attention features are generated by exploiting the spatial relationship between features in different layers. Unlike the joint attention module, the spatial feature fusion module focuses on the key points of key parts as the most information-rich part, which complements the joint attention module. Pooling operations are performed along the channel dimension and connected to generate a valid feature map. Subsequently, the connected feature maps are convolutionally operated to generate a spatial feature map N s ∈R 1×H×W Where N s It is a spatial feature fusion module. Two feature maps are generated through two pooling operations, and then they are concatenated along the channel size and convolved. The sigmoid function is used to convert N s (X) is scaled to 0-1 and multiplied by N accordingly s(X) Obtain spatial weights. The obtained weights emphasize more effective information within the spatial feature information layer and reduce more redundant information. The weight W∈R 1×H×W Obtained through calculation.
[0091]
[0092] Finally, the weights W and 1-W are multiplied by the features respectively, so that the output feature map has both overall dependency and global information, aggregates high-level and low-level spatial features, and obtains multi-scale fusion features. The multi-scale fusion features related to key parts are retained and highlighted, solving the problem of unclear image pixel relationship information and edge texture of key parts.
[0093] 1.3 Multi-scale matching cost volume aggregation module
[0094] The multi-scale matching cost volume contains the matching cost of each pixel under different candidate disparities. In order to improve accuracy, aggregation optimization is used to more accurately reflect the correlation between pixels. However, traditional methods only aggregate the matching cost volumes of each scale, ignoring the complementary relationship between cross-scale features. This leads to many-to-one pixel matching in places where the edge texture of key parts is unclear, resulting in inaccurate disparity estimation results. Therefore, in the multi-scale cost volume generation stage, the present invention introduces a global-local fusion (MGA) module to improve the discernibility of features. The key to the MGA module is to capture the contextual relationship between different features in the cost volume, obtain the similarity between any two features, and act as a weight when other features are reconstructed.
[0095] In the cost aggregation stage, Figure 5 As shown in the figure, a 3D aggregation module is introduced to obtain high-quality feature information and reduce the problem of blurred, occluded or unclear matching of objects in the input image. By generating an attention mask M∈R 1×H×W , perform element-wise product of all channel dimensions and input features. First, average pooling is performed on the input feature T in the channel dimension to obtain the global channel attention tensor B∈R 1×H×W , then B will B'∈R 1×H×W , convert it and input it into the 3D convolution layer, the output result is converted back to the original dimension, and the channel attention mask is generated through the Sigmoid function. Compared with the traditional aggregation method, inputting the result into the 3D convolution layer significantly reduces the amount of calculation and improves the calculation speed of the safe distance measurement. 1.4 Distance detection between operators and charged objects
[0096] The present invention uses the method of parallax regression to obtain the parallax, takes the feature map of the binocular image as input, and outputs the key point safety distance in real time. The specific steps are as follows:
[0097] Disparity calculation: Since there may be some noise in the cost volume, the feature map is output first, and then the soft argmax function is used to predict the probability of each disparity d.
[0098]
[0099] Among them, d max is the candidate range of the maximum disparity of this pixel, softmax(-c d ) is the probability of the disparity being d. Combined with the predicted disparity map D of the operation scene output by the network designed in the present invention, the depth value Z of the left image of the 10kV operation scene is calculated based on the known designed camera parameters.
[0100]
[0101] That is, the distance between the key point pixel and the target point in the three-dimensional world and the optical center of the camera imaging. The actual accurate value of the real distance obtained by a high-precision distance meter is used to train the model.
[0102] By establishing a spatial coordinate system, the spatial coordinates of the key feature points are obtained, the straight line equation of the wire is calculated, and the shortest distance D between these key points and the straight line is calculated to obtain the real-time safety distance.
[0103]
[0104] The proposed AMF network is a trainable network, and in the final disparity prediction step, SmoothL1 is used as the loss function to train the network, and the loss function is defined as:
[0105]
[0106] Where d is the real parallax, To predict disparity, N is the number of pixels in all key locations.
[0107] 2. Experiment
[0108] 2.1 Experimental Environment and Dataset
[0109] The experimental equipment of the present invention is as follows Figure 6As shown, an industrial-grade binocular camera with a focal length of 12mm is used, and images can be obtained by adjusting the focal length at a distance of 10-20m during on-site operation. The proposed AMF key point detection network is pre-trained using the KITTL2015 dataset, and then real binocular stereo images are collected on 10kV non-stop operation to construct a sample library for subsequent network training and experimental testing. The present invention uses 800 images, of which 640 are used for training and 160 are used for testing, using multiple scenes of 10kV non-stop operation, as shown in Table 1. At the same time, the method of the present invention is compared with the proposed stereo matching model, the traditional algorithm, and the algorithm based on deep learning.
[0110] Table 1 Introduction to the dataset used in this invention
[0111]
[0112] In the experiment, the PyTorch framework was used to train the network on two Nvidia 3090Ti GPUs. Adam was used as the optimizer, β1=0.9, β2=0.99 were set, and the image was cropped to H=256, W=512 during training. For the SceneFlow dataset, a total of 50 cycles of training were performed, with an initial learning rate of 0.01, and the learning rate was halved every two cycles. The industrial binocular camera used in the present invention is calibrated in the laboratory by MATLAB, and the internal and external parameter matrices of the camera, the rotation matrix, and the translation matrix are calculated by these matrices to obtain the essential matrix (EssentialMatrix) and then the fundamental matrix (FundamentalMatrix). The specifically labeled matrices are shown in Table 2.
[0113] Table 2 Internal and external calibration parameter values of the experimental binocular camera
[0114]
[0115] 2.2 Experimental introduction and comparative experiments
[0116] The present invention uses 1-pixel, 2-pixel and 3-pixel errors to quantitatively analyze the performance of the model proposed in the present invention. 1-3 pixel error means the percentage of pixels with parallax errors exceeding 1-3 in the total pixels of the left view. Table 2 shows the parallax error rate of SGBM, PSMNET, AnyNet, document 11, and the method of the present invention in 1-3 pixels. The EPE index is used to judge the accuracy of the model in the detection of key parts of 10kV live working. The specific results are shown in Table 3.
[0117] Table 3 Quantitative comparison of AMF network and other methods
[0118]
[0119] (1) The average error of traditional stereo matching (SGBM) in 1-pixel, 2-pixel, and 3-pixel is 36.2%, and the processing speed is 5.12s / picture, which makes it difficult to achieve the accuracy and real-time performance requirements of ranging in the 10kV non-stop operation scenario.
[0120] (2) The PSMNet stereo matching algorithm uses a large number of multi-scale 3D convolutions to optimize the matching cost volume. The average error of 1-pixel, 2-pixel, and 3-pixel is 23.4%, which is much lower than that of SGBM. However, the use of 3D convolution will perform a large number of operations, which greatly increases the network running time. The processing speed of this algorithm is 3.256s / picture. Like the SGBM algorithm, its real-time performance is seriously insufficient and does not meet the requirements.
[0121] (3) Real-time stereo matching AnyNet can meet industrial-grade requirements in terms of processing speed. Its average error in 1-pixel, 2-pixel, and 3-pixel is 18.9%, and the processing speed is 15.26s / sheet. The speed is significantly improved, but the accuracy cannot meet the requirements of 10kV non-stop operation, and it is impossible to accurately judge the distance between the edge of the key part and the charged body.
[0122] (4) Compared with the AMF network of document 11, the stereo matching algorithm proposed in the present invention improves the real-time frame rate by 28.9 frames / s, and the frame rate of the comparison algorithm is 12.89 frames / s. In addition, the average error of the stereo matching model of the present invention in 1-pixel, 2-pixel, and 3-pixel is 8.63%. The average error of document 11 when used to detect key parts of 10kV non-stop operation is 12.7%, which is inconsistent with the required accuracy of less than 10%. The key part distance detection method proposed in the present invention improves the accuracy of key part matching and charged body distance detection, and is better suitable for distance detection in this fixed scene.
[0123] In order to further verify the feasibility and accuracy of the method proposed in the present invention, the present invention conducted a high-precision fixed-point laser ranging experiment with non-stop workers as the object as the real distance and the binocular ranging method proposed in the present invention. The ranging experiment was carried out every 0.5m within 10 to 20m. The real-time distance between the worker and the charged object was obtained by two methods respectively. The relative errors of the two methods at different distances were calculated to measure whether the ranging performance of the method proposed in the present invention meets the requirements. The specific relative error visualization is as follows: Figure 7As shown in the figure, as the distance between the operator and the camera gradually increases, the relative error rate gradually decreases, and the relative error rate is the smallest between 10m-15m. When the distance continues to increase, the relative error rate rebounds, but overall it remains within a relative error of 5%.
[0124] In order to verify the test effect of the method of the present invention in the field, the actual distance between the face, hand, head and the live wire of the conductor was selected at 14.56m and 15.48m respectively in the field measurement, and the distance measured by the method of the present invention is shown in Table 4. It can be seen that the relative error is basically kept within 5% in the case of these two groups. Figure 7 The performance test results are consistent with the
[0125] Table 4 On-site measured data of non-stop power supply operation
[0126]
[0127] 2.3 Ablation Experiment
[0128] The present invention designs the following ablation experiment to verify the effectiveness of the two modules of the proposed binocular stereo matching neural network, cancels the verification set, and takes 200 images from the operation data set for training. At the same time, the models composed of CSM, MGA, and CSM+MGA are used to predict 200 test images and compare the >3pixel of these three parts. The final results show that the proposed AMF>3pixel is 2.9%. The results are shown in Table 5. The proposed AMF network is effectively suitable for distance detection of key parts of 10kV non-stop operation.
[0129] Table 5 Ablation experiment results
[0130]
[0131] From the results of the ablation experiment, we can see that the best model is the AMF network, which can provide better disparity information of key points. It is used for the safety distance measurement of the following modules. Due to the introduction of CSM, it helps to improve the aggregation of high-level and low-level spatial features, obtain the multi-scale fusion features of the image, retain and highlight the multi-scale fusion features related to key parts, and MGA acts as a weight to associate the feature correlation of each channel. Therefore, the ablation experiment can prove the effectiveness of the two designed modules.
[0132] 3. Conclusion
[0133] (1) A 10kV non-stop operation safety distance stereo vision method that integrates AMF and EGC is used to obtain the contours of key parts of personnel and live objects in the image through instance segmentation, and EGC is used to narrow the matching range. The AMF network is used to obtain multi-scale fusion features, and the CSM joint attention module of the network is used to obtain feature dependencies in different dimensions, and the MGA module captures the contextual relationship in the cost volume to improve the matching accuracy. During cost aggregation, 3D cost aggregation is used to reduce operating parameters and improve distance detection speed.
[0134] (2) In the 10kV non-stop operation scenario, the distance detection method proposed in the present invention shows excellent applicability and can cope with complex operating environments while maintaining high detection accuracy and speed, meeting the strict requirements of non-stop operation for safe distance measurement.
[0135] (3) Good detection results were achieved in the field test.
Claims
1. A 10kv non-stop power operation safety distance detection method integrating AMF and EGC, characterized in that: The following steps are involved: S1. Using an industrial binocular camera as the data input source, lock the working range of the operation, and use the instance segmentation network to obtain the segmented images and edge contours of the detected personnel and charged objects; S2. The segmented image is subjected to epipolar geometry constraints to obtain the area where the left image pixel and its matching point on the right image may appear; S3. These matching information are then fed into the AMF network to extract rich and powerful feature representations; S4. This network uses a cascade method to effectively integrate the features of the left and right images, construct a joint cost volume, and aggregate these costs through encoding, and then generate an accurate disparity map through disparity regression and calculation; S5. Finally, based on the generated disparity map, the operator’s safe proximity distance to electricity is accurately predicted, and the result is displayed in real time on the detection terminal device.
2. The 10kV non-stop operation safety distance detection method integrating AMF and EGC according to claim 1 is characterized in that: The feature points based on Is and EGC are pre-selected as follows: The RGB images of the left and right cameras obtained by instance segmentation are passed to the epipolar geometry constraint module to constrain the area where the left and right image matching points appear. The camera's internal and external parameter matrices, rotation matrix, and translation matrix are obtained by calibrating the industrial binocular camera in the laboratory through MATLAB. The essential matrix is calculated through these matrices to obtain the basic matrix. According to the calibration method, the internal and external parameters of the stereo camera are obtained; Point p is a point in the real scene, p1 and p2 are a pair of feature matching points of point p on the imaging plane, O1 and O2 are the centers of the left and right binocular cameras, K1 and K2 are the imaging planes of the cameras, l1 and l2 are their corresponding epipolar lines, and the epipolar line solving process is as follows: E=TR (1) From (1) and (2), we can get The coordinates of the pixel in the right or left image can be obtained by the following formula: By solving (4), we can get: (f 11 U L +f 21 V L +f 31 )U R +(f 12 U L +f 22 V L +f 32 )V R +(f 13 U L +f 23 V L +f 33 )=0 Combined with the equation of the line: aU R +bV R +c=0 Solve to obtain the corresponding polar lines of the points in the left image in the right image, and obtain the linear geometric constraint areas of the left and right RGB images; after obtaining the corresponding constraint areas, send them to the feature extraction network of the AMF network.
3. A 10kV non-stop operation safety distance detection method integrating AMF and EGC according to claim 1 or 2, characterized in that: The AMF network includes a CSM attention module, which uses the inter-layer attention to obtain the multi-scale fusion features of the input binocular image, hierarchically pays attention to different features and then transmits them to the feature aggregation module; Take two feature maps T from one layer of the network d T p ∈R C×H×W , where C, H, and W represent the number of channels, spatial height, and width of the feature map, respectively; where T d is obtained by downsampling the input features at each level, T P It is the result of inputting the network residual block of each layer; There is a convolution kernel pooling operation in the residual block, T P It has a large receptive field; first get the fusion feature: T = T d +T p , where T obtained is a simple fusion between low-level features and higher-level features.
4. The 10kV non-stop operation safety distance detection method integrating AMF and EGC according to claim 3 is characterized in that: The CSM attention module includes a joint attention module and a spatial feature fusion module; the joint attention module consists of local feature attention and global feature attention; in the local feature attention component, point-by-point convolution is used as a mechanism for local channel context aggregation, and each spatial position is used for channel-by-channel interaction; the local feature K(T) is calculated: K(T)=EW2 Conv (a(B(EW1 Conv (T))))) (2) a refers to the activation function sigmoid, and B(x) is a batch normalization function; The channel statistics p are generated by reducing the spatial dimension H×W of each channel in T; Where F(T) is the global average pooling function, which calculates the global feature P(T): P(T)=B(EW2 Conv (a(B(EW1 Conv (p))))) (4) In order to make the AMF network as lightweight as possible, it is necessary to fuse features of different scales in the joint attention module to combine local features with global features; Generate spatial attention features by exploiting the spatial relationship between features in different layers; Pooling operations are performed along the channel dimension and connected to generate a valid feature map; then, the connected feature map is convolutionally operated to generate a spatial feature map N s ∈R 1×H×W Where N s It is the spatial feature fusion module;Two feature maps are generated through two pooling operations, and then they are concatenated along the channel size and convolved. The sigmoid function is used to convert N s (X) is scaled to 0-1 and multiplied by N accordingly s (X) obtains the spatial weight; where the weight W∈R 1×H×W Obtained through calculation; Finally, the obtained weights W and 1-W are multiplied by the features respectively, so that the output feature map has both overall dependencies and global information, aggregates high-level and low-level spatial features, and obtains multi-scale fusion features.
5. The 10kV non-stop operation safety distance detection method integrating AMF and EGC according to claim 1 is characterized in that: The AMF network also includes an MGA module and a 3D aggregation module; The MGA module is used to improve the distinguishability of features in the multi-scale cost volume generation stage; The 3D aggregation module is used to obtain high-quality feature information in the cost aggregation stage to reduce the problem of object blur, occlusion or unclear matching in the input image by generating an attention mask M∈R 1×H×W , perform element-by-element product of all channel dimensions and input features; first, perform average pooling on the input feature T in the channel dimension to obtain the global channel attention tensor B∈R 1×H×W , then B will B'∈R 1×H×W , converted and input into the 3D convolution layer, the output is converted back to the original dimension and the channel attention mask is generated through the Sigmoid function.
6. The 10kV non-stop operation safety distance detection method integrating AMF and EGC according to claim 1 is characterized in that: The parallax is obtained by parallax regression, and the feature map of the binocular image is used as input to output the key point safety distance in real time. The specific steps are as follows: Disparity calculation: Since there may be some noise in the cost volume, the feature map is output first, and then the soft argmax function is used to predict the probability of each disparity d; Among them, d max is the candidate range of the maximum disparity of this pixel, softmax(-c d ) is the probability of disparity d; combined with the predicted operating scene disparity map D output by the network, the depth value Z of the left image of the 10kV operating scene is calculated based on the known designed camera parameters, That is, the distance between the key point pixel corresponding to the target point in the three-dimensional world and the optical center of the camera imaging; Use the actual accurate value of the real distance obtained by the high-precision distance meter to train the model; By establishing a spatial coordinate system to obtain the spatial coordinates of the key feature points, the straight line equation of the wire is calculated, and the shortest distance D between these key points and the straight line is calculated to obtain the real-time safety distance; The proposed AMF network is a trainable network, and in the final disparity prediction step, SmoothL1 is used as the loss function to train the network, and the loss function is defined as: Where d is the real disparity, d is the predicted disparity, and N is the pixel points of all key parts.
