A visual positioning method for a contact network wrist-arm installation manipulator

Abstract

The present invention discloses a visual positioning method for a manipulator for installing a contact network arm, which relates to the technical field of railway contact network maintenance, including capturing images of the installation site through a depth camera and preprocessing; inputting the preprocessed images into an improved YOLOv8 network model for feature extraction and detection; inputting 2D detection images into a visual processing module to obtain 3D positioning data; a manipulator control module controls the manipulator to perform corresponding movements according to the 3D positioning data; the visual processing module continuously pays attention to the target position during the movement of the manipulator, performs error detection, and adjusts the movement of the manipulator in real time; the on-site image and installation situation are fed back to the control terminal, and the operator monitors and manually intervenes in the on-site installation situation through the control terminal. The present invention introduces an environment-adaptive feature extraction module and a precise positioning optimization module, which significantly improves the recognition ability and positioning accuracy in a complex railway environment.

Description

Technical Field

[0001] The invention relates to the technical field of railway overhead contact network maintenance, in particular to the technical field of overhead contact network arm installation assisted by machine vision, and in particular to a visual positioning method for an overhead contact network arm installation manipulator. Background Art

[0002] As a key component in the railway power supply system that supports and fixes the contact network, the contact network arm is crucial to ensure the safe, reliable and stable operation of the train traction power supply. The contact network arm is mainly composed of positioners, pull wires, support tubes and other parts. Through precise installation and adjustment, the geometric parameters of the contact network can be ensured to meet the train operation requirements.

[0003] During long-term use, the contact network arm is subjected to complex environmental factors such as wind load, ice load, temperature change, and dynamic load when the train passes, and its installation accuracy is very likely to deviate. Obvious errors in the installation position or posture of the contact network arm will not only directly affect the quality of train current collection, but may also cause serious accidents such as contact network disconnection, resulting in huge economic losses and safety hazards. Therefore, achieving high-precision installation of the contact network arm is of great significance to ensure the safe and stable operation of the railway power supply system.

[0004] With the continuous advancement of automation technology, researchers have begun to try to apply these advanced technologies to the installation process of the contact network arm. There are still some problems in using computer vision technology to achieve precise positioning and intelligent control: 1. In the actual railway environment, under complex and changeable conditions such as strong light, rain and fog, and dark tunnels, traditional image processing methods are easily disturbed, affecting the detection and preliminary positioning accuracy of the contact network arm, and it is difficult to achieve stable and reliable recognition; 2. Some existing visual positioning methods have high computational complexity and are difficult to meet real-time requirements. They cannot achieve high-efficiency processing in the detection and positioning process of the contact network arm, affecting the overall installation efficiency; 3. Although computer vision technology has made significant progress in recent years, there is still room for improvement in the application of contact network arm installation, especially the precise positioning of the specific position of the manipulator grabbing the contact network arm is still insufficient, affecting the installation accuracy and efficiency. Summary of the invention

[0005] In order to overcome the above problems existing in the prior art, the present invention proposes a visual positioning method for a contact network wrist arm installation manipulator.

[0006] The technical solution adopted by the present invention to solve the technical problem is: a visual positioning method for a contact network wrist arm installation manipulator, comprising the following steps:

[0007] Step 1: Capture images of the installation site through a depth camera and pre-process the images;

[0008] Step 2, input the preprocessed image into the improved YOLOv8 network model for feature extraction and detection;

[0009] Step 3: Input the 2D detection image obtained in step 2 into the visual processing module, and the visual processing module combines the depth information obtained from the depth camera to obtain 3D positioning data;

[0010] Step 4, the 3D positioning data obtained in step 3 is transmitted to the manipulator control module in real time, and the manipulator control module performs path planning to generate motion trajectories and control instructions to control the manipulator to perform corresponding movements;

[0011] Step 5: The visual processing module continuously focuses on the target position during the movement of the robot, performs error detection, and adjusts the movement of the robot in real time;

[0012] Step 6: The depth camera, manipulator control module, and visual processing module feed back the on-site image and installation status to the control terminal, and the operator monitors and manually intervenes in the on-site installation status through the control terminal;

[0013] The improved YOLOv8 network model includes an input layer, a CSP multi-scale feature extraction module, an environment adaptive feature extraction module, a multi-head self-attention mechanism, and a precise positioning optimization module. The environment adaptive feature extraction module is embedded in the CSP multi-scale feature extraction module, and the image is output through the environment adaptive feature extraction module, the multi-head self-attention mechanism, and the precise positioning optimization module in sequence.

[0014] In the above-mentioned visual positioning method for installing a manipulator for a contact network arm, the environment perception unit of the environment adaptive feature extraction module utilizes global average pooling and a fully connected layer to extract global environmental information from the input feature map, and dynamically adjusts the network's feature extraction strategy under different environments; the precise positioning optimization module improves the positioning accuracy of the contact network arm through anchor point optimization, structural perception, and an improved positioning loss function.

[0015] In the above-mentioned visual positioning method for a manipulator installed on a catenary arm, the specific processing process of the environment adaptive feature extraction module is as follows: after the input feature map is processed by global average pooling and a fully connected layer, the input feature map is converted into an environment feature vector; based on the environment feature vector, the original feature map is adaptively modulated, and the specific formula is:

[0016] ;

[0017] ;

[0018] in, and is the weight matrix of the fully connected layer, is the activation function, X is the input feature map, e is the environment feature vector, X' is the modulated feature map, is a learnable scaling factor, Represents a channel-by-channel multiplication operation;

[0019] The feature map modulated by the environmental feature vector is processed by dilated convolution with different dilation rates. The specific formula is:

[0020] ;

[0021] in, The expansion rate is of The dilated convolution operation, , , Represent different expansion rates, It represents the concatenation result of feature maps processed by dilated convolution with three different dilation rates;

[0022] Use 1*1 convolution to fuse multi-scale features and generate the final feature map F.

[0023] In the above-mentioned visual positioning method for a contact network wrist arm installation manipulator, the specific processing process of the precise positioning optimization module is as follows:

[0024] Set up a fixed-size but densely-positioned anchor network:

[0025] ;

[0026] in, is the center coordinate of the anchor point, represents the predefined standard width of the overhead line arm, represents the predefined standard height of the overhead line arm, is the index number of the anchor point, from 1 to n, which is used to uniquely identify each anchor point; n is the total number of anchor points, indicating how many anchor points are set at the image or specific feature map scale; is the anchor point set, that is, the set of all predefined anchor points.

[0027] The structure perception layer is introduced to extract the geometric structure features of the contact network arm through convolution operation to generate the structural feature map S:

[0028] ;

[0029] in, is the structure-aware convolution operation, and F is the final feature map generated by the environment adaptive feature extraction module;

[0030] Introducing structural loss function:

[0031] ;

[0032] ;

[0033] in, yes , and is a balance coefficient, which is used to control the weight distribution between IoU loss and structural consistency loss. and , the best balance can be found between positioning accuracy and geometric structure consistency; is the structural consistency loss, which is used to measure the predicted structural features With real characteristics The difference between

[0034] Introduce a direction perception mechanism to integrate the orientation information of the overhead contact network arm:

[0035] ;

[0036] in, and are the predicted and actual catenary arm orientation angles, respectively.

[0037] In the above-mentioned visual positioning method for a contact network wrist arm installation manipulator, the specific process of the multi-head self-attention mechanism is as follows:

[0038] Linearly project the input features to generate queries ,key Sum :

[0039] ;

[0040] in, is a learnable weight matrix, is the number of channels of the input feature map, is the hidden dimension of the attention mechanism;

[0041] Using the complete attention calculation formula, we calculate the query and key to generate attention weights, which are then applied to the values On the top, we get the weighted feature representation:

[0042] ;

[0043] in, is a scaling factor, mainly used to stabilize the gradient during training;

[0044] The multi-head mechanism performs the above process on multiple subspaces and concatenates the results into the final output:

[0045] ;

[0046] in, is the output projection matrix, j is the number of heads.

[0047] In the above-mentioned visual positioning method for a contact network arm installation manipulator, step 3 specifically includes:

[0048] According to the camera's intrinsic parameters and depth map, the position of the target in 3D space is calculated by the following formula:

[0049] ;

[0050] Where: and is the main point of the camera, and is the focal length, is the depth value, u is the horizontal coordinate, v is the pixel coordinate in the vertical direction, X is the horizontal coordinate in 3D space, and Y is the vertical coordinate in 3D space; through the known 3D-2D point pairs, the PnP algorithm calculates the rotation matrix R and translation vector t of the target to determine the precise position and direction of the contact network arm:

[0051] ;

[0052] in, yes Image point, It corresponds to point, B is the camera intrinsic parameter matrix;

[0053] The estimated 3D position and attitude are temporally smoothed using a Kalman filter.

[0054] In the above-mentioned contact network arm-mounted manipulator visual positioning method, the manipulator control module path planning in step 4 adopts the RRT algorithm, and generates the optimal path from the current position of the manipulator to the target through random sampling and local path optimization. The generated path generates a smooth manipulator motion trajectory through quintic polynomial interpolation.

[0055] In the above-mentioned method for visual positioning of a manipulator installed on a catenary arm, the step 5 is specifically as follows: continuously tracking the target position through a visual processing module, calculating the joint angle correction value based on the error between the detection result and the expected trajectory, and adjusting the manipulator movement in real time:

[0056] ;

[0057] in, is the Jacobian matrix, is the joint angle correction value, is the target position expected by the system, It is the current position of the target detected in real time by the visual system or other sensors. is the position error between the desired position and the current estimated position.

[0058] The beneficial effect of the present invention is that the present invention detects and accurately locates the contact network arm through the improved YOLOv8 network, introduces the environment adaptive feature extraction module and the precise positioning optimization module, and significantly improves the recognition ability and positioning accuracy of the system in a complex railway environment. Compared with the prior art, the present invention avoids the complex preprocessing of illumination and background, and greatly simplifies the image data processing process.

[0059] The environment-adaptive feature extraction module of the present invention can dynamically adjust the feature extraction strategy according to the environmental information collected in real time, so as to effectively adapt to complex lighting conditions such as strong light, shadow, tunnel, etc., enhance the anti-interference ability of the system, and greatly improve the detection accuracy.

[0060] The precise positioning optimization module of the present invention focuses on improving the precise positioning capability of the overhead contact arm by optimizing the anchor point settings and introducing the structural perception layer. Compared with the traditional target detection method, PLOM has customized the optimization for the geometric features and environmental characteristics of the overhead contact arm, which can significantly improve the positioning accuracy under complex backgrounds, reduce false detections and missed detections, and provide more reliable visual information for the installation operation of the robot arm.

[0061] The multi-head self-attention mechanism introduced in the present invention further enhances the global feature processing capability of the system. By dynamically weighting the feature information of different regions and channels, the network can maintain higher stability and accuracy in environments with complex backgrounds and changing lighting, thus ensuring the reliability and consistency of detection.

[0062] The present invention has a higher degree of automation. Through the deep combination of the improved YOLOv8 network and the visual processing module, high-precision detection and positioning of the contact network arm can be achieved, which effectively reduces manual intervention and tedious preprocessing links, reduces the risk of subjective judgment errors, and significantly improves the intelligence level of contact network arm installation, thus promoting the development of railway maintenance operations towards automation and intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a schematic diagram of the process of the present invention;

[0064] Figure 2It is a schematic diagram of the improved YOLOV8 network structure of the present invention;

[0065] Figure 3 is a bar chart comparing mAP of the basic model and the improved model of the present invention under different conditions in the embodiments of the present invention;

[0066] Figure 4 is a histogram comparing the recall of the basic model and the improved model of the present invention under different conditions in the embodiments of the present invention;

[0067] Figure 5 is a histogram comparing the mAP of the basic model and the improved model of the present invention under different occlusion conditions in an embodiment of the present invention;

[0068] Figure 6 is a histogram comparing the IoU of the basic model and the improved model of the present invention under different occlusion conditions in an embodiment of the present invention;

[0069] Figure 7 This is a comparison chart of mAP between the basic model and the improved model of the present invention.

[0070] Figure 8 It is a Recall comparison chart of the basic model in the embodiment of the present invention and the improved model of the present invention. DETAILED DESCRIPTION

[0071] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific real-time methods.

[0072] The present invention discloses a visual positioning method for a contact network wrist arm installation manipulator, which comprises the following main steps:

[0073] Step 1: Capture images of the installation site through a depth camera and pre-process the images.

[0074] Step 2: Input the preprocessed image into the improved YOLOv8 network model for feature extraction and detection.

[0075] The improved YOLOv8 network model includes an input layer, a CSP multi-scale feature extraction module, an environment adaptive feature extraction module, a multi-head self-attention mechanism, and a precise positioning optimization module. The environment adaptive feature extraction module is embedded in the CSP multi-scale feature extraction module, and the image is output through the environment adaptive feature extraction module, the multi-head self-attention mechanism, and the precise positioning optimization module in sequence.

[0076] Step 3: Input the 2D detection image obtained in step 2 into the visual processing module, and the visual processing module obtains 3D positioning data in combination with the depth information obtained from the depth camera.

[0077] Step 4, the 3D positioning data obtained in step 3 is transmitted to the manipulator control module in real time, and the manipulator control module performs path planning to generate motion trajectories and control instructions to control the manipulator to perform corresponding movements.

[0078] Step 5: The visual processing module continuously focuses on the target position during the movement of the robot, performs error detection, and adjusts the movement of the robot in real time.

[0079] Step 6: The depth camera, manipulator control module, and visual processing module feed back the on-site image and installation status to the control terminal. The operator monitors and manually intervenes in the on-site installation status through the control terminal. At the same time, new image data is collected regularly to update the weights and parameter configurations of the YOLOv8 network model to adapt to the changing operating conditions. The system has a built-in anomaly detection mechanism to monitor anomalies in the detection and control process in real time. For new operating environments or changing conditions, the system can perform online fine-tuning to continuously optimize the performance of the model and ensure the stability of detection and positioning.

[0080] In this embodiment, step 1 specifically includes: Image data acquisition: Use the Intel RealSense D455 depth camera to be fixedly installed at about 0.5 meters from the front end of the engineering vehicle mechanical arm. The camera angle and height are adjusted according to the working environment and the installation requirements of the contact network arm to ensure that the camera viewing angle can fully cover all installation positions of the contact network arm. The depth camera acquires images at a sampling frequency of 30 fps, the image resolution is set to 1920×1080 pixels, the acquisition range of the depth image is 0.6-6 meters, and each frame of the image contains both color information and depth information, with a depth accuracy of ±1mm. Ensure that the detailed image data and depth information of the target area can be effectively captured, providing a reliable visual basis for subsequent detection and positioning.

[0081] Data preprocessing includes: (1) Denoising: A bilateral filter is used to denoise the RGB image and depth image. The core advantage of bilateral filtering is that it can retain edge information while reducing noise. The pixel value after filtering is calculated as follows:

[0082] ;

[0083] Where: is the normalization factor, is the range kernel function, is a spatial kernel function. This method is particularly suitable for processing targets with obvious edge features such as overhead wire arms.

[0084] (2) Illumination compensation: Use contrast-limited adaptive histogram equalization (CLAHE) technology to enhance image contrast. The CLAHE algorithm avoids excessive noise enhancement by limiting the slope of the histogram and performs histogram equalization in local areas, effectively improving image quality under complex lighting conditions such as strong light and shadows.

[0085] (3) Depth image completion: Guided filtering is used to fill the holes in the depth image. This method uses the RGB image as a guide to effectively fill the missing areas in the depth image while maintaining the depth edge. The core formula of guided filtering is:

[0086] ;

[0087] Where: is the pixel value of the guidance image (RGB image), is the pixel value of the output image, and It is in the window The linear coefficients calculated in . These preprocessing steps significantly improve the accuracy and robustness of subsequent object detection and localization, especially in complex railway environments. The RGB image is normalized to the range [0,1] and resized to 640×640 to match the network input. Finally, the depth image is registered with the RGB image to ensure the accuracy of the spatial information.

[0088] In this example, the improved YOLOv8 network model structure is as follows Figure 2 As shown in the figure, the preprocessed image is first input into the improved YOLOv8 network, and CSPDarknet53 is used as the backbone network to extract multi-scale features. In order to enhance the adaptability of the network in complex environments, the environment adaptive feature extraction module (EAFEM) is embedded after the 3rd, 4th, and 5th CSP modules. The core of the design of EAFEM is to extract global environmental features through global average pooling (GAP) and fully connected layers, and dynamically modulate the feature map based on these environmental information, thereby improving the performance under complex lighting and background conditions.

[0089] The mathematical expression of EAFEM is:

[0090] ;

[0091] Where: and denote the input and output feature maps respectively, is a multi-layer perceptron, for Activation function.

[0092] The environment adaptive feature extraction module (EAFEM) first improves the network's recognition ability under complex lighting conditions by dynamically adjusting the network's feature extraction strategy in different environments. The environment perception unit in EAFEM uses global average pooling (GAP) and a fully connected layer to extract global environmental information from the input feature map. Assume that the input feature map is , is the number of channels of the input eigenvalues, is the size of the feature map in the vertical direction, that is, the height of the feature map, corresponding to the number of vertical pixels of the image. L is the size of the feature map in the horizontal direction, that is, the width of the feature map, corresponding to the number of horizontal pixels of the image. After global average pooling and full connection layer processing, the feature map is converted into an environmental feature vector . This process is expressed by the following formula:

[0093] ;

[0094] Where: and is the weight matrix of the fully connected layer, is the activation function. The main function of the environment perception unit is to dynamically adjust the network's feature extraction capabilities under different lighting and background conditions by extracting environmental features to adapt to complex railway environments such as strong light, shadows, and tunnels. Based on the environmental information output by the environment perception unit, EAFEM adaptively modulates the original feature map. Through channel-level multiplication operations, the network can dynamically enhance features related to the current environment while suppressing irrelevant noise. The modulated feature map The calculation formula is:

[0095] ;

[0096] Where: is a learnable scaling factor, Represents a channel-by-channel multiplication operation. Through this adaptive modulation, the network can adaptively enhance features related to the current environment, making it more sensitive to changes in complex backgrounds and lighting conditions.

[0097] In order to enhance the network's receptive field in complex backgrounds, EAFEM also introduces multi-scale dilated convolution. Dilated convolution can expand the receptive field without increasing the number of parameters, thereby capturing a wider range of contextual information. Specifically, the modulated feature map It will be processed by dilated convolution with different dilation rates. , , , respectively represent different expansion scales. The formula is as follows:

[0098] ;

[0099] Where: The expansion rate is of The dilated convolution operation, , , Represent different expansion rates, such as , represents normal convolution, Indicates moderate expansion, Indicates a large-scale expansion. The representation is the concatenation of feature maps processed by three dilated convolutions with different dilation rates. By introducing multi-scale dilated convolutions, the network can capture more detailed information, especially in environments with uneven lighting or complex backgrounds, which improves the recognizability of the contact network arm.

[0100] Finally, EAFEM uses Convolution fuses multi-scale features to generate the final feature map , the calculation formula is:

[0101] ;

[0102] Through this process, the network integrates information from different scales, making it possible to stably extract key information of the contact network arm in a complex railway environment.

[0103] After EAFEM extracts features, the system further improves feature quality through the multi-head self-attention mechanism (MHSA). The MHSA mechanism dynamically adjusts the importance of features by calculating the correlation of each position in the feature map, allowing the network to better focus on key areas related to the overhead contact arm.

[0104] Implementation of Multi-Head Self-Attention Mechanism (MHSA): 8 attention heads are set, each head learns different feature information independently, and dynamically adjusts the importance of features by learning the correlation in the feature graph. The attention mechanism enhances the global processing capability of features by integrating spatial attention and channel attention. To better play its role, the attention mechanism is strategically embedded after EAFEM output and before PLOM input, so that the model can better process local and global feature information in complex scenes, thereby improving the detection and positioning accuracy of the overhead line arm.

[0105] The core of the Multi-Head Self-Attention mechanism is to dynamically adjust the importance of features by learning the correlation between input features. The mechanism maps input features to three spaces: query, key, and value, determines the attention weight by calculating the similarity between the query and the key, and then applies these weights to the value to weight the feature output.

[0106] Given input features ,in represents the spatial dimension of the feature map (height times width), M is the number of channels, and the specific calculation process of the multi-head self-attention mechanism is as follows:

[0107] First, the input features are linearly projected to generate the query ,key Sum :

[0108] ;

[0109] Where: is a learnable weight matrix, is the hidden dimension of the attention mechanism. Next, using the complete attention calculation formula, we calculate the query and key to generate attention weights, which are then applied to the values On the top, we get the weighted feature representation:

[0110] ;

[0111] Where: is a scaling factor, which is mainly used to stabilize the gradient during training. Finally, the multi-head mechanism performs the above process on multiple subspaces and concatenates the results into the final output:

[0112]

[0113] Where: is the output projection matrix, j is the number of heads.

[0114] In this embodiment, in order to give full play to the role of the attention mechanism and work in conjunction with the existing improved modules, the attention mechanism is strategically embedded in two key positions of YOLOv8. First, the multi-head self-attention layer is directly embedded after the output of the EAFEM module. This design allows the network to further integrate global context information after processing local features, so that the recognition ability under complex backgrounds is improved. Secondly, the feature map is weighted by multi-head self-attention before the input of the PLOM module. This arrangement optimizes the features entering PLOM, allowing the network to focus more on important feature areas when accurately positioning. The selection of these positions is to make full use of the environmental adaptability of EAFEM and the positioning optimization capability of PLOM, ensuring that the network can obtain more refined feature representations when processing local and global features.

[0115] After the introduction of the multi-head self-attention mechanism, the network has a significant optimization effect in feature processing. First, the attention mechanism can capture the long-distance dependencies in the feature map, making up for the limitations of EAFEM in global information processing, so that the network can better understand the position of the contact network arm in the image and the relationship with the background. Secondly, the attention mechanism further improves the expressiveness of the features by learning the similarity between features and dynamically weighting the features of different positions and channels. The attention mechanism before PLOM enables the network to focus on key positions during precise positioning, effectively improving positioning accuracy.

[0116] Under the action of the multi-head mechanism, the network can learn feature associations from different angles, improving the ability to express features from multiple perspectives. This design performs well in scenes with changing lighting and occlusion. By combining global and local feature information, the network shows stronger environmental adaptability in the recognition and positioning tasks of the overhead contact arm.

[0117] The detection head of YOLOv8 is used to perform target classification and bounding box regression, and output the target category, bounding box coordinates and confidence score. The combination of feature enhancement and target detection makes the system more stable and accurate in complex backgrounds. In order to further improve the positioning accuracy, the system uses the precise positioning optimization module PLOM. PLOM improves the accuracy and consistency of the bounding box through anchor point optimization, structure perception and improved positioning loss function.

[0118] PLOM first optimized the anchor point settings and designed a fixed-size but densely located anchor point grid specifically for the overhead contact arm. The anchor points are in the form of:

[0119] ;

[0120] Where: is the center coordinate of the anchor point, and are predefined standard sizes of overhead line arms. represents the predefined standard width of the overhead line arm, Represents the predefined standard height of the overhead line arm. is the index number of the anchor point, from 1 to n, which is used to uniquely identify each anchor point; n is the total number of anchor points, indicating how many anchor points are set at the image or specific feature map scale. is the anchor set, that is, the set of all predefined anchors. Compared with the traditional multi-scale anchor design, this fixed anchor setting can reduce redundant calculations and focus on more accurate target positioning. In order to better capture the geometric structural features of the contact network arm, PLOM introduces a structure perception layer. This layer extracts the geometric structural features of the contact network arm through convolution operations and generates a structural feature map , and its calculation formula is:

[0121] ;

[0122] is the structure-aware convolution operation, and F is the final feature map generated by the environment-adaptive feature extraction module. This structural feature map captures the specific geometric shape of the overhead line arm, which helps to further improve the positioning accuracy.

[0123] In the design of the positioning loss function, PLOM combines IoU loss and structural consistency loss to ensure the accuracy of bounding box positioning. Its loss function is:

[0124] ;

[0125] Where: yes , and is a balance coefficient, which is used to control the weight distribution between IoU loss and structural consistency loss. and , the best balance can be found between positioning accuracy and geometric consistency. In general, these coefficients can be selected through cross-validation to optimize model performance. is the structural consistency loss. The structural consistency loss is used to measure the predicted structural features. With real characteristics The difference between the two is:

[0126] .

[0127] The introduction of structural loss enables the PLOM module to consider both position information and geometric structure characteristics when locating the overhead contact arm.

[0128] Further improvement of positioning accuracy depends on the introduction of a direction perception mechanism in the non-maximum suppression process. This mechanism integrates the orientation information of the overhead line arm and enhances the accuracy of the detection results. The specific calculation formula is as follows:

[0129] ;

[0130] Where: and are the predicted and actual catenary arm orientation angles, respectively. By introducing orientation information, the network can better filter out false detections, further improving the detection accuracy.

[0131] These two improvements, the Environment Adaptive Feature Extraction Module (EAFEM) and the Precise Positioning Optimization Module (PLOM), work together to significantly improve YOLOv8's ability to identify and locate the catenary arm in complex railway environments. EAFEM enhances the network's feature extraction capabilities in complex environments, while PLOM further improves positioning accuracy by optimizing anchor points and combining structural features and orientation information. These improvements provide reliable visual information support for the manipulator's precise grasping and installation operations, allowing the system to maintain efficient and accurate performance in harsh environments.

[0132] In this embodiment, the synergy between the attention mechanism and the EAFEM and PLOM modules is reflected in the following aspects: the EAFEM module is responsible for extracting local features for environmental adaptation, and the attention mechanism integrates local and global information. The network thus gains the ability to cope with environmental changes and understand context, and enhances its adaptability in complex environments. The attention mechanism also delivers global weight optimization features to the PLOM module, allowing the system to locate the target area more accurately, thereby further improving the overall recognition accuracy.

[0133] In this embodiment, the specific method of 3D pose estimation in step 3 is: using the depth information captured by the depth camera, converting the 2D detection result into position information in 3D space. Based on the camera's intrinsic parameters and depth map, the system calculates the position of the target in 3D space using the following formula:

[0134] ;

[0135] Where: and is the main point of the camera, and is the focal length, is the depth value, u is the horizontal coordinate, v is the pixel coordinate in the vertical direction, X is the horizontal coordinate in 3D space, and Y is the vertical coordinate in 3D space. Next, the system uses the PnP (Perspective-n-Point) algorithm to estimate the 3D posture of the target. Through the known 3D-2D point pairs, the PnP algorithm calculates the rotation matrix R and translation vector t of the target, thereby determining the precise position and direction of the contact network arm. The core optimization problem can be expressed as:

[0136] ;

[0137] Where: yes Image point, It corresponds to point, is the camera internal parameter matrix. To reduce noise and jitter, the system uses a Kalman filter to perform temporal smoothing on the estimated 3D position and attitude to ensure stable pose estimation results. The state update and measurement update equations of the Kalman filter are as follows:

[0138] ;

[0139] ;

[0140] ;

[0141] Where: is the state estimation, is the error covariance matrix, is the Kalman gain, is the state transfer matrix, is the observation matrix, and are the covariance matrices of process noise and observation noise respectively, and k is the current moment in the discrete time series.

[0142] Step 4 is as follows: Based on the estimated 3D position and posture of the contact network arm, the system uses the RRT algorithm for path planning. The algorithm generates the optimal path from the current position of the manipulator to the target through random sampling and local path optimization to ensure the smoothness and feasibility of the path. The generated path generates a smooth manipulator motion trajectory through quintic polynomial interpolation. The trajectory interpolation formula is as follows:

[0143] ;

[0144] coefficient arrive The position, velocity, and acceleration constraints of the starting and ending points are solved. The continuity of the trajectory is guaranteed by the velocity and acceleration boundary conditions in the interpolation polynomial. In order to improve the tracking accuracy of the manipulator, the system compensates the generated trajectory based on the dynamic model. The dynamic equation takes into account factors such as torque, inertia, and gravity:

[0145] ;

[0146] Where: is the joint torque, is the inertia matrix, are the Coriolis force and the centrifugal force, is gravity, is the external force on the end effector, is the joint acceleration, the second-order derivative of the system state; is the joint velocity, the first derivative of the system state.

[0147] Step 5 is as follows: During the robot's task execution, the system continuously tracks the target position through visual servoing. Based on the error between the detection result and the expected trajectory, the system calculates the joint angle correction value and adjusts the robot's motion in real time:

[0148] ;

[0149] Where: is the Jacobian matrix, is the correction value of the joint angle. At the same time, the system dynamically adjusts the control gain through an adaptive control mechanism to ensure that the manipulator can flexibly adjust its strategy in different environments and achieve high-precision visual servo control.

[0150] The data processing flow integrates the improved YOLOv8 network, attention mechanism, accurate 3D pose estimation and robot control technology to achieve high-precision recognition, positioning and operation of the overhead contact arm in a complex railway environment. Each processing step has been carefully designed to ensure the robustness, efficiency and accuracy of the system, providing reliable technical support for automated installation tasks.

[0151] The entire process of converting visual positioning results into robot control instructions is described in detail, including the robot motion planning, control instruction generation, path planning and execution steps. The entire control scheme is designed to ensure that the robot can accurately perform grasping and installation tasks, especially in complex railway environments, and perform high-precision operations on the contact network arm.

[0152] The control starting point of the manipulator is the visual positioning information of the target object (contact wire arm). First, the improved YOLOv8 network and the depth camera work together to generate the 3D pose of the target. Then, based on the position and direction of the target, the control system generates a smooth path from the current position of the manipulator to the target through the path planning module.

[0153] Based on the smooth path, the system uses the path information to guide the movement of the robot. The path planning results are sent to the robot through the control terminal to ensure the smoothness and feasibility of the motion trajectory. This enables the robot to accurately reach the target position and perform grasping or installation tasks.

[0154] Based on the generated path, the system needs to convert the path planning into the joint motion instructions of the manipulator. In order to achieve precise control, the system calculates the angles of each joint through inverse kinematics (IK).

[0155] for The inverse kinematics equation for a manipulator with 100 degrees of freedom is:

[0156] ;

[0157] Where: Represents the angle vector of each joint, is the desired position and posture of the end effector. This equation is used to determine the rotation angle of each joint of the manipulator to ensure that the manipulator can reach the predetermined position and posture. In order to avoid singular solutions and achieve continuity of motion, the system uses an optimization algorithm based on Levenberg-Marquardt to select the optimal solution, thereby maintaining the smooth operation of the manipulator.

[0158] After obtaining the target angles of each joint, the control system executes these angles through the joint servo motors. The servo motors achieve precise positioning through position closed-loop control (PID control):

[0159] ;

[0160] Where: is the torque of the ath joint, , and are the proportional, differential and integral gain coefficients respectively, and are the desired and current joint angles. This control strategy can effectively reduce the position error and ensure that each joint remains stable during trajectory tracking.

[0161] In actual motion, the execution of the manipulator is not only affected by the target angle determined by inverse kinematics, but also needs to overcome gravity, inertia and external disturbances. In order to ensure the accuracy of path execution, the control system uses a dynamic compensation control strategy. Dynamic model compensation is applied in the following ways:

[0162] (1) Dynamically adjust the drive torque based on the current joint state and load conditions, especially for torque compensation under fast movement and large load conditions.

[0163] (2) Combined with path planning, it ensures that the robot can smoothly follow the predetermined trajectory during execution.

[0164] Through this dynamic compensation, the actual movement of the manipulator is closer to the planned trajectory, especially in complex railway operating environments, significantly reducing deviations caused by load changes.

[0165] In order to ensure the accuracy of the manipulator during the grasping and installation process, the system is designed with a visual servo feedback control mechanism. Visual feedback is applied during the specific operation to make fine control adjustments. Visual servo continuously obtains the relative position deviation between the target and the end effector, and dynamically adjusts the joint angle to ensure that the manipulator is accurately aligned with the target position. This feedback control mechanism enables the manipulator to respond quickly to external interference and always maintain high-precision positioning and operation of the target, thereby significantly improving the system's operating accuracy.

[0166] When grabbing the catenary arm, the system uses a control strategy that combines force control and position control. First, the end of the manipulator switches to force control mode after contacting the target, and the force sensor ensures the stability of the grasping force. Then, when the system determines that the manipulator has stably grasped the target, it switches back to position control mode to complete the handling and installation of the catenary arm. This process is performed in two steps: grasping and installation:

[0167] The crawling operation includes:

[0168] 1. Visual alignment: Confirm the accuracy of the grasping position through visual feedback and fine-tune the posture of the end effector according to the position of the target.

[0169] 2. Force-controlled grasping: When the end of the robot touches the target, it switches to force-controlled mode and uses a force sensor to monitor the grasping force to ensure that the end grasping force is within the set range to avoid damage to the target due to excessive force.

[0170] 3. Position fine-tuning: During the grasping process, the system adjusts the posture of the end effector in real time to ensure the firmness and accuracy of the grasping.

[0171] The installation process includes:

[0172] 1. Path tracking: The robot moves the grasped target to the installation location along a preset path.

[0173] 2. Alignment and fixation: When the target reaches the installation point, the robot fine-tunes the position based on visual feedback to ensure precise alignment. Through multiple adjustments, the stability of the target is guaranteed during the installation process.

[0174] The control scheme includes key links such as path planning, inverse kinematics solution, dynamic compensation, and gripping and installation. The manipulator uses visual feedback to complete the recognition, positioning and operation tasks of the wrist and arm in a complex railway environment. The system uses multiple safety mechanisms to maintain precise operation and overcome environmental interference, ultimately achieving reliable automated operation.

[0175] In this embodiment, in order to verify the effectiveness of the improved YOLOv8 network in the catenary arm recognition and positioning tasks, a series of experiments were conducted in the actual railway environment. These experiments are intended to verify the performance of the improved YOLOv8 network (including EAFEM and PLOM) and the multi-head self-attention mechanism, as well as the superiority of the combination of the two over the original YOLOv8.

[0176] A dataset containing a variety of complex lighting and background conditions was constructed, with a total of 1,000 images annotated. In order to adapt to different environmental conditions and complex backgrounds, a step-by-step iterative training strategy was adopted, and a dynamic learning rate scheduler was used for adjustment. The entire training process was carried out for a total of 150 rounds, and the batch size and initial learning rate were reasonably selected. In order to improve training efficiency and accelerate model convergence, a mixed precision training technique was used.

[0177] The experiment was conducted on a high-speed railway line, and test scenes including daytime, nighttime, tunnels, strong light and other complex environments were selected. A professional depth camera was used to collect 1,000 high-definition images of the overhead contact arm at the installation site, covering the overhead contact arm in various typical situations. The data set was randomly divided into a training set (700 images) and a test set (300 images). The experiment was aimed at the verification experiment of the improved YOLOv8 model and the analysis of its results. The experiment is divided into three main parts: verification of the environment adaptive feature extraction module (EAFEM), verification of the precise positioning optimization module (PLOM), and performance evaluation of the overall improved model.

[0178] In order to evaluate the advantages of the proposed target detection method based on the combination of the environment adaptive feature extraction module (EAFEM), the precise positioning optimization module (PLOM) and the multi-head self-attention mechanism (MHSA), this embodiment conducted experimental comparative analysis in three aspects:

[0179] 1) To verify the effectiveness of the environment adaptive feature extraction module, this embodiment designs a comparative experiment to compare the detection effect of the EAFEM module under different environmental conditions with the basic YOLOv8 model, analyzes the adaptability of EAFEM under three typical environmental conditions: darkness, rain and fog, and strong light, and quantitatively analyzes the performance improvement brought by the module in environmental adaptability. The experimental results are as follows: Figure 3-4 shown.

[0180] from Figure 3 and Figure 4 It can be seen from the comparison that the improved model with EAFEM has achieved significant performance improvement under all environmental conditions. Especially in a dark environment, the mAP of the basic model is only 65.3%, while the improved model is increased to 75.9%; in a rainy and foggy environment, the mAP is increased from 70.2% to 79.5%; in a strong light environment, the mAP is increased from 73.5% to 81.7%. The recall rate index also shows a similar improvement trend, indicating that the EAFEM module can effectively improve the detection ability of the model in extreme environments. The performance improvement of the improved model is mainly due to the environmental adaptive feature extraction mechanism of the EAFEM module. This module can optimize for different environmental conditions by adaptively adjusting the feature extraction strategy: enhancing the extraction ability of weak features under low light conditions, improving the recognition of features in rainy and foggy environments, and maintaining the stability of features under strong light interference. This adaptive mechanism enables the model to better cope with various complex environmental conditions.

[0181] 2) To verify the effectiveness of the precise positioning optimization module, this example compares and analyzes the positioning accuracy improvement effect of the PLOM module in complex scenarios. By comparing with the basic YOLOv8 model, the advantages of this module in improving target positioning accuracy are verified. The experimental results are as follows Figure 5-6 As shown, Figure 5 The model's performance in terms of mAP is shown. Under mild occlusion conditions, mAP increases from 76.5% to 82.8%; under moderate occlusion conditions, it increases from 71.2% to 78.9%; and under severe occlusion conditions, it increases from 64.8% to 73.5%.

[0182] from Figure 6It can be seen that after the introduction of PLOM, the positioning accuracy of the model has been significantly improved under various occlusion conditions. Specifically, under mild occlusion conditions, the IoU of the basic model is 0.795, while the improved model is increased to 0.842, an increase of 0.047; under moderate occlusion conditions, the IoU is increased from 0.758 to 0.815, an increase of 0.057; under severe occlusion conditions, the IoU is increased from 0.702 to 0.775, an increase of 0.073. The data shows that the PLOM module can effectively improve the positioning accuracy of the model under different occlusion conditions, and the more severe the occlusion, the more obvious the improvement effect.

[0183] This shows that the PLOM module improves the model's target detection performance, mainly in two aspects: on the one hand, the refined design of the bounding box regression strategy improves the model's positioning ability in complex backgrounds; on the other hand, the attention mechanism effectively reduces background interference and enhances the focus accuracy of the target area. These improvements significantly improve the model's target positioning effect.

[0184] 3) In the complete verification of the improved model, the improved solution integrating the three modules of EAFEM, PLOM and MHSA was comprehensively compared with the basic YOLOv8 model to verify the overall effect of the improved solution. Figure 7-8 shown.

[0185] like Figure 7 and Figure 8 As shown in the experimental results, the improved model showed better convergence characteristics and higher performance ceiling during training. Experimental data show that the mAP of the improved model reached 87.2%, an increase of 9.9 percentage points from 77.3% of the basic model; the Recall index increased from 75.8% to 85.9%, an increase of 10.1 percentage points. These data fully reflect that the improvement scheme has a significant effect.

[0186] The excellent performance of the improved model comes from the synergy of three modules: EAFEM provides stable feature extraction capabilities, PLOM ensures accurate target positioning, and MHSA enhances the model's ability to understand global information. The collaborative design of multiple modules has achieved performance improvements in environmental adaptability, positioning accuracy, and feature expression, and ultimately achieved significant improvements in overall performance.

[0187] The experimental comparison results show that the improved scheme has obvious advantages in all indicators. The environment adaptive feature extraction module effectively improves the model's environmental adaptability, the precise positioning optimization module improves the positioning accuracy of the target, and the multi-head self-attention mechanism enhances the feature expression ability. These technical improvements jointly build a high-performance target detection system, providing reliable technical support for target detection applications in complex dynamic scenes.

[0188] Based on the above method, this embodiment also discloses a contact network arm installation manipulator visual positioning system, in which the data image flow and module interaction are as follows: Figure 1 As shown, the visual positioning system of the present invention mainly includes an improved YOLOv8 network, a visual processing module, a manipulator control module, a manipulator, a control terminal, a depth camera and a computer device.

[0189] The improved YOLOv8 network is one of the core algorithms of this system. It is deeply optimized based on the original YOLOv8 architecture. The network runs on a computer device and is responsible for processing image data from a depth camera. By adding a multi-scale feature fusion module, the network can effectively extract target features in complex environments and achieve rapid and accurate recognition and preliminary positioning of the catenary arm. The improved YOLOv8 network is tightly integrated with the attention mechanism module to jointly improve the recognition performance and positioning accuracy of the system.

[0190] The visual processing module is the bridge between the target recognition algorithm and the manipulator control. This module runs on the computer device and is responsible for further processing the output results of the improved YOLOv8 network and attention mechanism. It performs a series of advanced visual processing tasks, including target tracking, posture estimation, 3D reconstruction, etc. The visual processing module combines the 2D image information with the depth information to generate the precise 3D coordinates and posture information of the catenary arm, providing the necessary data support for the precise control of the manipulator.

[0191] The manipulator control module is responsible for converting the position and posture information output by the visual processing module into motion control instructions for the manipulator. This module contains motion planning algorithms and trajectory optimization algorithms, which can generate the optimal grasping and installation path according to the current environment and task requirements. The manipulator control module is tightly integrated with the manipulator hardware, ensuring execution accuracy through real-time feedback control, while considering safety constraints to avoid possible collisions and interference.

[0192] The control terminal is the human-machine interface of the system, usually an industrial-grade tablet or portable controller. It provides operators with functions such as system status monitoring, parameter adjustment, task planning and emergency control through a graphical user interface (GUI). The control terminal and computer equipment are connected through a wireless network or wired connection to achieve real-time data interaction and control command transmission.

[0193] The depth camera is the front-end data acquisition device of this system and is installed at an appropriate position on the robot arm. It is responsible for capturing high-quality three-dimensional image data of the installation environment, including depth information and RGB information. The depth camera uses binocular stereo vision technology and can stably acquire images in various complex environments (such as strong light, weak light, high temperature, etc.). The captured image data is transmitted to the computer equipment in real time through a high-speed data transmission interface, providing basic data for subsequent image processing and target recognition.

[0194] The computer device is the core data processing unit of the entire system, responsible for running the improved YOLOv8 network and attention mechanism, and is also the computing platform for the visual processing module and the manipulator control module. The computer device is connected to the depth camera, control terminal, and manipulator control module through a high-speed interface, processes data from each part in real time, and executes the corresponding algorithms and control logic.

[0195] In the whole system, each part is closely connected through high-speed data interface and control bus. The image data captured by the depth camera is first transmitted to the computer device, which performs feature extraction and target detection by running the improved YOLOv8 network, and optimizes the detection results through the attention mechanism. The processed data is passed to the visual processing module to generate accurate positioning information. Subsequently, the visual processing module sends the positioning information to the manipulator control module, which performs motion planning and execution according to the instructions, and finally completes the grasping and installation of the contact network arm. The whole process is monitored in real time by the control terminal. The user can view the operating status of the system through the terminal interface and adjust the necessary parameters.

[0196] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. A visual positioning method for a contact network arm installation manipulator, characterized in that: The following steps are involved: Step 1: Capture images of the installation site through a depth camera and pre-process the images; Step 2, input the preprocessed image into the improved YOLOv8 network model for feature extraction and detection; Step 3: Input the 2D detection image obtained in step 2 into the visual processing module, and the visual processing module combines the depth information obtained from the depth camera to obtain 3D positioning data; Step 4, the 3D positioning data obtained in step 3 is transmitted to the manipulator control module in real time, and the manipulator control module performs path planning to generate motion trajectories and control instructions to control the manipulator to perform corresponding movements; Step 5: The visual processing module continuously focuses on the target position during the movement of the robot, performs error detection, and adjusts the movement of the robot in real time; Step 6: The depth camera, manipulator control module, and visual processing module feed back the on-site image and installation status to the control terminal, and the operator monitors and manually intervenes in the on-site installation status through the control terminal; The improved YOLOv8 network model includes an input layer, a CSP multi-scale feature extraction module, an environment adaptive feature extraction module, a multi-head self-attention mechanism, and a precise positioning optimization module. The environment adaptive feature extraction module is embedded in the CSP multi-scale feature extraction module, and the image is sequentially output through the environment adaptive feature extraction module, the multi-head self-attention mechanism, and the precise positioning optimization module; The environment perception unit of the environment adaptive feature extraction module uses global average pooling and fully connected layers to extract global environment information from the input feature map and dynamically adjust the feature extraction strategy of the network in different environments; The precise positioning optimization module improves the positioning accuracy of the overhead contact network arm through anchor point optimization, structure perception, and improved positioning loss function; The specific processing process of the environment adaptive feature extraction module is as follows: after the input feature map is processed by global average pooling and a fully connected layer, the input feature map is converted into an environment feature vector; based on the environment feature vector, the original feature map is adaptively modulated, and the specific formula is: ； ； in, and is the weight matrix of the fully connected layer, is the activation function, X is the input feature map, e is the environment feature vector, X' is the modulated feature map, is a learnable scaling factor, Represents a channel-by-channel multiplication operation; The feature map modulated by the environmental feature vector is processed by dilated convolution with different dilation rates. The specific formula is: ； in, The expansion rate is of The dilated convolution operation, , , Represent different expansion rates, It represents the concatenation result of feature maps processed by dilated convolution with three different dilation rates; Use 1×1 convolution to fuse multi-scale features and generate the final feature map F.

2. A method for visual positioning of a contact network arm installation manipulator according to claim 1, characterized in that: The specific processing process of the precise positioning optimization module is as follows: Set up a fixed-size but densely-positioned anchor network: ； in, is the center coordinate of the anchor point, represents the predefined standard width of the overhead line arm, represents the predefined standard height of the overhead line arm, is the index number of the anchor point, from 1 to n, used to uniquely identify each anchor point; n is the total number of anchor points; is the set of anchor points; The structure perception layer is introduced to extract the geometric structure features of the contact network arm through convolution operation to generate the structural feature map S: ； in, It is a structure-aware convolution operation; Introducing structural loss function: ； ； in, yes , and Is the balance coefficient, which is used to control the weight distribution of IoU loss and structural consistency loss; by adjusting and , find the best balance between positioning accuracy and geometric structure consistency; is the structural consistency loss, which is used to measure the predicted structural features With real characteristics The difference between Introduce a direction perception mechanism to integrate the orientation information of the overhead contact network arm: ； in, and are the predicted and actual catenary arm orientation angles, respectively.

3. A method for visual positioning of a contact network arm installation manipulator according to claim 1, characterized in that: The specific process of the multi-head self-attention mechanism is as follows: Perform linear projection on the input features to generate queries ,key Sum : ； in, is a learnable weight matrix, is the number of channels of the input feature map, is the hidden dimension of the attention mechanism; Using the complete attention calculation formula, we calculate the query and key to generate attention weights, which are then applied to the values On the top, we get the weighted feature representation: ； in, is the scaling factor; The multi-head mechanism performs the above process on multiple subspaces and concatenates the results into the final output: ； in, is the output projection matrix, j is the number of heads.

4. A method for visual positioning of a contact network arm installation manipulator according to claim 1, characterized in that: The step 3 specifically includes: According to the camera's intrinsic parameters and depth map, the position of the target in 3D space is calculated by the following formula: ； Where: and is the main point of the camera, and is the focal length, is the depth value, u is the horizontal pixel coordinate, v is the vertical pixel coordinate, X is the horizontal coordinate in 3D space, and Y is the vertical coordinate in 3D space; Through the known 3D-2D point pairs, the PnP algorithm calculates the rotation matrix R and translation vector t of the target to determine the precise position and orientation of the catenary arm: ； in, yes Image point, It corresponds to point, B is the camera intrinsic parameter matrix; The estimated 3D position and attitude are temporally smoothed using a Kalman filter.

5. The method for visual positioning of a contact network arm installation manipulator according to claim 1, characterized in that: In step 4, the robot control module performs path planning using the RRT algorithm, and generates an optimal path from the current position of the robot to the target through random sampling and local path optimization. The generated path generates a smooth robot motion trajectory through quintic polynomial interpolation.

6. A method for visual positioning of a contact network arm installation manipulator according to claim 1, characterized in that: The step 5 is specifically as follows: continuously tracking the target position through the visual processing module, calculating the joint angle correction value based on the error between the detection result and the expected trajectory, and adjusting the manipulator movement in real time: ； in, is the Jacobian matrix, is the joint angle correction value, is the expected target position, is the current position of the target detected in real time, is the position error between the expected target position and the target's current position.

Images