A camera-equipped welding robot and trajectory planning method

Abstract

The application discloses a welding robot with a location camera and a trajectory planning method, and belongs to the technical field of welding robots. The robot contains a location camera, an overhead camera and other components, and the location camera is at the end of the mechanical arm. The trajectory planning method covers laser SLAM and visual SLAM navigation positioning, improved A and DWA path planning, and a convolutional neural network target detection module. The V-LOAM algorithm of the laser SLAM is combined with a range measurement and mapping algorithm, and the VINS-Fusion algorithm of the visual SLAM fuses multiple information. The improved A algorithm introduces an adaptive weight, and the DWA algorithm selects the optimal speed through a dynamic window. The CNN is used in three steps for fault detection, a 25-layer AlexNet architecture is adopted, real-time verification and identification are realized on a MATLAB platform combined with a RaspberryPi4ModelB, information is transmitted to a GUI for monitoring power equipment. The application can accurately position, reasonably plan a path, effectively detect faults, adapt to complex environments and improve the performance of the welding robot.

Description

Technical Field

[0001] This invention belongs to the field of welding robot technology, specifically a welding robot with a positioning camera and a trajectory planning method. Background Technology

[0002] Welding technology is widely used in numerous fields of modern manufacturing. The emergence of welding robots has improved welding efficiency and quality, but traditional welding robots still face many problems when dealing with complex and changing working environments. For example, in terms of positioning, due to the complexity of the welding environment and potential deviations in workpiece placement, traditional robots struggle to quickly and accurately locate welding points, affecting welding precision and efficiency. Regarding path planning, existing algorithms may not adapt to complex indoor and outdoor environments, leading to unreasonable robot movement paths, increased energy consumption and working time, and even potential collisions with obstacles. Furthermore, traditional methods are often insufficiently timely and accurate for fault detection and identification during the welding process, making it difficult to effectively monitor welding objects such as electrical equipment. With the continuous improvement of industrial automation, the performance requirements for welding robots are also increasing. Therefore, developing a welding robot capable of more precise positioning, more rational path planning, and effective fault detection has become an urgent problem to be solved. Summary of the Invention

[0003] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0004] In view of the problems existing in the above and / or prior art, the present invention is proposed.

[0005] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a welding robot with a positioning camera and a trajectory planning method.

[0006] As a preferred embodiment of the present invention, the present invention provides a welding robot with a positioning camera, which includes: a positioning camera, a top-view camera, an overall camera, a positioning camera fixing plate, a positioning camera head, and a welding head.

[0007] The positioning camera is located at the end of the robotic arm of the welding robot, and it can follow the robotic arm to complete omnidirectional movement.

[0008] The top-view camera is located on the forearm of the welding robot arm, and it takes top-view photos, scans, and videos of the object being welded.

[0009] The overall camera is located on the support of the welding robot, and it takes pictures, scans and videos of the object being welded.

[0010] The positioning camera includes a positioning camera mounting plate, a positioning camera head, and a welding head. The positioning camera head is fixed to the end of the welding robot's robotic arm via the positioning camera mounting plate, and it moves in all directions following the welding head.

[0011] This invention also provides a trajectory planning method for a welding robot with a positioning camera, which includes a navigation and positioning module for laser SLAM and visual SLAM, a path planning algorithm module for improved A* and DWA, and a target detection method module for convolutional neural networks.

[0012] Preferably, the navigation and positioning module of the laser SLAM and visual SLAM uses the VLOAM algorithm to implement radar SLAM, utilizing range measurement and mapping algorithms in cooperation. High-frequency, low-fidelity odometer measurement and low-frequency, high-fidelity scan matching generate high-frequency and high-precision individual motion estimates. Odometer measurement primarily uses point cloud data acquired by the laser radar to calculate the robot's motion information. The laser radar collects point cloud data of the surrounding environment at a certain frequency (10 times per second). The point cloud data collected by the laser radar at two adjacent times t1 and t2 are respectively... Point cloud A point cloud containing N1 points. It contains N2 points.

[0013] (1) Key feature points are extracted from the point cloud using feature extraction algorithms. Commonly used feature extraction algorithms are based on the geometric properties of points, such as curvature. For point clouds... Points in Calculate its curvature Select curvature greater than a certain threshold k th The points are taken as feature points. Let the extracted feature point set be... Similarly from Extract feature point set

[0014]

[0015] (2) Find the correspondence between feature points at two time points using a matching algorithm. For feature points Calculate its FPFH descriptor Similarly, for [the FPFH descriptor], calculate its [the FPFH descriptor]. Matching point pairs are found by comparing the distances between descriptors. Let the matched point pairs be... in

[0016] (3) Based on the corresponding point pairs, calculate the relative motion transformation matrix from t1 to t2 using methods such as the least squares method. Let corresponding point pairs If the quantity is M, then the following equation can be solved using the least squares method:

[0017]

[0018] The formula for mileage measurement can be expressed as follows: Let the lidar coordinate system be L, and the robot body coordinate system be B. The coordinate transformation relationship for point p in different coordinate systems is as follows:

[0019] In odometer measurement, the relative motion transformation matrix is ​​obtained through a series of feature extraction and matching calculations. So that for time t2, the points in the point cloud Its corresponding position at time t is estimated as follows:

[0020] Preferably, the navigation and localization module of the laser SLAM and visual SLAM is characterized by: scanning matching further optimizing map construction based on odometry. Let the constructed map point cloud be M, and the robot pose estimate obtained through odometry at the current moment be T. r The point cloud acquired by the lidar at the current moment is P. c .

[0021] (1) Move the current point cloud P c Transform to a map coordinate system, i.e.

[0022] (2) Finding through scanning matching algorithm The optimal match between the point cloud M and the map point cloud M. For the ICP algorithm, let m be the number of points in the map point cloud M. i (i = 1, 2, ..., N) m Current point cloud Point p in the middle j (j = 1, 2, ..., N) p By calculating the distances between points, each p is found. j The nearest neighbor m in M i Construct the corresponding relationship {(p j ,m i )}.

[0023] (3) Let the optimized robot pose obtained by scanning matching be T. o The objective of the scanning and matching is to minimize the following objective function: in It refers to points in a map point cloud. This represents the points in the current point cloud transformed to the map coordinate system, and is the number of points participating in the matching. For the ICP algorithm, N = N p .

[0024] Preferably, the navigation and positioning module of the laser SLAM and visual SLAM uses the VINSFusion algorithm to implement visual SLAM. Front-end visual information processing extracts visual information and performs tracking, while back-end information fusion and optimization processes the feature point tracking information obtained from the front end. The front-end visual information processing includes feature extraction and feature tracking, while the back-end information fusion and optimization includes fusing IMU and GPS information and using optimization algorithms.

[0025] Preferably, in the improved A* algorithm, a weight δ is introduced, making the heuristic function f(n) = δg(n) + (1-ω)h(n). Here, g(n) is the actual cost from the starting node to the current node n, and h(n) is the estimated cost from the current node n to the target node.

[0026] The weight δ is adaptively adjusted based on the robot's position. Let the robot's current position be (x, y) and the target position be (x, y). g ,y g Define the distance function. When d is large, to improve search efficiency, the weight of g(n) is increased, i.e., δ is increased; when d is small, to improve path quality, the weight of h(n) is increased, i.e., δ is decreased. The specific adjustment method can be achieved through an exponential function.

[0027] After completing the above, a path search is performed. The path search process is as follows:

[0028] The algorithm starts the search from the initial node and adds it to an open list called OpenList, which stores nodes to be expanded. Simultaneously, a closed list called ClosedList is maintained to store nodes that have already been expanded.

[0029] In each iteration, the node n with the smallest f(n) value is selected from the OpenList as the current node for expansion. For the neighboring nodes of the current node, g(m) = g(n) + c(n,m) is calculated, where c(n,m) is the movement cost from node n to node m. Simultaneously, h(m) is calculated, and then f(m) is calculated according to the adaptive dynamic weighting strategy.

[0030] If node m is not in OpenList, add it to OpenList; if node m is already in OpenList and the newly calculated f(m) is less than the original value, update the relevant information of node m.

[0031] The search ends when the target node is added to the ClosedList. The search then backtracks from the target node to obtain the path from the starting node to the target node.

[0032] Preferably, in the DWA algorithm, a dynamic window [v] is defined in the robot's velocity space. min ,v max ]×[ω min ,ω max ], where v is the robot's linear velocity and ω is the robot's angular velocity. This dynamic window limits the range of possible velocities of the robot at the current moment.

[0033] Within the dynamic window, the robot's trajectory is calculated for each possible velocity based on its current position and environmental information. Let the robot's current position be (x, y, θ) (where θ is the robot's orientation angle). For each velocity, the position after a certain time can be calculated using kinematic equations.

[0034] For each velocity within the dynamic window, an evaluation function G(v,ω) is calculated. This evaluation function comprehensively considers multiple factors, such as path safety (whether it will collide with obstacles), path smoothness, and path proximity to the target point. Let the distance from the path to the target point be d. t The curvature of the path is κ, and the distance to the obstacle is d. o The evaluation function can then be defined as G()G(v,ω)=αd t +βκ+γd o , where α, β and γ are weighting coefficients used to adjust the importance of different factors.

[0035] Preferably, the fault identification and classification process is carried out in three stages, including real-time capture of images of power equipment, fault identification using CNN, and conveying fault information to the operator through images and fault location; the CNN network uses the 25-layer CNN architecture AlexNet, the fault classification program is implemented in the MATLAB platform, the weighting matrix in the MATLAB program is run on a Raspberry Pi 4 Model B with 4GB RAM to obtain real-time verification and fault identification, and images captured by the camera in the robot system and USB camera are provided to the trained CNN network for comparison to identify faults and convey the information to the graphical user interface.

[0036] This invention discloses a welding robot with a positioning camera and a trajectory planning method, which has the following beneficial effects:

[0037] 1. By integrating information from multiple sensors, including LiDAR, visual cameras, IMU, and GPS, this project constructed a multimodal SLAM system. This system design fully utilizes the advantages of each sensor, achieving more comprehensive and accurate environmental perception. Based on multi-sensor fusion, the robot can more adaptably cope with diverse indoor and outdoor scenarios.

[0038] 2. A V-LOAM algorithm is used to implement radar SLAM, while a VINS-Fusion algorithm is used to implement visual SLAM. This cooperative operation effectively integrates laser and visual information, achieving high-frequency and high-precision single motion estimation, enhancing the system's reliability in complex dynamic environments, and providing more reliable position estimation for navigation tasks.

[0039] 3. In navigation tasks, the A* algorithm is used for global path planning to achieve autonomous navigation in fixed environments. In complex dynamic environments, the DWA algorithm is used to handle local path planning tasks, taking into account obstacles under unknown environmental information, thus improving the stability and flexibility of the navigation system in dynamic environments. This intelligent path planning and decision-making mechanism enables the robot to cope with various navigation challenges more efficiently. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0041] Figure 1 This is a structural diagram of the autonomous navigation system for complex dynamic environments based on multimodal SLAM in this invention;

[0042] Figure 2 This is a diagram of the DFACascadeR-CNN network structure in this invention;

[0043] Figure 3 This is a flowchart illustrating the path planning and fault identification process of the welding robot in this invention.

[0044] Figure 4 This is a side view of the welding robot in this invention;

[0045] Figure 5 This is a partial view of the positioning camera of the welding robot in this invention;

[0046] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0047] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the examples in the specification.

[0049] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0051] As a preferred embodiment of the present invention, the present invention provides a welding robot with a positioning camera, which includes: a positioning camera 1, a top-view camera 2, an overall camera 3, a positioning camera fixing plate 101, a positioning camera head 102, and a welding head 103.

[0052] The positioning camera 1 is located at the end of the robotic arm of the welding robot, and it can follow the robotic arm to complete omnidirectional movement.

[0053] The top-view camera 2 is located on the forearm of the welding robot arm, and it takes top-view photos, scans and videos of the object being welded.

[0054] The overall camera 3 is located on the support of the welding robot, and it takes pictures, scans and videos of the object being welded.

[0055] The positioning camera 1 includes a positioning camera fixing plate 101, a positioning camera head 102, and a welding head 103. The positioning camera head 102 is fixed to the end of the robotic arm of the welding robot through the positioning camera fixing plate 101, and it follows the welding head 103 to complete omnidirectional movement.

[0056] The present invention also provides a trajectory planning method for a welding robot with a positioning camera, which includes: a navigation and positioning module for laser SLAM and visual SLAM, a path planning algorithm module for improved A* and DWA, and a target detection method module for convolutional neural networks.

[0057] As a preferred embodiment of the present invention, the navigation and positioning module of laser SLAM and visual SLAM integrates two SLAM schemes. Based on different environmental characteristics, it selects the optimal scheme through system fault detection to cope with complex and ever-changing environments, thereby improving the accuracy of outdoor positioning. After mapping is completed, global path planning and local path planning are performed, demonstrating ideal overall performance in complex and dynamic environments with multiple indoor and outdoor applications.

[0058] This invention employs the V-LOAM algorithm to implement radar SLAM, utilizing a collaborative approach between range measurement and mapping algorithms to achieve high-frequency, low-fidelity mileage measurement and low-frequency, high-fidelity scan matching, thereby generating high-frequency and high-precision individual motion estimation. Simultaneously, it employs the VINS-Fusion algorithm to implement visual SLAM, with the front end extracting visual information and performing tracking, and the back end processing the feature point tracking information obtained from the front end, fusing external information such as IMU and GPS, and using optimization algorithms to derive the current state in real time, exhibiting strong robustness in complex dynamic environments.

[0059] As a preferred embodiment of the present invention, the V-LOAM algorithm achieves radar SLAM through the collaborative operation of a range measurement algorithm and a mapping algorithm. Its core lies in generating high-frequency and high-precision single motion estimates through high-frequency, low-fidelity range measurement and low-frequency, high-fidelity scan matching. It includes range measurement and scan matching.

[0060] In a preferred embodiment of the present invention, the mileage measurement is primarily performed by calculating the robot's motion information using point cloud data acquired by a lidar. The lidar collects point cloud data of the surrounding environment at a certain frequency (10 times per second). The point cloud data collected by the lidar at two adjacent times t1 and t2 are respectively... Point cloud A point cloud containing N1 points. It contains N2 points.

[0061] (1) First, key feature points are extracted from the point cloud using a feature extraction algorithm. Commonly used feature extraction algorithms are based on the geometric properties of points, such as curvature. For point clouds... Points in Calculate its curvature Select curvature greater than a certain threshold k th The points are taken as feature points. Let the extracted feature point set be... Similarly from Extract feature point set

[0062]

[0063] (2) Then, the correspondence between feature points at two time points is found using a matching algorithm. A commonly used matching method is descriptor-based matching, using the FPFH descriptor. For feature points... Calculate its FPFH descriptor Similarly, for [the FPFH descriptor], calculate its [the FPFH descriptor]. Matching point pairs are found by comparing the distances between descriptors. Let the matched point pairs be... in

[0064] (3) Based on the corresponding point pairs, the relative motion transformation matrix from t1 to t2 can be calculated using methods such as the least squares method. Let corresponding point pairs If the quantity is M, then the following equation can be solved using the least squares method:

[0065]

[0066] Wherein, this transformation matrix It contains translation and rotation information, which can represent the robot's movement between these two moments.

[0067] The formula for mileage measurement can be expressed as follows: Let the lidar coordinate system be L, and the robot body coordinate system be B. The coordinate transformation relationship for point p in different coordinate systems is as follows:

[0068] In odometer measurement, the relative motion transformation matrix is ​​obtained through a series of feature extraction and matching calculations. So that for time t2, the points in the point cloud Its corresponding position at time t is estimated as follows:

[0069] In a preferred embodiment of the present invention, the scanning matching is an optimization of map construction based on odometry. It is performed at a lower frequency, once per second, to ensure higher accuracy. Let the constructed map point cloud be M, and the robot pose estimate obtained from odometry at the current moment be T. r The point cloud acquired by the lidar at the current moment is P. c .

[0070] (1) First, the current point cloud P c Transform to a map coordinate system, i.e.

[0071] (2) Then, find the matching algorithm. The optimal match between the point cloud M and the map point cloud M. For the ICP algorithm, let m be the number of points in the map point cloud M. i (i = 1, 2, ..., N)m Current point cloud Point p in the middle j (j = 1, 2, ..., N) p By calculating the distances between points, each p is found. j The nearest neighbor m in M i Construct the corresponding relationship {(p j ,m i )}.

[0072] (3) Let the optimized robot pose obtained by scanning matching be T. o The objective of the scanning and matching is to minimize the following objective function: in It refers to points in a map point cloud. This represents the points in the current point cloud transformed to the map coordinate system, and is the number of points participating in the matching. For the ICP algorithm, N = N p .

[0073] By optimizing this objective function, a more accurate robot pose estimate can be obtained, thereby updating the map.

[0074] As a preferred embodiment of the present invention, the VINS-Fusion algorithm extracts and tracks visual information at the front end, processes the feature point tracking information obtained from the front end at the back end, fuses external information such as IMU and GPS, and uses optimization algorithms to determine the current state in real time, exhibiting strong robustness in complex dynamic environments. It includes: front-end visual information processing and back-end information fusion and optimization.

[0075] As a preferred embodiment of the present invention, the front-end visual information processing includes feature extraction and feature tracking.

[0076] In a preferred embodiment of the present invention, the visual information in the feature extraction mainly comes from image data captured by a camera. The camera captures images at a certain frame rate (e.g., 30 frames per second). For each frame, feature extraction is performed first. The present invention uses the ORB algorithm as an example, which detects key points with significant features in the image and calculates a descriptor for each key point.

[0077] Let the i-th frame be and its resolution be m×n. For image I... k For each pixel (x, y) in the image, its keypoint status is determined using the ORB algorithm. This calculation involves comparing the grayscale values ​​of the surrounding area of ​​the pixel. After feature extraction, the keypoint is determined from image I. k The extracted feature point set is F k ={f k,i}, where i = 1, 2, ..., N k Nk N represents the number of feature points extracted from the k-th frame of the image. k The size depends on the content of the image and the parameter settings of the feature extraction algorithm.

[0078] As a preferred embodiment of the present invention, in the feature tracking, for two adjacent frames I... k-1 and I k For feature point I in the previous frame image k-1 In the current frame image I k The tracking is performed in this way. This embodiment uses a descriptor-based matching method for tracking.

[0079] For feature point f k-1,i ∈F k-1 The similarity between its descriptor and the feature point descriptors in the current frame image is calculated. Methods for calculating similarity include Hamming distance. Let f... k-1,i The descriptor is D k-1,i For the current frame image I k Feature point f in k,i ∈F k Its descriptor is D k,j Then the similarity S ij The Hamming distance H(D) can be calculated. k-1,i D k,j The similarity is measured by the similarity of all feature points; the higher the similarity, the smaller the Hamming distance. The best-matching feature point f is found by comparing the similarity of all feature points. k,i ∈F k Thus, the correspondence between feature points {(f k-1,i ,f k,j In this way, motion information of feature points between two adjacent frames can be obtained. Meanwhile, to improve the accuracy and stability of tracking, other strategies can be employed, such as setting a similarity threshold to retain only correspondences with similarity higher than the threshold; or using a multi-scale tracking method to track feature points at different scales.

[0080] As a preferred embodiment of the present invention, the backend information fusion and optimization includes two parts: fusion of IMU and GPS information and optimization algorithm.

[0081] In a preferred embodiment of the present invention, the IMU in the fused IMU and GPS information can provide the robot's acceleration and angular velocity information. The IMU measures acceleration and angular velocity at a relatively high frequency. Let the acceleration measured by the IMU at time t be a. t =(a t,x ,a t,y ,a t,z ), angular velocity is ω t =(ωt,x ,ω t,y ,ω t,z GPS can provide the robot's location information. GPS provides location information at a relatively low frequency. Let the location provided by GPS at time p be... GPS,t =(p GPS,t,x ,p GPS,t,y ,p GPS,t,z ).

[0082] In the backend, the information from the IMU and GPS is fused with the feature point tracking information obtained from the frontend. First, based on the acceleration and angular velocity information from the IMU, the robot's pose change can be predicted using methods such as integration. For linear acceleration a... t,x The change in velocity Δv can be obtained by integration. x Integrating the velocity yields the position change Δx. Simultaneously, considering the angular velocity ω... t,x The change in orientation can be obtained by calculating the rotation matrix. For nonlinear cases, a more complex kinematic model is required for calculation. Then, the prediction result is corrected using GPS position information. Let the fused robot pose estimate be T. r,t During the fusion process, it is necessary to consider the weight allocation of information from different sensors. For example, when the GPS signal is good, GPS information can be given higher weight; when the GPS signal is poor, the IMU and front-end visual information should be relied upon more.

[0083] As a preferred embodiment of the present invention, in order to obtain more accurate robot pose estimation, an optimization algorithm is used to process the fused information. Let z be the observed value of the feature points obtained through feature point tracking in different frame images. k,i According to the robot pose estimation T r,t The calculated principal component analysis (PCA) yields the predicted feature point values. The optimization algorithm aims to minimize the following objective function:

[0084] To solve this objective function, an iterative method is required. In each iteration, based on the current robot pose estimation T... r,t Calculate the predicted value of the feature point Then, the objective function value E(T) is calculated. Based on the gradient information of the objective function, the robot pose estimation T is adjusted. r,t This causes the value of the objective function to gradually decrease. After multiple iterations, when the value of the objective function converges to a certain extent, the optimized robot pose T is obtained. o This allows us to obtain the current status in real time.

[0085] As a preferred embodiment of the present invention, the improved A* and DWA path planning algorithm module specifically comprises: an improvement of the A* algorithm and path planning for DWA.

[0086] As a preferred embodiment of the present invention, the improved A* algorithm introduces a weight δ, making the heuristic function f(n) = δg(n) + (1-ω)h(n). Here, g(n) is the actual cost from the starting node to the current node n, and h(n) is the estimated cost from the current node n to the target node.

[0087] The weight δ is adaptively adjusted based on the robot's position. Let the robot's current position be (x, y) and the target position be (x, y). g ,y g Define the distance function. When d is large, to improve search efficiency, the weight of g(n) is increased, i.e., δ is increased; when d is small, to improve path quality, the weight of h(n) is increased, i.e., δ is decreased. The specific adjustment method can be achieved through an exponential function.

[0088] As a preferred embodiment of the present invention, after completing the above, a path search is performed, and the path search process is as follows:

[0089] The algorithm starts the search from the initial node and adds it to an open list called OpenList, which stores nodes to be expanded. Simultaneously, a closed list called ClosedList is maintained to store nodes that have already been expanded.

[0090] In each iteration, the node n with the smallest f(n) value is selected from the OpenList as the current node for expansion. For the neighboring nodes of the current node, g(m) = g(n) + c(n,m) is calculated, where c(n,m) is the movement cost from node n to node m. Simultaneously, h(m) is calculated, and then f(m) is calculated according to the adaptive dynamic weighting strategy.

[0091] If node m is not in OpenList, add it to OpenList; if node m is already in OpenList and the newly calculated f(m) is less than the original value, update the relevant information of node m.

[0092] The search ends when the target node is added to the ClosedList. The search then backtracks from the target node to obtain the path from the starting node to the target node.

[0093] As a preferred embodiment of the present invention, in the DWA algorithm, a dynamic window [v] is defined in the robot's velocity space. min ,v max ]×[ωmin ,ω max ], where v is the robot's linear velocity and ω is the robot's angular velocity. This dynamic window limits the range of possible velocities of the robot at the current moment.

[0094] Within the dynamic window, the robot's trajectory is calculated for each possible velocity based on its current position and environmental information. Let the robot's current position be (x, y, θ) (where θ is the robot's orientation angle). For each velocity, the position after a certain time can be calculated using kinematic equations.

[0095] For each velocity within the dynamic window, an evaluation function G(v,ω) is calculated. This evaluation function comprehensively considers multiple factors, such as path safety (whether it will collide with obstacles), path smoothness, and path proximity to the target point. Let the distance from the path to the target point be d. t The curvature of the path is κ, and the distance to the obstacle is d. o The evaluation function can then be defined as G()G(v,ω)=αd t +βκ+γd o , where α, β and γ are weighting coefficients used to adjust the importance of different factors.

[0096] The robot's current speed is selected based on the speed with the highest evaluation function value. The robot is then moved according to this speed, and the dynamic window and evaluation function are recalculated at the new position. The optimal speed is then selected again, thereby achieving local path planning.

[0097] As a preferred embodiment of the present invention, in the object detection method module of the convolutional neural network

[0098] The fault identification and classification process of this method is carried out in three stages: (1) real-time capture of images of power equipment; (2) fault identification using CNN; (3) conveying fault information to the operator through images and fault location.

[0099] To achieve autonomous classification of line faults, the image analysis algorithm of the monitoring system utilizes a Convolutional Neural Network (CNN) for classification. CNN is a deep learning algorithm used for image analysis and classification. CNN has been shown to effectively reduce the number of parameters without sacrificing model quality. Here, the CNN network uses the 25-layer AlexNet CNN architecture. The fault classification program is implemented in the MATLAB platform. The weighting matrix in the MATLAB program is run on a Raspberry Pi 4 Model B with 4GB RAM for real-time validation and fault identification.

[0100] Images captured using a camera in a robotic system and a USB camera are fed into a trained CNN network. Comparing these images with features extracted from the training images, the CNN identifies faults and categorizes the captured images accordingly. This process is repeated for each captured image at 5-second intervals, and the information is then fed into a graphical user interface (GUI), enabling real-time monitoring of the electrical equipment.

[0101] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A welding robot with a positioning camera, comprising: a positioning camera 1, a top-view camera 2, an overall camera 3, a positioning camera mounting plate 101, a positioning camera head 102, and a welding head 103. The positioning camera 1 is located at the end of the robotic arm of the welding robot, and it can follow the robotic arm to complete omnidirectional movement. The top-view camera 2 is located on the forearm of the welding robot arm, and it takes top-view photos, scans and videos of the object being welded. The overall camera 3 is located on the support of the welding robot, and it takes pictures, scans and videos of the object being welded. The positioning camera 1 includes a positioning camera fixing plate 101, a positioning camera head 102, and a welding head 103. The positioning camera head 102 is fixed to the end of the robotic arm of the welding robot through the positioning camera fixing plate 101, and it follows the welding head 103 to complete omnidirectional movement.

2. A trajectory planning method for a welding robot with a positioning camera, characterized in that, It includes: navigation and localization modules for laser SLAM and visual SLAM, path planning algorithm modules for improved A* and DWA, and object detection method modules for convolutional neural networks.

3. The trajectory planning method for a welding robot according to claim 2, characterized in that, The navigation and localization module of the laser SLAM and visual SLAM employs the VLOAM algorithm to implement radar SLAM, utilizing a range measurement algorithm and a mapping algorithm in collaborative operation. High-frequency, low-fidelity odometer measurement and low-frequency, high-fidelity scan matching generate high-frequency and high-precision individual motion estimates. Odometer measurement primarily uses point cloud data acquired by the laser radar to calculate the robot's motion information. The laser radar collects point cloud data of the surrounding environment at a certain frequency (10 times per second). The point cloud data collected by the laser radar at two adjacent time points t1 and t2 are respectively... and Point cloud A point cloud containing N1 points. It contains N2 points. (1) Key feature points are extracted from the point cloud using feature extraction algorithms. Commonly used feature extraction algorithms are based on the geometric properties of points, such as curvature. For point clouds... Points in Calculate its curvature Select curvature greater than a certain threshold k th The points are taken as feature points. Let the extracted feature point set be... Similarly from Extract feature point set (2) Find the correspondence between feature points at two time points using a matching algorithm. For feature points Calculate its FPFH descriptor Similarly, for [the FPFH descriptor], calculate its [the FPFH descriptor]. Matching point pairs are found by comparing the distances between descriptors. Let the matched point pairs be... in (3) Based on the corresponding point pairs, calculate the relative motion transformation matrix from t1 to t2 using methods such as the least squares method. Let corresponding point pairs If the quantity is M, then the following equation can be solved using the least squares method: The formula for mileage measurement can be expressed as follows: Let the lidar coordinate system be L, and the robot body coordinate system be B. The coordinate transformation relationship for point p in different coordinate systems is as follows: In odometer measurement, the relative motion transformation matrix is ​​obtained through a series of feature extraction and matching calculations. So that the points in the point cloud at time t2 Its corresponding position at time t is estimated as follows:

4. The trajectory planning method for a welding robot according to claim 2, characterized in that, The navigation and localization modules for laser SLAM and visual SLAM employ scan matching, which further optimizes map construction based on odometry. Let the already constructed map point cloud be M, and the robot pose estimate obtained through odometry at the current moment be T. r The point cloud acquired by the lidar at the current moment is P. c . (1) Move the current point cloud P c Transform to a map coordinate system, i.e. (2) Finding through scanning matching algorithm The optimal match between the point cloud M and the map point cloud M. For the ICP algorithm, let m be the number of points in the map point cloud M. i (i = 1, 2, ..., N) m Current point cloud Point p in the middle j (j = 1, 2, ..., N) p By calculating the distances between points, each p is found. j The nearest neighbor m in M i Construct the corresponding relationship {(p j ,m i )}. (3) Let the optimized robot pose obtained by scanning matching be T. o The objective of the scanning and matching is to minimize the following objective function: in It refers to points in a point cloud of a map. This represents the points in the current point cloud transformed to the map coordinate system, and is the number of points participating in the matching. For the ICP algorithm, N = N p .

5. The trajectory planning method for a welding robot according to claim 2, characterized in that, The navigation and positioning module for laser SLAM and visual SLAM employs the VINSFusion algorithm to implement visual SLAM. Front-end visual information processing extracts and tracks visual information, while back-end information fusion and optimization processes the feature point tracking information obtained from the front end. The front-end visual information processing includes feature extraction and tracking, while the back-end information fusion and optimization includes fusing IMU and GPS information with optimization algorithms.

6. The trajectory planning method for a welding robot according to claim 1, characterized in that... The improved path planning algorithm module for A* and DWA: As a preferred embodiment of the present invention, the improved A* algorithm introduces a weight δ, making the heuristic function f(n) = δg(n) + (1-ω)h(n). Here, g(n) is the actual cost from the starting node to the current node n, and h(n) is the estimated cost from the current node n to the target node. The weight δ is adaptively adjusted based on the robot's position. Let the robot's current position be (x, y) and the target position be (x, y). g ,y g Define the distance function. When d is large, to improve search efficiency, the weight of g(n) is increased, i.e., δ is increased; when d is small, to improve path quality, the weight of h(n) is increased, i.e., δ is decreased. The specific adjustment method can be achieved through an exponential function. After completing the above, a path search is performed. The path search process is as follows: The algorithm starts the search from the initial node and adds it to an open list called OpenList, which stores nodes to be expanded. Simultaneously, a closed list called ClosedList is maintained to store nodes that have already been expanded. In each iteration, the node n with the smallest f(n) value is selected from the OpenList as the current node for expansion. For the neighboring nodes of the current node, g(m) = g(n) + c(n,m) is calculated, where c(n,m) is the movement cost from node n to node m. Simultaneously, h(m) is calculated, and then f(m) is calculated according to the adaptive dynamic weighting strategy. If node m is not in OpenList, add it to OpenList; if node m is already in OpenList and the newly calculated f(m) is less than the original value, update the relevant information of node m. The search ends when the target node is added to the ClosedList. The search then backtracks from the target node to obtain the path from the starting node to the target node.

7. The trajectory planning method for a welding robot according to claim 1, characterized in that... The improved path planning algorithm module of A* and DWA is described below: In the DWA algorithm, a dynamic window [v] is defined in the robot's velocity space. min ,v max ]×[ω min ,ω max ], where v is the robot's linear velocity and ω is the robot's angular velocity. This dynamic window limits the range of possible velocities of the robot at the current moment. Within the dynamic window, the robot's trajectory is calculated for each possible velocity based on its current position and environmental information. Let the robot's current position be (x, y, θ) (where θ is the robot's orientation angle). For each velocity, the position after a certain time can be calculated using kinematic equations. For each velocity within the dynamic window, an evaluation function G(v,ω) is calculated. This evaluation function comprehensively considers multiple factors, such as path safety (whether it will collide with obstacles), path smoothness, and path proximity to the target point. Let the distance from the path to the target point be d. t The curvature of the path is κ, and the distance to the obstacle is d. o The evaluation function can then be defined as G()G(v,ω)=αd t +βκ+γd o , where α, β and γ are weighting coefficients used to adjust the importance of different factors.

8. The trajectory planning method for a welding robot according to claim 1, characterized in that, The convolutional neural network target detection method module: The fault identification and classification process is carried out in three stages, including real-time capture of welding equipment images, using CNN to identify faults, and conveying fault information to the operator through images and fault location; The CNN network uses the 25-layer CNN architecture AlexNet, and the fault classification program is implemented on the MATLAB platform. The weighting matrix in the MATLAB program is run on a Raspberry Pi 4 Model B with 4GB RAM to obtain real-time verification and fault identification. Images captured by the camera in the robot system and USB camera are provided to the trained CNN network for comparison to identify faults and transmit information to the graphical user interface.

Images