A visual servoing submarine cable maintenance auxiliary method and system based on PSO-BP

Abstract

The application discloses a kind of visual servo submarine cable maintenance auxiliary method and system based on PSO-BP, the method is based on artificial potential field method on the basis of the characteristics of complex submarine environment, then carry out environment modeling, then carry out path search, construct intelligent path planning framework;After reaching fault point, for the different horizontal and vertical edge intersection of damaged submarine cable, further collect image information using camera;Finally, using the visual servo control of double purpose and based on image feature, by inputting position information into PSO-BP neural network, to obtain each joint angle, adjust joint, complete submarine cable maintenance task.The application can quickly find out safe, smooth, reasonable path;Combined with the characteristics of submarine cable, the visual servo system of submarine cable maintenance manipulator is designed based on PSO-BP neural network, which significantly improves the work efficiency of submarine cable maintenance, reduces the operation difficulty in complex marine environment, and can effectively complete the maintenance task of submarine cable fault point.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, specifically relating to a visual servo-assisted method and system for submarine cable maintenance based on PSO-BP. Background Technology

[0002] With the rapid development of artificial intelligence, more and more experts are focusing on and researching how to use AI to assist or replace humans in underwater operations. Submarine cables are widely used in industries such as offshore oil and gas and are crucial facilities. Damage to submarine cables often leads to power outages in one or more production units, significantly impacting production efficiency. Current submarine cable repair mainly includes fault area surveying, mud blowing, seabed salvage, and post-processing. Most of these tasks need to be carried out in complex marine environments, therefore, there is an urgent need to design a submarine cable repair method or system to assist or replace humans in completing the repair work.

[0003] Currently, there are many underwater intelligent working systems, but the systems for submarine cable maintenance are basically incomplete. The main difficulties in having a system complete submarine cable maintenance work on the seabed are: (1) The complex terrain and numerous obstacles in the seabed environment make it difficult for the maintenance system to quickly find and reach the fault area; (2) Due to the poor imaging quality in the seabed environment, there are many types of submarine cable faults with significant differences, resulting in a large error in the feature identification of submarine cable fault points; (3) The visual servo control calculation work in the seabed environment is complex and greatly affected by environmental noise, which makes the difficulty of submarine cable maintenance work increase exponentially.

[0004] In the path planning process, there are four main categories: traditional algorithms, intelligent biomimetic algorithms, computer graphics algorithms, and other algorithms. Considering the specific underwater conditions and combining the data obtained from the survey, the artificial potential field method in the traditional algorithm can solve this problem well. The path it plans is safe, smooth, and simple to describe. However, in special cases, it may also encounter local optima. Therefore, we need to design and improve the gravitational field appropriately.

[0005] Because light is absorbed and scattered underwater, the quality of underwater optical imaging is severely affected. To improve the accuracy of underwater image target detection and recognition, the collected images need to be processed first. After obtaining the enhanced image, this paper takes the faulty section of a submarine cable as the research target. Based on the fact that horizontal and vertical edges intersect outside the faulty section, a feature point recognition method for submarine cable fault sections based on different edge types is proposed. By classifying the horizontal and vertical edge types in the image processing algorithm, the target cutting points are extracted.

[0006] Visual servo control systems are a crucial component of submarine cable cutting robots. Currently, visual servoing is divided into position-based visual servoing (PBVS) and image-based visual servoing (IBVS). Position-based visual servoing estimates the target's pose relative to the camera, compares this pose with the current pose, and then adjusts the robot arm accordingly. However, because PBVS fixes the camera's position, it cannot acquire complete features in some mobile operations. Image-based visual servoing requires estimating the Jacobian matrix of the image during control, but this computation is complex and highly susceptible to environmental noise. Adding neural networks to visual servoing can avoid calculating the image's Jacobian matrix, simplifying the calculation process and making control smoother. However, this method requires specifying the dimensions of the target object, which is necessary for submarine cable repair work. Summary of the Invention

[0007] Purpose of the invention: In order to solve the problem of the high difficulty of submarine cable inspection and maintenance, this invention proposes a visual servo submarine cable inspection and maintenance assistance method and system based on PSO-BP, which can assist or replace humans in completing the inspection and maintenance of submarine cable fault points.

[0008] Technical solution: This invention provides a visual servo-based submarine cable maintenance assistance method based on PSO-BP, comprising the following steps:

[0009] (1) Based on the results of the route survey, collect and process the three-dimensional data in the scene;

[0010] (2) Using the artificial potential field method, autonomous motion under gravity and repulsion is simulated, and obstacles are avoided by the gravity and repulsion field function;

[0011] (3) By analogy with machine learning, path optimization is performed to reach the fault area of ​​the submarine cable;

[0012] (4) After arriving at the fault area, the camera was used to collect images and information based on the different intersections of the transverse and longitudinal edges of the damaged submarine cable; the target cutting point was extracted with the submarine cable fault point as the research target.

[0013] (5) The PSO-BP neural network is optimized by the particle swarm optimization algorithm to obtain the PSO-BP neural network. Using a binocular camera and visual servo control based on image features, the position information is input into the PSO-BP neural network to obtain the angle of each joint, adjust the joint of the robotic arm to cut the damaged submarine cable, and complete the submarine cable repair work.

[0014] Furthermore, the implementation process of step (1) is as follows:

[0015] The survey employs a variety of engineering geophysical exploration equipment to investigate obstacles and submarine cables based on fault location data, including the location and height of obstacles, seabed depth, topography, geomorphology, and the direction, exposure, and deformation of submarine cables.

[0016] Furthermore, the implementation process of step (2) is as follows:

[0017] The maintenance assistance system is attracted to the target point and repulsed by obstacles, simulating the autonomous movement of an object under the influence of attraction and repulsion.

[0018] The gravitational field function in the construction is U a :

[0019]

[0020] The derivative of the gravitational field with respect to distance is F. a :

[0021]

[0022] The repulsive field function is constructed as follows:

[0023]

[0024] The derivative of the repulsive field with respect to distance is defined as the gradient of the repulsive field:

[0025]

[0026] The overall artificial potential field is a linear superposition of the gravitational and repulsive fields:

[0027] U = U a +U r (5)

[0028] Where ξ is the scale factor, ρ(q,q) goal ) represents the distance from the system to the target point, η represents the repulsive radius, and ρ(q, q) represents the distance from the system to the target point. obs ) represents the distance between the object and the obstacle, and ρ0 represents the radius at which each obstacle can exert its influence; the gravitational repulsive field function is modified to solve the problem of motion trajectory oscillation caused by the approximate equality of gravitational and repulsive forces.

[0029] Furthermore, the implementation process of step (3) is as follows:

[0030] The established artificial potential field is analogous to the objective function in machine learning. Using the gradient descent algorithm, the movement position of the maintenance system is continuously adjusted so that the total energy of the maintenance system in the artificial potential field continues to decrease. When the energy reaches the minimum point, that is, when the objective function converges to the global optimum, the path search ends and a reasonable movement path is obtained.

[0031] Furthermore, the implementation process of step (4) is as follows:

[0032] Images of the fault area are acquired, then preprocessed to segment the background and extract the shape of the submarine cable. The images are then binarized to obtain a binary image. The horizontal and vertical edges of the submarine cable are extracted from the binary image, and the intersection of the horizontal and vertical edges is determined. This intersection is used as the feature point of the damaged submarine cable. Finally, normal points are removed to identify the fault point.

[0033] Furthermore, the optimization process of the BP neural network using the particle swarm optimization algorithm in step (5) is as follows:

[0034] Based on the training of the BP neural network, uncalibrated visual servo control is performed, and iterative practice is continuously carried out to obtain a well-trained BP neural network that maps the joint angles of the robotic arm to the target feature points.

[0035] The Particle Swarm Optimization (PSO) algorithm is used to optimize the Backpropagation (BP) neural network, thereby accelerating its convergence speed. First, the structure and parameters of the neural network are determined. The velocity and position of the particles are initialized, and their fitness is calculated. Then, each particle is evaluated to determine if it has reached the preset number of iterations. If so, the globally optimal weights are assigned to the BP network, and the angles of each joint are predicted using the BP neural network to rotate the robotic arm joints to cut the submarine cable. If not, the particle velocity and position are optimized, and the above process is repeated.

[0036] Furthermore, the implementation process of the modified gravitational repulsive field function is as follows:

[0037] When the system is too far from the target point, the gravitational force exerted by the target point on the system is too large. This gravitational force is much greater than the repulsive force exerted by nearby obstacles on the system, leading to a collision with the obstacles. A new repulsive force function is introduced to effectively counteract part of the gravitational force. Based on the original repulsive force field, the influence of the distance between the target point and the system is reduced, thereby effectively avoiding collisions. The new repulsive force function is:

[0038]

[0039] Where ρ1 represents the minimum distance at which a new repulsive force is introduced into the environment.

[0040] Based on the same inventive concept, this invention also proposes a visual servoing submarine cable repair assistance system based on PSO-BP, including a path planning module, a sensor module, a central processing unit module, a visual servoing module, a server module, and external devices. The external devices initially identify obstacles and the condition of the submarine cable and transmit the data to the path planning module. The path planning module performs path planning using the artificial potential field method. The sensor module is mainly used to collect surrounding environmental data and then transmit the data to the central processing unit module in real time. The central processing unit performs image stitching, image enhancement processing, and target feature point extraction. A camera is installed on the end effector in the visual servoing module to simulate an eye on a hand, using binocular, image feature-based visual servoing to control the robotic arm to cut the submarine cable. The server module is responsible for establishing communication with the personnel and sending and receiving signals.

[0041] Beneficial Effects: Compared with existing technologies, the beneficial effects of this invention are as follows: 1. This invention uses a path planning scheme based on the artificial potential field method. During path optimization, it employs an analogy machine learning algorithm, comparing the established artificial potential field to the objective function in machine learning. It continuously adjusts the movement position of the maintenance system, causing the objective function to converge to the global optimum, thus obtaining a reasonable movement path and ultimately enabling the maintenance system to accurately reach the submarine cable fault area. 2. Utilizing gradient descent for path search greatly accelerates the convergence speed of the objective function and increases the stability of the path search results. 3. A method for horizontal and vertical edge segmentation is proposed, removing normal points on the submarine cable, integrating horizontal and vertical edges, and extracting target feature points. This efficiently and accurately identifies submarine cable fault points, providing a guarantee for subsequent cutting work. 4. Using PSO-BP-based visual servo control to complete the maintenance work avoids calculating the image Jacobian matrix, simplifying the calculation process and improving work efficiency. Attached Figure Description

[0042] Figure 1 Schematic diagram of the working principle of a visual servo submarine cable inspection system;

[0043] Figure 2 Flowchart of a visual servoing submarine cable maintenance assistance method based on PSO-BP;

[0044] Figure 3 Flowchart of the method for identifying feature points of damaged submarine cables;

[0045] Figure 4 The flowchart for vision servoing based on PSO-BP;

[0046] Figure 5 Schematic diagram of a visual servo submarine cable inspection system;

[0047] Figure 6 This is a schematic diagram of a BP neural network structure. Detailed Implementation

[0048] The present invention will now be described in further detail with reference to the accompanying drawings.

[0049] This invention proposes a visual servo-assisted method for submarine cable maintenance based on the Particle Swarm Optimization-Back Propagation (PSO-BP) algorithm. Figure 1 To illustrate its working principle, firstly, an intelligent path planning framework is constructed. This involves establishing a gravitational field function to avoid obstacles and using gradient descent to optimize the path and find a reasonable route. Secondly, considering the characteristic that the external surface of the submarine cable fault point will have intersections of horizontal and vertical edges due to exposure and deformation, cameras are used to further collect key information. Finally, the position information can be input into a PSO-BP neural network to obtain the angles of each joint, adjust the joints, and complete the submarine cable maintenance task. Figure 2 As shown, the specific steps include:

[0050] Step 1: Based on the results of the route survey, collect and process the 3D data in the scene.

[0051] Route surveying is an important preventative maintenance method for submarine facilities, used to determine the location, shape, and obstacle changes of the entire submarine cable route from a macroscopic perspective. The survey uses a variety of engineering geophysical survey equipment to identify obstacles and the condition of submarine cables, including obstacle locations, seabed depth, topography, submarine cable routing, exposure, deformation, etc., providing a basis for the next step of repair and construction.

[0052] Step 2: Use the artificial potential field method to simulate autonomous motion under gravity and repulsion, and avoid obstacles by using the gravity and repulsion field function.

[0053] The system performs environmental modeling, employing the artificial potential field method to simulate autonomous motion under gravitational and repulsive forces. Then, it utilizes optimization algorithms for path search, constructing an intelligent path planning framework. Upon receiving fault point detection information, the system begins operation, descending while observing the surrounding environment. After observing the target point and obstacles, modeling begins. Assuming the system's relationship with the target point is one of attraction, and with obstacles a repulsive force, the system simulates the autonomous motion of an object under these forces, establishing the gravitational and repulsive force field function, described as follows:

[0054] The gravitational field function in the construction is U a :

[0055]

[0056] The derivative of the gravitational field with respect to distance is F. a :

[0057]

[0058] The repulsive field function is constructed as follows:

[0059]

[0060] The derivative of the repulsive field with respect to distance is defined as the gradient of the repulsive field:

[0061]

[0062] The overall artificial potential field is a linear superposition of the gravitational and repulsive fields:

[0063] U = U a +U r (5)

[0064] Where ξ is the scale factor, ρ(q,q) goal ) represents the distance from the system to the target point, η represents the repulsive radius, and ρ(q, q) represents the distance from the system to the target point. obs ρ represents the distance between the object and the obstacle, and ρ0 represents the radius from which each obstacle can exert its influence. The gravitational repulsive field function is modified to address issues such as trajectory oscillations caused by approximately equal gravitational and repulsive forces.

[0065] By increasing system damping to reduce the oscillation amplitude of an object, the problem of trajectory oscillation caused by approximately equal gravitational and repulsive forces can be effectively solved. When the system is too far from the target point, the gravitational force exerted by the target point on the system is too large, far exceeding the repulsive force exerted by nearby obstacles, leading to a collision. To solve this problem, a new repulsive force function is introduced to effectively counteract part of the gravitational force. Based on the original repulsive force field, the influence of the distance between the target point and the system is reduced, thereby effectively avoiding collisions. The new repulsive force function is:

[0066]

[0067] Here, ρ1 represents the minimum distance at which the new repulsive force is introduced in the environment. By modifying the gravitational repulsive force field function, the relationship between the target point and the obstacle on the system can be described more accurately and effectively, thus effectively expanding the scope of application of this model.

[0068] Step 3: Optimize the path using machine learning to reach the faulty area of ​​the submarine cable.

[0069] The established artificial potential field is analogous to the objective function in machine learning. Using the gradient descent algorithm, the movement position of the maintenance system is continuously adjusted so that the total energy of the maintenance system in the artificial potential field continues to decrease. When the energy reaches the minimum point, that is, when the objective function converges to the global optimum, the path search ends and a reasonable movement path is obtained.

[0070] Step 4: After arriving at the fault area, use a camera to collect images and information at the different intersections of the transverse and longitudinal edges of the damaged submarine cable; take the fault point of the submarine cable as the research target and extract the target cutting point.

[0071] like Figure 3 As shown, it was observed that most fault points on submarine cables were externally damaged, causing differences between the horizontal and vertical edges and normal points. A method for horizontal and vertical edge segmentation is proposed to remove the influence of normal points, and finally, the horizontal and vertical edges are integrated to extract target feature points. First, images of the approximate damage location of the submarine cable are acquired. Then, the images are preprocessed, the background is segmented, and the cable's outline is extracted. Next, binarization is performed to obtain a binarized image. In the binarized image, the horizontal and vertical edges of the submarine cable are extracted separately, and the intersection of the horizontal and vertical edges is determined. This intersection is used as the feature point of the damaged submarine cable. Finally, normal points are removed, and the working point is identified.

[0072] Step 5: Optimize the BP neural network using the particle swarm optimization algorithm to obtain the PSO-BP neural network. Using a binocular camera and image feature-based visual servo control, position information is input into the PSO-BP neural network to obtain the angles of each joint. The robotic arm joints are then adjusted to cut the damaged submarine cable, completing the submarine cable repair work. Specifically... Figure 4 As shown.

[0073] like Figure 5 As shown, using binocular, image feature-based visual servo control, the robot arm obtains the angles of each joint by inputting position information into the PSO-BP neural network, and adjusts the joints to cut the damaged submarine cable, thus completing the submarine cable repair work.

[0074] A backpropagation (BP) network is a multilayer feedforward network trained using the backpropagation algorithm. It can learn and store a large number of input-output pattern mappings without needing to reveal the mathematical equations describing these mappings beforehand. For example... Figure 6 As shown, uncalibrated visual servo control can be performed based on BP neural network training. Through continuous iterative learning, a network is obtained that effectively reflects the mapping relationship between target feature points and the joint angles of the robotic arm in the maintenance system. Integrating the PSO algorithm into the neural network performs a general global search, obtaining optimized weights and thresholds for the BP network, significantly improving its learning ability and accelerating convergence. First, the structure and parameters of the BP neural network are set, and the training data is normalized. Then, the particle velocity and position are initialized; particle fitness is calculated; each particle is evaluated, and if the conditions are met, the globally optimal weights are assigned; otherwise, the particle velocity and position are optimized, and the particle fitness is calculated again. Finally, the BP neural network is used to predict the joint angles of the robotic arm.

[0075] Based on the same inventive concept, this invention also proposes a visual servoing submarine cable inspection assistance system based on PSO-BP, including a path planning module, a sensor module, a central processing unit module, a visual servoing module, a server module, and external devices. The external devices initially identify obstacles and the condition of the submarine cable and transmit the data to the path planning module; the path planning module performs path planning using the artificial potential field method; the sensor module is mainly used to collect surrounding environmental data and then transmit the data to the central processing unit module in real time; the central processing unit performs image stitching, image enhancement processing, and target feature point extraction; a camera is installed on the end effector in the visual servoing module to simulate an eye on a hand, using binocular, image feature-based visual servoing to control the robotic arm to cut the submarine cable; the server module is responsible for establishing communication with personnel and sending and receiving signals.

[0076] The maintenance system rotates the robotic arm to cut the faulty section of the submarine cable based on the angle obtained from training, and then contacts the staff through the server module to bring the faulty submarine cable out of the water to complete the auxiliary maintenance work.

[0077] In summary, this invention employs a path planning scheme based on the artificial potential field method, resulting in a safe, smooth, and easily describable path. Utilizing gradient descent for path search significantly accelerates the convergence of the objective function to the global optimum, increasing the stability of the path search results. A method for identifying feature points of submarine cable fault segments based on different edge types is proposed, which can efficiently and accurately locate cable fault points, providing assurance for subsequent cutting work. Finally, a vision servo-controlled robot based on PSO-BP is used to complete the maintenance work, avoiding the calculation of the image Jacobian matrix, simplifying the calculation process, and improving work efficiency.

Claims

1. A visual servo-based submarine cable maintenance assistance method based on PSO-BP, characterized in that, Includes the following steps: (1) Based on the results of the route survey, collect and process the three-dimensional data in the scene; (2) Using the artificial potential field method, autonomous motion under gravity and repulsion is simulated, and obstacles are avoided by using the gravity and repulsion field function; (3) By analogy with machine learning, path optimization is performed to reach the fault area of ​​the submarine cable; (4) After arriving at the fault area, for the different intersections of the transverse and longitudinal edges of the damaged submarine cable, use a camera to further collect images and information; take the submarine cable fault point as the research target and extract the target cutting point; (5) The PSO-BP neural network is optimized by the particle swarm optimization algorithm to obtain the PSO-BP neural network. Using a binocular camera and visual servo control based on image features, the position information is input into the PSO-BP neural network to obtain the angle of each joint, adjust the joint of the robotic arm to cut the damaged submarine cable, and complete the submarine cable repair work. The implementation process of step (2) is as follows: The maintenance assistance system is attracted to the target point and repulsed by obstacles, simulating the autonomous movement of an object under the influence of attraction and repulsion. The gravitational field function under construction is : （1） The derivative of the gravitational field with respect to distance is : （2） The repulsive field function is constructed as follows: （3） The derivative of the repulsive field with respect to distance is defined as the gradient of the repulsive field: （4） The overall artificial potential field is a linear superposition of the gravitational and repulsive fields: （5） in, As a scale factor, The distance from the system to the target point, Where is the repulsion radius, Represents the distance between an object and an obstacle. Represents the radius at which each obstacle can exert its effect; corrects the gravitational repulsive field function to solve the problem of trajectory oscillation caused by approximately equal gravitational and repulsive forces; The implementation process of the modified gravitational repulsive field function is as follows: When the system is too far from the target point, the gravitational force exerted by the target point on the system is too large. This gravitational force is much greater than the repulsive force exerted by nearby obstacles on the system, leading to a collision with the obstacles. A new repulsive force function is introduced to effectively counteract part of the gravitational force. Based on the original repulsive force field, the influence of the distance between the target point and the system is reduced, thereby effectively avoiding collisions. The new repulsive force function is: （6） in, This represents the minimum distance at which a new repulsive force is introduced into the environment.

2. The visual servoing submarine cable maintenance assistance method based on PSO-BP according to claim 1, characterized in that, The implementation process of step (1) is as follows: The survey employs a variety of engineering geophysical exploration equipment to investigate obstacles and submarine cables based on fault location data, including the location and height of obstacles, seabed depth, topography, geomorphology, and the direction, exposure, and deformation of submarine cables.

3. The visual servoing submarine cable maintenance assistance method based on PSO-BP according to claim 1, characterized in that, The implementation process of step (3) is as follows: The established artificial potential field is analogous to the objective function in machine learning. Using the gradient descent algorithm, the movement position of the maintenance system is continuously adjusted so that the total energy of the maintenance system in the artificial potential field continues to decrease. When the energy reaches the minimum point, that is, when the objective function converges to the global optimum, the path search ends and a reasonable movement path is obtained.

4. The visual servoing submarine cable maintenance assistance method based on PSO-BP according to claim 1, characterized in that, The implementation process of step (4) is as follows: Images of the fault area are acquired, then preprocessed to segment the background and extract the shape of the submarine cable. The images are then binarized to obtain a binary image. The horizontal and vertical edges of the submarine cable are extracted from the binary image, and the intersection of the horizontal and vertical edges is determined. This intersection is used as the feature point of the damaged submarine cable. Finally, normal points are removed to identify the fault point.

5. The visual servoing submarine cable maintenance assistance method based on PSO-BP according to claim 1, characterized in that, The optimization process of the BP neural network using the particle swarm optimization algorithm in step (5) is as follows: Based on the training of the BP neural network, uncalibrated visual servo control is performed, and iterative practice is continuously carried out to obtain a well-trained BP neural network that maps the joint angles of the robotic arm to the target feature points. The Particle Swarm Optimization (PSO) algorithm is used to optimize the Backpropagation (BP) neural network, thereby accelerating its convergence speed. First, the structure and parameters of the neural network are determined. The velocity and position of the particles are initialized, and their fitness is calculated. Then, each particle is evaluated to determine if it has reached the preset number of iterations. If so, the globally optimal weights are assigned to the BP network, and the angles of each joint are predicted using the BP neural network to rotate the robotic arm joints to cut the submarine cable. If not, the particle velocity and position are optimized, and the above process is repeated.

6. A visual servoing submarine cable maintenance assistance system based on PSO-BP, implemented using the visual servoing submarine cable maintenance assistance method based on PSO-BP as described in any one of claims 1 to 5, characterized in that, The system includes a path planning module, a sensor module, a central processing unit (CPU) module, a visual servoing module, a server module, and external devices. The external devices initially identify obstacles and the condition of the submarine cable, and transmit the data to the path planning module. The path planning module performs path planning using an artificial potential field method. The sensor module primarily collects surrounding environmental data and transmits it to the CPU module in real time. The CPU performs image stitching, image enhancement, and target feature point extraction. A camera is installed on the end effector in the visual servoing module to simulate an eye on a hand, using binocular, image feature-based visual servoing to control the robotic arm to cut the submarine cable. The server module is responsible for establishing communication with personnel and sending and receiving signals.

Images