Pipeline inspection bionic robotic fish based on double-vision system and control method of pipeline inspection bionic robotic fish

Abstract

The invention discloses a pipeline inspection bionic robotic fish based on a double-vision system and a control method thereof, and relates to the technical field of underwater robots. The robotic fish comprises a machine body and actuator unit, a measuring dual-camera subsystem, an upper processing and man-machine interaction unit, a control unit and a power supply unit. The machine body and actuator unit comprises a single-joint bionic fish body shell, a steering engine and a tail fin; the dual-camera subsystem comprises a line patrol subsystem and an identification subsystem; the upper processing unit is used for interaction and event summarization; and the control unit controls the steering engine according to the segmented adaptive control law. The control method comprises the steps of system initialization, image acquisition and return, line patrol and recognition image processing, upper computer event processing and the like. Through dual-vision function decoupling and segmented adaptive control, stable line patrol, rapid steering and reliable event detection in a complex pipeline environment are realized, and the system has the advantages of high real-time performance, stable control, low cost and the like, and is suitable for a pipeline patrol scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater robots, in particular to a pipeline inspection bionic robotic fish based on a dual vision system and a control method thereof, which is suitable for automatic inspection operation in municipal pipe network, small and medium-sized water surface pipeline and other scenes. BACKGROUND

[0002] Pipeline transportation is an important fluid transportation method, and its regular inspection is crucial for preventing leakage and ensuring safety. At present, the inspection of underwater and water surface pipelines is still mainly operated by manual operation. The common way is for divers or operators to visually inspect, take photos or videos through cameras and handheld detection devices, and then return to the shore for manual analysis. This method has the problems of high operation risk, low inspection efficiency and great influence of water quality and light conditions.

[0003] Although some industries have adopted remotely operated vehicles (ROVs) for auxiliary inspection, such equipment is mostly dependent on imports, has high costs, complex operation, and still relies on manual operation, which requires high personnel training and cannot achieve automatic inspection and analysis. At the same time, remote ROVs are mainly used for deep-sea oil and gas pipeline or cable inspection scenes and are not suitable for regular inspection of municipal pipe networks or small and medium-sized water surface pipelines. Therefore, a new system architecture and control method are urgently needed to solve the above problems.

[0004] Using underwater robots for pipeline inspection is an efficient and safe means. Bionic robotic fish has great potential in this field due to its high propulsion efficiency, good maneuverability, and small disturbance to flow field. The existing inspection robots generally have serious coupling of vision system functions, high control response delay, and poor environmental adaptability, and a new system architecture and control method are urgently needed to solve the above technical bottlenecks. SUMMARY

[0005] The present application aims to solve the problems of insufficient underwater pipeline inspection technology samples, mainly manual operation, low automation, coupled vision system functions, insufficient control precision and other problems, and provides a bionic robotic fish for pipeline inspection based on a dual vision system and a control method thereof. By constructing a functional division system of the dual vision subsystem, and moving the key closed loop of vision processing and heading control to the line inspection end, combined with a segmented adaptive control law, stable line inspection, rapid turning and reliable event detection in complex pipeline environments are achieved.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] The present application provides a bionic robotic fish for pipeline inspection based on a dual vision system, which comprises a body and an actuator unit, a measurement dual-camera subsystem, an upper processing and human-computer interaction unit, a control unit and a power supply unit.

[0008] The body and actuator unit comprises a single-joint bionic fish body shell, a rudder at the joint, and a tail fin connected to the rudder;

[0009] The measurement double-camera subsystem comprises a line-patrolling subsystem and an identification subsystem; the line-patrolling subsystem is provided with a line-patrolling camera aiming obliquely downward in the front direction of the pipeline, for collecting images and performing preprocessing and straight-line regression, and for real-time calculation and output of the deviation angle of the robotic fish relative to the central axis of the pipeline and the deviation distance ; the identification subsystem is provided with an identification camera for viewing the external area of the pipeline, for detecting specific targets outside the pipeline, and for sending an identification signal “Detected” or “None” to the upper processing and human-computer interaction unit;

[0010] The upper processing and human-computer interaction unit is in communication connection with the double-camera subsystem through a communication interface, for receiving the identification signal from the identification subsystem and performing counting, visual output, and voice broadcast, and for receiving each live parameter information from the line-patrolling subsystem for visual display, while displaying the live shooting images; the upper processing and human-computer interaction unit is decoupled from the line-patrolling visual system and the rudder in the control loop, and does not participate in the instant steering control, but only makes event summary and human-computer interaction;

[0011] The control unit is integrated in the line-patrolling subsystem, for receiving the deviation angle and the deviation distance , and calculating the control amount of the rudder according to the piecewise adaptive control law, and mapping the control amount to the theoretical turning angle of the rudder;

[0012] The power supply unit is used to provide stable voltage power supply for the rudder, the double-camera subsystem, and the upper processing and human-computer interaction unit.

[0013] Further, the drivable angle domain of the rudder is 60°-120°.

[0014] Further, the identification subsystem further comprises a debouncing state machine unit, which is configured to send an event signal only once when the identification state jumps from “None” to “Detected”; if the target is not detected for a preset reset time, the state is reset to “None”.

[0015] Further, the control unit comprises a state and control amount unit, a heading control unit, and a constraint and disturbance processing unit, in particular:

[0016] The state and control amount unit is used to obtain navigation state parameters, the state parameters at least comprising a heading angle , a deviation angle , pipe direction angle α and deviation distance , heading angle , deviation angle , pipe direction angle α satisfies the relationship ; control quantity is generated according to the state parameter ; , wherein is a control coefficient; the control quantity is converted into a rudder theoretical rotation angle through a certain mapping relationship , that is , and the specific relationship is defined by the actual situation; and then driving the tail fin to swing, so as to affect the linear speed v and angular speed of the tail fin swing of the bionic robotic fish

[0017] heading angle (angle unit: °): the included angle between the direction of the robotic fish and the reference axis;

[0018] deviation angle (angle unit: °): the included angle between the direction of the robotic fish and the pipe, that is, the regression line;

[0019] pipe direction angle α (angle unit: °): the included angle between the pipe, that is, the regression line and the reference axis;

[0020] deviation distance (pixel unit): the distance from the image center point to the regression line;

[0021] The heading control unit is used to control the robotic fish to travel along the pipe axis direction, and adaptively adjust the control strategy under different working conditions to realize the coordinated control of stable straight-line navigation, rapid steering response and event recognition.

[0022] Under the normal heading / position control working condition, the heading control unit generates a driving control quantity to make the deviation angle and the deviation distance converge to 0, so that the robotic fish travels stably along the pipe axis direction; if the deviation angle and the deviation distance are both less than a preset threshold, the generated control quantity is close to zero, at which time only the original period and amplitude of the tail fin are maintained to swing, at which time the navigation body can be regarded as being in a stable straight-line navigation state.

[0023] Under the turning working condition, when the absolute value of the deviation angle or the absolute value of the deviation distance exceeds a preset threshold, the heading control unit outputs a rapid steering control quantity to shorten the response time and avoid overshoot or off-line, and ensure smooth turning.

[0024] In the event recognition working condition, the heading control unit maintains the above-mentioned closed-loop control process undisturbed and guarantees reliable counting of the system to the identified event.

[0025] The constraint and disturbance processing unit is used for analyzing and processing visual noise, target breakage or partial absence, etc., and limiting the angle domain restriction and action beat / rate restriction of the single rudder machine actuator.

[0026] Further, the upper processing and human-computer interaction unit comprises an upper computer, a display module and an audio module; the upper computer can adopt Raspberry Pi 5 or Raspberry Pi 4B; the display module can adopt an LCD1602 display screen in I2C communication; and the audio output of the audio module is realized through a Bluetooth sound box.

[0027] The application provides a control method of a pipeline inspection bionic robotic fish based on a dual-vision system, comprising the following steps:

[0028] 1) system initialization and calibration, sequentially comprising hardware initialization, camera calibration and parameter loading, rudder self-checking and zeroing, communication interface initialization and system synchronization and preparation;

[0029] 2) the dual-camera subsystem performs image acquisition and feedback, the line inspection camera acquires real-time images of the pipeline in front of the robotic fish, the recognition camera acquires real-time overhead images of the pipeline, and all acquired images are synchronously fed back to the upper processing and human-computer interaction unit with time stamps and frame numbers;

[0030] 3) the line inspection camera built-in program completes line inspection image processing and rudder control, and according to whether an effective pipeline connected domain is detected, corresponding operations are respectively performed to realize automatic heading correction;

[0031] 4) the recognition camera built-in program completes recognition image processing and target detection, and feeds back detection information to the upper processing and human-computer interaction unit;

[0032] 5) the upper processing and human-computer interaction unit completes recognition event processing and feedback, line inspection feedback and image feedback.

[0033] In step 1), the specific steps of the system initialization and calibration comprise: 1.1) hardware initialization: starting the dual-camera subsystem, the upper processing and human-computer interaction unit and the rudder, completing power supply detection and communication connection confirmation;

[0034] 1.2) camera calibration and parameter loading: performing distortion-free calibration on the line inspection camera and the recognition camera, loading the intrinsic matrix and distortion coefficient thereof; and reading a system configuration file, including threshold parameters, morphological kernel size, communication baud rate, feedback period, reset time, etc.

[0035] 1.3) Rudder self-checking and zeroing: Perform zero position calibration and action detection on the rudder, confirm that the steering sensitivity and limit angle are within the safe range;

[0036] 1.4) Communication interface initialization: Establish serial or VCP communication channel between the host computer and the camera, check data transmission stability and packet loss rate;

[0037] 1.5) System synchronization and preparation: Dual camera module starts frame synchronization mechanism to ensure that the image acquisition frame rate and control loop period are consistent, the system enters standby state, and prepares to start real-time inspection and identification tasks.

[0038] In step 2), the specific steps of the dual camera subsystem for image acquisition and transmission include:

[0039] 2.1) Line inspection camera image acquisition: The line inspection camera acquires real-time images of the pipe in front of the robot fish, which is used for line inspection analysis;

[0040] 2.2) Identification camera image acquisition: The identification camera acquires real-time overhead images of the pipe for detection analysis;

[0041] 2.3) Synchronous transmission: All acquired images are synchronized with timestamps and frame numbers and transmitted to the host computer for processing and human-computer interaction.

[0042] In step 3), the line inspection camera built-in program completes line image processing and rudder control. If an effective pipe connected domain is detected, it performs image thresholding, morphological operation, connected domain extraction, straight line regression and deviation calculation, working condition discrimination and motion control, and data transmission in sequence. If no effective pipe connected domain is detected, the line loss detection and protection mechanism is triggered. Specifically:

[0043] Image thresholding: Perform thresholding operation on the images acquired by the line inspection camera to separate the pipe and background. Fixed threshold or adaptive threshold algorithm can be selected according to the lighting conditions.

[0044] Morphological operation: Perform erosion operation on the binary image to remove noise points, and then perform dilation operation to restore connectivity, ensuring the integrity of the pipe region.

[0045] Connected domain extraction: Extract the largest connected domain as the region of interest (ROI) to accurately select the pipe, and perform regression and calculation in this region.

[0046] Straight line regression and deviation calculation: Fit the pipe regression line in the ROI, and calculate the deviation distance and deviation angle of the robot fish relative to the pipe ​The regression algorithm can employ either the least squares method or RANSAC (Random Sample Consensus Algorithm), the latter of which can improve the robustness of the fit under conditions of image noise or local occlusion.

[0047] Operating condition identification and motion control: based on deviation distance and deviation angle Determine the current operating condition (e.g., going straight or turning) and calculate the servo control input. Theoretical turning angle of servo motor Theoretical steering angle of a servo motor After being limited and smoothed, the signal is sent to the servo motor to achieve automatic heading correction.

[0048] Data feedback: This will transmit the deviation angle. Deviation distance Control quantity Parameters and operating condition information are synchronously transmitted back to the host computer visualization platform for analysis.

[0049] If no valid pipe connectivity is detected, the LostLine detection and protection mechanism is triggered. First, the robotic fish executes a line-finding strategy, including: slight left and right rudder sweeps; increasing the ROI and lowering the threshold. If no pipe is still detected, a "LostLine" status is reported, and the speed is reduced or movement is paused.

[0050] Furthermore, the core control law for the working condition judgment and motion control is based on the deviation angle. Deviation distance Calculate the theoretical turning angle of the servo motor using the input. To improve turning performance and steady-state accuracy, the core control law employs a comprehensive mechanism combining dual-channel fusion, three-segment adaptive angle domain, beat speed limiting, saturation anti-shake, and safe homing, including:

[0051] Dual-channel error synthesis: deviation angle Deviation distance The calculations are performed using either P / PD or PID (the proportional gain can be set separately). Differential gain Proportional gain Differential gain ), to obtain the corresponding control quantity This is then mapped to the theoretical turning angle of the servo motor. ;

[0052] Active angular domain adaptive (segmented control): given angle ,distance ; Micro deviation area ( and ):Regulation For example, A1 is ±20°, the tail fin only swings slightly symmetrically, maintaining straight navigation and avoiding overshoot; in the middle deviation zone ( or ): Cornering limit For example, A2 is dynamically adjusted within the range of ± (30° to 45°); the adjustment is based on factors including the deviation convergence speed, environmental noise level, or visual confidence, thereby achieving an adaptive balance between "correction speed" and "oscillation suppression"; large deviation / sharp turn zone ( or ): Extension For example, A3 can be dynamically adjusted within the range of ± (60° to 85°) to provide sufficient angular velocity. It can quickly correct deviations from sharp bends and large deviations without going off track.

[0053] Clock / rate limit: Set servo_delay as the servo action clock cycle (typically 120~160ms), and perform "reciprocating oscillation-sampling" within each clock cycle to limit the control quantity update rate and angular velocity rise, thereby avoiding actuator saturation and oscillation;

[0054] Saturation and anti-shake mechanism: for the theoretical angle of the servo motor Amplitude limiting is applied to obtain the actual turning angle. : Superimpose back-and-forth scans within a small area to suppress local optimum traps and dead zone lock-in phenomena;

[0055] Reset Strategy (Safety / Shutdown): In the event of shutdown or detection of a serious anomaly, Forced reset ( This protects the implementing agency and reduces the risk of false triggering.

[0056] In step 4), the built-in program of the recognition camera completes the recognition image processing and target detection, and sends the detection information back to the host computer. The specific steps are as follows:

[0057] 4.1) Distortion Correction: Distortion correction is performed on the images captured by the recognition camera to eliminate lens distortion;

[0058] 4.2) Target contour region search: Search for candidate regions in the corrected image that conform to the set geometric contour features;

[0059] 4.3) Color Statistics and Target Determination: Perform color statistics on candidate areas, such as identifying characteristic colors like black oil leaks and minerals; if the color conditions meet the preset threshold, proceed to the area filtering step.

[0060] 4.4) Area Limiting Processing: Perform area filtering on candidate targets, retaining only target areas that meet the predetermined area range;

[0061] 4.5) State determination (including filter jitter mechanism): according to the detection result of the target area, the state determination is made by comprehensively considering multiple frames of detection and filter jitter mechanism; the filter jitter mechanism avoids repeated identification caused by temporary loss of target by setting a minimum time (reset time) for allowing state change; if a target area meeting all conditions is detected, the identification state jumps to "Detected", and remains for a complete reset period without the target area appearing again; if not, the "None" identification state is maintained; a multi-frame confirmation mechanism (such as detecting the target for 3 consecutive frames to confirm "Detected") can be used to further improve the identification stability and accuracy;

[0062] 4.6) Data return: the identification state, target position and quantity information are returned to the host computer of the upper processing and human-computer interaction unit.

[0063] In step 5), the upper processing and human-computer interaction unit completes the identification event processing and feedback, line patrol feedback and image feedback, which specifically includes:

[0064] 5.1) Identification event jitter filtering again: after receiving the event information from the identification camera, the host computer (main control board) sets a reset time again to stabilize the identification state twice;

[0065] 5.2) Identification event recording: when the identification state jumps from "None" to "Detected", the host computer records the event and counts the number of targets;

[0066] 5.3) Identification event output and feedback: the host computer outputs the identification event and state information to external devices through serial communication or Bluetooth, and performs visual feedback on the display module, and triggers the voice broadcast module to prompt;

[0067] 5.4) Line patrol information feedback: after receiving the line patrol information, the host computer can display the parameters such as 、 、 and working condition information on the visual interface to assist in adjustment and monitoring;

[0068] 5.5) Image feedback: the host computer interface displays the live images from the dual cameras in real time, and superimposes the regression line, robot fish center point, deviation distance vertical line and deviation angle arc line to provide intuitive visual monitoring.

[0069] Compared with the prior art, the beneficial effects of the present application are:

[0070] 1. By assigning the dual-camera subsystem to line patrol closed-loop control and event recognition respectively, and moving the image processing and control calculation to the embedded processing unit built-in the camera, the host computer only bears data aggregation and human-computer interaction, realizing function decoupling, avoiding mutual blocking between function modules due to shared computing power or configuration conflicts, realizing extremely low delay work efficiency, and improving the real-time performance and reliability of the system.

[0071] 2. The segmented adaptive control law enables the robotic fish to provide sufficient steering torque to achieve rapid maneuvering in large deviation conditions, and to fine-tune and suppress overshoot in small deviation conditions, ensuring the stability and passability of line patrol.

[0072] 3. The beat limiting mechanism prevents actuator saturation oscillation; the visual preprocessing and debouncing state machine design effectively suppresses underwater optical interference and false triggering, making event detection more reliable.

[0073] 4. The system architecture of the present application is clear in hierarchy, and each unit has clear function, and the parameters can be flexibly adjusted according to the actual pipeline environment and task requirements.

[0074] 5. Compared with existing multi-propeller or high-end ROV / AUV systems, the present application adopts a single-joint rudder + tail fin bionic structure, which can be realized with common visual modules and central control equipment, and has simple structure and low cost, which is conducive to popularization and large-scale application. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 is a schematic diagram of the overall hardware structure of the bionic robotic fish system of the present application;

[0076] Figure 2 is a software logic and data flow diagram of the control method of the present application;

[0077] Figure 3 is a schematic diagram of the principle of active angle domain adaptation in the segmented adaptive control law;

[0078] Figure 4 is a schematic diagram of the wing structure of the robotic fish;

[0079] Figure 5 is a schematic diagram of the exploded structure of the robotic fish shell;

[0080] Figure 6 is a physical assembly reference diagram;

[0081] Figure 7 is a top view schematic diagram of the angle relationship;

[0082] Figure 8 is a pipeline recognition effect diagram in a complex noise environment;

[0083] Figure 9 is a system frame rate comparison test result diagram. Detailed Implementation

[0084] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The following embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0085] Example 1: Bionic Robotic Fish for Pipeline Inspection

[0086] This embodiment of the biomimetic robotic fish for pipeline inspection consists of two parts: the robotic fish shell and the electronic component system. A physical reference of the assembled product is shown below. Figure 6 As shown.

[0087] like Figure 4 , 5 As shown, the robotic fish's outer shell consists of four waterproof structures: a transparent front cover 10, a main body 11, a top cover 12, and wings 13. The wings 13, symmetrically located on both sides of the main body, maintain the fish's balance after entering the water. The transparent front cover 10 serves as the optical window for the line-following camera and is removable to adjust its internal structure. The top cover 12 is removable for easy adjustment of its internal structure and can accommodate an LCD1602 color screen, switches, etc. The main body 11 houses the main control board, power supply module, line-following camera, and wiring system. A transparent slot (which can be sealed and waterproof) is designed at the top front of the main body to mount the recognition camera. A servo motor mounting base is located at the tail of the main body; the servo motor and the bionic tail fin are mechanically connected to form a propulsion mechanism. The wings 13 are symmetrically fixed on both sides of the main body to ensure the robotic fish's stability during swimming and prevent tilting.

[0088] like Figure 1 As shown, the electronic component system is installed on or inside the robotic fish shell 9, including: power supply module 1, host computer 2 (main control board), recognition camera 3, line-following camera 4, servo motor 5, bionic tail fin 6, display module 7, and external audio module 8; the robotic fish shell, servo motor 5, and bionic tail fin 6 form the body and actuator unit; the recognition camera 3 and line-following camera 4 form the measurement dual-camera subsystem; the host computer 2, display module 7, and external audio module 8 form the host processing and human-machine interaction unit; the power supply module 1, host computer 2 (main control board), recognition camera 3, line-following camera 4, servo motor 5, bionic tail fin 6, and display module 7 are integrated and installed inside the robotic fish shell 9.

[0089] The power supply module 1 respectively provides stable voltage power supply for the steering engine 5, the line patrol camera 3, the identification camera 4 and the upper computer 2; the identification camera 3 and the line patrol camera 4 are connected in communication with the upper computer 2 through a USB serial port (115200 bps, USB VCP mode), so as to realize information communication and power supply; the line patrol camera 4 (an integrated control unit) sends a PWM control signal to the steering engine 5 through a Dupont wire; the steering engine 5 is connected with the upper computer 2 through a Dupont wire, so as to complete power supply; the display module 7 is connected with the upper computer 2 through a Dupont wire, so as to complete power supply and communication; the external audio module 8 is wirelessly communicated with the upper computer 2 through 2.4G communication (including but not limited to Bluetooth); the steering engine 5 is mechanically connected with the bionic tail fin 6, so as to push the tail fin to swing; the steering engine 5 is arranged at a joint, and the bionic tail fin 6 is connected with the steering engine 5; the upper computer 2 can remotely communicate with a computer through VNC or SSH and the like, so as to realize data monitoring and debugging.

[0090] In the embodiment, the power supply module 1 adopts a Raspberry Pi UPS 5V uninterrupted power supply; the upper computer 2 (a main control board) adopts a Raspberry Pi 5 single-board computer; the identification camera 3 adopts an OpenMV H7 Plus, and is installed on the front upper part of the fish body to look down the pipeline; the line patrol camera 4 adopts an OpenMV H7 Plus, and is installed on the front of the fish body; the steering engine 5 adopts a Savox SC-1256; the steering engine has a drivable angle range of 60°-120°, so as to be compatible with different models and adjustment requirements; the bionic tail fin 6 is customized according to the specifications of the steering engine and is mechanically connected; the display module 7 adopts an LCD1602, which is used for displaying an identification state and counting; the external audio module 8 adopts a Bluetooth speaker, which is used for voice broadcast of identification events.

[0091] The control unit includes a state and control quantity unit, a heading control unit and a constraint and disturbance processing unit;

[0092] The state and control quantity unit is used for acquiring navigation state parameters obtained by regression or geometric calculation of the vision subsystem, and the state parameters at least include:

[0093] heading angle : the included angle between the navigation direction of the robotic fish and the reference axis;

[0094] deviation angle : the included angle between the navigation direction of the robotic fish and the pipeline, i.e. the regression line;

[0095] pipeline direction angle α: the included angle between the pipeline, i.e. the regression line, and the reference axis;

[0096] The above three relationships are shown in the following table. Figure 7

[0097] deviation distance : the distance from the image center point to the regression line;

[0098] ​And generate control variable according to state parameter : , wherein is control coefficient; the control variable is converted into rudder theoretical rotation angle through certain mapping relationship , that is , the specific relationship is defined by actual situation. Further drive the tail fin swing, to affect the linear speed v and angular velocity of the bionic robotic fish tail fin swing.

[0099] The heading control unit is used to control the robotic fish to travel along the pipe axis direction, and adaptively adjust the control strategy under different working conditions to realize the coordinated control of stable straight sailing, rapid steering response and event identification.

[0100] In the conventional heading / position control working condition, the heading control unit generates driving control variable to make the deviation angle , the deviation distance converge to 0, so that the robotic fish travels stably along the pipe axis direction; if the deviation angle , the deviation distance are less than the preset threshold, the generated control variable is close to zero, at this time only the original period and amplitude of the tail fin swing are maintained, at this time the vehicle can be regarded as being in stable straight sailing state.

[0101] In the turning working condition, when the absolute value of the deviation angle or the absolute value of the deviation distance exceeds the preset threshold, the heading control unit outputs rapid steering control variable to shorten the response time and avoid overshoot or off-line, and ensure smooth turning.

[0102] In the event identification working condition, the heading control unit maintains the above-mentioned closed-loop control process without disturbance, and guarantees the reliable counting of the system to the identified events.

[0103] The constraint and disturbance processing unit is configured to limit the following constraint conditions: visual noise (including reflection, turbid water, shadow), target fracture or shielding condition; angle domain limit of single rudder machine actuator; and action tempo / rate limit.

[0104] The identification subsystem further includes a debouncing state machine unit configured to send an event signal only once when the identification state jumps from "None" to "Detected" (i.e. rising edge of the detection signal); if the target is not detected for a preset reset time, the state is reset to "None".

[0105] Some key parameters (derived from the present implementation, which can be quantified as an example):

[0106] Actuator angle domain: , ;

[0107] Error threshold: , ; Pixels, Pixels;

[0108] Beat (speed control) (The smaller the value, the faster the tail fin swings. When turning, it can be shortened to 120-160 ms to increase the turning response.);

[0109] Visual threshold (example): connected domain brightness threshold (72, 255), area threshold ≥100 pixels; identify target LAB color threshold (0, 50, -100, 44, -128, 18), area limit 50-500 pixels;

[0110] De-bouncing recognition: host computer timeout 0.3-0.5s; recognition camera loss reset 450ms.

[0111] Embodiment 2: Control method of pipeline inspection bionic robotic fish based on dual vision system

[0112] Referring to Figure 2 , the embodiment includes the following steps:

[0113] Step 1: System initialization and calibration

[0114] In a preferred embodiment, after the system is powered on, the system automatically completes the following in sequence:

[0115] 1.1 Start the dual-camera subsystem, host computer and servo, complete power supply detection and communication connection confirmation.

[0116] 1.2 Load the de-distortion parameters and run the configuration file, including threshold (72, 255), communication baud rate 115200, return cycle 100 ms, reset time 450 ms.

[0117] 1.3 Perform servo zero calibration, detect limit angle ±90°.

[0118] 1.4 Establish a USB VCP communication channel and start the frame synchronization mechanism to make the two-way camera acquisition frame rate (30 fps) consistent with the control loop period.

[0119] Step 2: Dual-camera image acquisition and return

[0120] 2.1 The line patrol camera uses OpenMV H7 Plus, installed in the forward position of the fish body, and real-time acquisition of the front pipeline image.

[0121] 2.2 Recognition camera also uses OpenMV H7 Plus, installed above the fish body, collects overhead images for target recognition.

[0122] Image frames are attached with timestamps and frame numbers, and are synchronized back to the host computer (Raspberry Pi 4B, Raspberry Pi OS system) through the USB VCP interface.

[0123] Step 3: Image processing and steering control of line patrol camera (parallel with step 4), according to whether the effective pipe connected domain is detected, it is divided into two branches;

[0124] 3.1 When the effective pipe connected domain is detected, the following operations are performed:

[0125] 3.1.1 Thresholding processing: fixed threshold segmentation (72, 255) is performed on the collected image to separate the pipe from the background.

[0126] 3.1.2 Morphological operation: erosion (3x3 pixels) and dilation (3x3 pixels) are performed in sequence to remove noise and restore pipe connectivity.

[0127] 3.1.3 Connected domain extraction: select the largest connected domain as ROI.

[0128] 3.1.4 Straight line regression: least squares regression is performed in the ROI to fit the pipe center line, and the deviation distance (pixels) and the deviation angle (angle unit: °) are obtained.

[0129] 3.1.5 Working condition discrimination and control.

[0130] Further, in 3.1.5, the following are performed in sequence:

[0131] ① As shown in Figure 3 (Schematic diagram of the principle of active angle domain adaptation in piecewise adaptive control law), according to the deviation angle , the deviation distance , the different micro, medium and large deviation interval ranges, the control amount is obtained by linear combination, and then through the mapping relationship, such as (k is a constant), the theoretical steering angle of the steering engine is obtained.

[0132] ② The angle threshold of the medium deviation zone is set to ±45°, and the angle threshold of the large deviation zone is set to ±80°, and the is limited to obtain the actual steering angle of the steering engine .

[0133] ③ The servo action tempo servo_delay is set to 120-160 ms to realize the "back and forth swing - sampling" control rhythm;

[0134] ④ The control signal is output to the Savox SC-1256 servo, which drives the tail fin to realize posture correction.

[0135] 3.1.6 Data return: the deviation angle , the deviation distance , the control amount and working condition information are returned to the host computer synchronously.

[0136] 3.2 When no valid pipeline connection domain is detected, the system enters the line loss protection mode and performs the following operations:

[0137] 3.2.1 The servo performs a small left-right sweep to appropriately expand the ROI range and reduce the threshold value, and attempts to find the effective pipeline.

[0138] 3.2.2 If no pipeline is detected for 7 consecutive frames, report the "LostLine" state and reduce the propulsion frequency to pause the movement.

[0139] Step 4: Identify camera image processing and target detection (parallel with step 3)

[0140] 4.1 Perform distortion correction on the identified camera image.

[0141] 4.2 Search for circular contour candidate regions in the corrected image.

[0142] 4.3 Calculate the color threshold black_threshold = in the candidate region for identifying black marker points or oil stains.

[0143] 4.4 Perform area limiting screening on the target, and retain effective targets of 50-500 pixels.

[0144] 4.5 Use a multi-frame confirmation mechanism: only when the same target is detected for 3 consecutive frames is it determined to be "Detected".

[0145] 4.6 Vibration filtering mechanism processing: set the reset time to 450 ms, and trigger the event when the identification state changes from "None" to "Detected".

[0146] 4.7 Return the target position, identification state and quantity information to the host computer synchronously.

[0147] Step 5: Host computer event processing and feedback

[0148] 5.1 The host computer receives the identification and line tracking data, performs rising edge detection and secondary vibration filtering on the identification event.

[0149] 5.2 Count the number of times of statistical identification and record the event log.

[0150] 5.3 Display the live image from the line inspection camera on the host computer display interface, such as the notebook interface connected to the remote wireless network, and draw (optional content or not executed) the regression line, the center point of the robotic fish, the deviation distance, the vertical line, and the deviation angle, etc. information from the live image from the identification camera to achieve visual monitoring. At the same time, all information from the two cameras can be viewed on the host computer.

[0151] 5.4 Call the LCD1602 display screen to output the identification state and counting results, and simultaneously perform voice broadcast through the Bluetooth sound.

[0152] 5.5 When the identification link is abnormal, the line inspection closed loop remains independent operation to ensure the priority of the heading control.

[0153] Example 3: Simulation experiment and result analysis

[0154] To verify the advantages of the present application in light adaptability, line inspection regression robustness, visual stability in complex noise environment, and real-time performance after system decoupling, the group carried out simulation experiments and actual measurement verification under four typical scenes.

[0155] 1. Test of line inspection lightness adaptation range

[0156] This experiment aims to verify the environmental adaptability of the line inspection visual subsystem of the present application in a wide lightness range. The experiment was conducted in Zhangzhou, Fujian, under natural daylight conditions in November, from 6:30 in the morning to 18:00 in the evening, covering typical light scenes such as weak light in the morning, side light in the morning, strong light at noon, and twilight light.

[0157] During the test, the line inspection visual algorithm can stably obtain the connected domain of the pipeline by adjusting the threshold parameter, the image segmentation is clear, the noise area is controlled, and the straight line regression input is reliable. In the indoor environment at night, by sequentially turning on 1, 2, 3, and 4 ceiling lights of about 20W, the artificial lighting conditions from dim to high brightness are simulated, and adaptive threshold values can be obtained under different brightness levels, so that the pipeline remains stable and visible after binarization.

[0158] The results show that the line inspection visual system of the present application has good lightness adaptability under natural light and artificial light, and can stably output the effective pipeline region under wide dynamic light conditions, providing reliable input for subsequent regression and control.

[0159] 2. Test of line inspection regression robustness

[0160] ​​The experiment aims to verify the convergence ability of the invented line inspection algorithm and the segmented adaptive control law under different initial deviation conditions and the passability in the elbow pipe scene.

[0161] The experiment uses a white PVC pipe with a diameter of 80 mm to lay a rectangular closed pipe loop at the bottom of the pool, and the biomimetic robot fish is put into the water at different deviation angles of 0°-40° on the straight line segment to simulate the possible attitude deviation in the real environment.

[0162] The results show that within the test range, the robot fish can rely on the connected domain extraction, robust straight line regression and segmented adaptive control strategy to quickly realize the convergence of the deviation angle and the deviation distance, and automatically return to the direction of the pipe. Subsequently, at the 90° right-angle elbow pipe, the control system automatically cuts into the large deviation area angle domain and the shorter beat turning mode, enabling the robot fish to accurately complete the turning without interrupting the line inspection. Multiple tests have not appeared off-line, verifying the line inspection stability and control robustness of the invention under deviation working conditions and sharp turning scenes.

[0163] 3. Line inspection stability test in complex noise environment

[0164] The experiment aims to verify the recognition stability of the invented visual preprocessing and ROI adaptive selection mechanism in the presence of strong noise interference.

[0165] In the experiment, a small amount of lead blocks close to the color of the white pipe are thrown near the pipe, and a typical noise background is formed by using the reflective area at the bottom of the pool, the water surface light spot and the local shadow. The traditional visual method is prone to problems such as incomplete pipe connected domain, noise being mistakenly included in the ROI, and regression line jumping in such an environment. The invention can automatically exclude non-pipe interference targets such as lead blocks and reflective points through strategies such as connected domain area screening, morphological erosion-inflation to restore the main structure of the pipe, and quality evaluation of the candidate straight line regression area, so that the extracted ROI is stable and concentrated in the real pipe area. Even in the case of partial pipe profile loss, the main shape can still be restored and regression can be completed.

[0166] The experimental results show that the invention can still maintain stable regression and smooth control output in a complex noise environment, and there is no drift and off-line during the entire line inspection process, significantly improving the visual robustness and environmental adaptability. The image processing effect of the experiment is shown in Figure 8 .

[0167] As Figure 8 can be seen, the pipe profile is complete, the main body is clearly identified, the regression line (green line) is accurately drawn, and the red rectangular frame marks the target area, and the interference noise on both sides is completely isolated and excluded.

[0168] 4. Function decoupling test

[0169] The experiment aims to verify the real-time advantage of the system architecture based on dual vision decoupling and control, and the identification function moving down compared with the traditional single-board central processing scheme.

[0170] The comparative scheme uses Raspberry Pi 5 to run the line inspection and identification two OpenCV vision tasks at the same time under QQVGA resolution, and undertakes rudder control and display output, and the actual frame rate is usually 10-15 FPS (frames per second), the control loop is frequently delayed due to image processing blocking, especially in right-angle turning or large deviation correction, and line deviation is prone to occur due to control lag. In the present application, the line inspection camera independently completes thresholding, morphological processing, connected domain extraction, regression fitting and rudder control amount calculation on the local end, the identification camera is only responsible for event detection and reporting, and the upper computer is only used for data summarization and human-computer interaction, so that each task no longer competes for the same computing power source.

[0171] In actual measurement, the present application can stably maintain a line inspection processing capacity of about 45 FPS or more under the same QQVGA resolution, the rudder update rhythm is stable and the response is timely, the identification link exception does not affect the line inspection closed loop operation, the overall system delay is significantly reduced, and the real-time performance and reliability are obviously better than those of the traditional scheme. The experimental frame rate is as shown in Figure 9 .

[0172] In summary, the four types of experiments of the embodiment verify the effectiveness of the present application from four aspects of light adaptability, control robustness under deviation working condition, vision stability under complex noise scene and real-time performance of the system architecture. The results show that the present application can realize stable, fast, low-delay and non-easy-to-deviate line inspection control in various typical pipeline inspection environments, and maintain good vision robustness in complex scenes, which fully proves that the present application scheme can realize the technical effects described in the specification.

[0173] The technical innovations of the present application include:

[0174] 1. Dual vision decoupling and machine vision processing and control function moving down: the forward line inspection camera completes target region extraction processing and control amount calculation on the local end, and directly drives the rudder closed loop; the overhead identification camera only detects target events and reports; the upper computer is only used for statistics and human-computer interaction. The control loop delay and link coupling are significantly reduced, the abnormal isolation is strong (the identification link exception does not affect the heading closed loop), and the track is more stable.

[0175] 2. Three-section type segmented adaptive angular domain control: according to , The control is divided into three sections of small deviation (straight line stability) / medium deviation (adjustment) / large deviation (turning), and different rudder angle domain limits are given. When the deviation is small, the vibration is suppressed and the steady-state error is small; when the deviation is medium, the deviation is corrected gradually and smoothly, the execution is accurate, and oscillation is avoided; when the deviation is large / acute, the angular velocity is sufficient, the deviation is corrected quickly without overshoot.

[0176] 3. Cooperative scheduling of error-angle-tick: when switching sections, the angle domain limit and the control tick (servo_delay) are adjusted synchronously; when entering the large deviation area, the angle domain is expanded and the tick is shortened; when returning to the small deviation area, the angle domain is converged and the tick is extended; the present application can be combined with angular velocity / acceleration limits. Adaptive trade-off between "response speed-stability" is made to reduce oscillation and actuator saturation.

[0177] 4. Dynamic ROI and confidence weighted straight line regression: adaptively adjust the ROI based on the connected domain size and quality indicators, automatically exclude unreasonable interference areas; map the regression residual / confidence (such as fitting score) to the control weight or the correction amount of the segment switching threshold. In the present application, the error input is more reliable in noise scenes such as reflection and local fracture, reducing misleading control.

[0178] 5. Heading maintenance and decay strategy for short-term line loss: when there is no valid line feature for N consecutive frames, maintain the last control output and decay exponentially until the feature is recovered or a safe mode is entered. The present application does not interrupt the route under strong occlusion and short-term disturbance of reflection, reducing the probability of line loss; if it is out of line due to force majeure, it can also stop in time to prevent accidental collision.

[0179] 6. Hierarchical fault tolerance and self-repair mechanism: fault detection and mode switching are set for identification link, communication link, vision quality and control output; identify abnormal output fault location and problem in host computer (main control board); line patrol fails safety shutdown and return alarm information. The robustness and safety of the present application system are significantly improved, meeting the availability requirements of actual patrol.

[0180] 7. Multi-feature fusion target recognition (geometry + color + size) in the identification subsystem: in the identification subsystem, joint determination is made through contour geometry detection, LAB color statistics, size determination, etc. The present application reduces false detection in visual noise such as reflection, shadow, etc. and water background, and the event triggering is more reliable.

[0181] 8、Parameterization and adjustable implementation: the key control parameters in the application, including but not limited to the steering action beat and angular domain limiting parameter, the deviation angle threshold and deviation distance threshold, the straight line regression quality threshold, the visual brightness threshold, the connected domain area threshold, the identification event reset time, the PID controller channel proportional / integral / differential gain, the identification target area limit, the color statistics threshold, are parameterized definition. These parameters can be flexibly adjusted according to the hardware platform performance, environmental conditions (such as water area illumination, flow rate) or task requirements. Parameterized design makes the application have high portability and adaptability. In different application scenarios, only parameter calibration is needed to complete system optimization, without changing the overall control architecture, thereby reducing the deployment difficulty and improving the engineering feasibility.

[0182] The above merely describes the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A pipeline inspection bionic robotic fish based on a dual vision system, characterized in that, include: The body and actuator unit includes a single-joint bionic fish shell, a servo motor located at the joint, and a tail fin connected to the servo motor; The measurement dual-camera subsystem includes a line-following subsystem and an identification subsystem; the line-following subsystem is equipped with a line-following camera that is angled downwards and aligned with the front of the pipeline, used to acquire images and perform preprocessing and linear regression, and to calculate and output the deviation angle θ and deviation distance ρ of the robotic fish relative to the central axis of the pipeline in real time. The identification subsystem is equipped with an identification camera that looks down at the external area of ​​the pipeline, used to detect specific targets on the outside of the pipeline, and send the identification signal "Detected" or "None" to the host computer; The host processing and human-machine interaction unit communicates with the dual-camera subsystem through a communication interface. It is used to receive the recognition signals from the recognition subsystem and perform counting, visualization output and voice broadcasting. It also receives the real-time parameter information of the patrol subsystem and displays it visually, while displaying real-time captured images. It does not participate in real-time steering control. The control unit, integrated into the line-following subsystem, is used to receive the deviation angle θ and deviation distance ρ, calculate the control quantity u of the servo motor according to the piecewise adaptive control law, and then map it to the theoretical rotation angle δ of the servo motor. The power supply unit is used to provide uninterrupted constant voltage input to the servo motor, the dual-camera measurement subsystem, and the host processing and human-machine interaction unit.

2. The bionic robotic fish for pipeline inspection based on the dual vision system according to claim 1, characterized in that, The servo motor has a drive angle range of 60° to 120°. 3.The pipeline inspection bionic robotic fish based on binocular vision system of claim 1, wherein, The identification subsystem also includes a de-jitter state machine unit, which is configured to send an event signal only once when the identification state changes from "None" to "Detected"; if no target is detected for a preset reset time, the state is reset to "None".

4. The biomimetic robotic fish for pipeline inspection based on a dual-vision system according to claim 1, characterized in that, The control unit includes a status and control quantity unit, a heading control unit, and a constraint and disturbance processing unit; The status and control unit is used to acquire navigation status parameters, which include at least the heading angle ψ, deviation angle θ, pipeline direction angle α, and deviation distance ρ, and to generate a control quantity u based on the status parameters. The heading control unit is used to control the robotic fish to travel along the tube axis. Under normal heading / position control conditions, it generates drive control quantities to make the deviation angle θ and deviation distance ρ converge to 0. Under turning conditions, it outputs rapid steering control quantities. Under event recognition conditions, it maintains the closed-loop control process undisturbed. The constraint and disturbance processing unit is used to limit visual noise, target breakage or obstruction, angular domain limitation of single servo actuator, and action cycle / rate limitation.

5. The biomimetic robotic fish for pipeline inspection based on a dual-vision system according to claim 4, characterized in that, In the state and control unit, the heading angle ψ, the deviation angle θ, and the pipeline direction angle α satisfy the relationship ψ=θ+α.

6. The biomimetic robotic fish for pipeline inspection based on a dual-vision system according to claim 1, characterized in that, The display module of the host processing and human-computer interaction unit is an LCD1602 display screen with I²C communication, and the audio output is realized through a Bluetooth speaker.

7. A control method for a biomimetic robotic fish for pipeline inspection based on a dual-vision system as described in any one of claims 1 to 6, characterized in that, Includes the following steps: Step 1: System initialization and calibration, including hardware initialization, camera calibration and parameter loading, servo self-test and zeroing, communication interface initialization, and system synchronization and preparation; Step 2: The dual-camera subsystem performs image acquisition and transmission. The line-following camera acquires real-time images of the pipeline in front of the robot fish, and the recognition camera acquires real-time top-view images of the pipeline. All acquired images, along with timestamps and frame numbers, are synchronously transmitted back to the host computer. Step 3: The built-in program of the line-following camera completes the line-following image processing and servo control. Depending on whether a valid pipeline connection area is detected, it performs corresponding operations to achieve automatic heading correction. Step 4: The camera's built-in program completes image processing and target detection, and sends the detection information back to the host computer; Step 5: The host computer completes the recognition event processing and feedback, line-following feedback and image feedback.

8. The control method for a biomimetic robotic fish for pipeline inspection based on a dual vision system according to claim 7, characterized in that, In step three, when a valid pipe connectivity is detected, image thresholding, morphological operations, connectivity extraction, linear regression and deviation calculation, working condition discrimination and motion control, and data transmission are performed sequentially. When no valid pipeline connectivity is detected, the line loss detection and protection mechanism is triggered.

9. The control method for a biomimetic robotic fish for pipeline inspection based on a dual vision system according to claim 7, characterized in that, In step four, distortion correction, target contour region search, color statistics and target determination, area limiting processing, status determination and data feedback are performed sequentially.

10. The control method for a biomimetic robotic fish for pipeline inspection based on a dual vision system according to claim 8, characterized in that, The segmented adaptive control law for condition discrimination and motion control adopts a comprehensive mechanism of dual-channel fusion, three-segment adaptive angular domain, beat speed limiting, saturation anti-shake, and safe return, including dual-channel error synthesis, active angular domain adaptation, beat / rate limiting, saturation and anti-shake mechanism, and return strategy.

Images