A confidence-based adaptive visual servoing system for barnacle cleaning

Abstract

The application relates to the technical field of underwater operation robot control, and discloses a barnacle cleaning visual servo system based on confidence self-adaption, which comprises a sensing device, an edge device, a control device and a chassis. The edge device is allocated with shared memory, a sensing node executes target detection to output target boundary box coordinates and detection confidence, and data is asynchronously written into the shared memory through double-buffered data blocks and an atomic state flag bit; a decision node non-blockingly extracts sensing data to calculate spatial deviation, and according to the detection confidence, carries out non-linear interpolation mapping on linear velocity and angular velocity gain to generate chassis motion control instructions. The decision node is internally provided with an anti-lock timing state machine, and when the target is continuously lost, the chassis is triggered to spin search and a global image sensing mode is triggered. Through asynchronous data interaction, the application avoids bottom layer control delay, uses confidence self-adaption to adjust and prevent motion concussion, and improves the reliability of system continuous operation.

Description

Technical Field

[0001] This invention relates to the field of underwater robot control technology, specifically a barnacle cleaning visual servo system based on confidence-adaptive technology. Background Technology

[0002] Marine organisms attaching to ship hulls (such as barnacles, mussels, sea squirts, and algae) is a long-standing problem facing the global shipping industry, and underwater robots are currently commonly used for cleaning. However, existing ship hull cleaning robots still have several technical shortcomings in practical operations.

[0003] In terms of sensing operations, most existing products still rely on operators to observe low-quality underwater videos for manual judgment and control, which can easily lead to fatigue and missed detections during prolonged operation. Although some products are equipped with cameras, they are usually only used for monitoring and observation, and do not incorporate deep learning technology to achieve real-time target detection. At the same time, due to physical interferences such as uneven lighting and low contrast in the underwater environment, directly applying general visual algorithms often fails to obtain stable detection results.

[0004] In terms of control decision-making, existing equipment lacks an effective closed-loop linkage between visual recognition results and underlying motion control. Conventional methods mostly use fixed-gain proportional controllers for position tracking, failing to incorporate the key variable of target detection confidence into the control considerations. When underwater lighting is poor or the target is obstructed, resulting in low detection confidence, the robot still approaches the target at the fixed-gain speed, which can easily lead to incomplete cleaning or accidental damage to the ship's paint.

[0005] Furthermore, the existing system's hardware and software architecture is ill-suited to the demands of underwater visual servoing. Target detection algorithms are computationally intensive, and their frame rates on edge computing devices are typically low (e.g., 5-8Hz). Because existing systems often lack effective thread decoupling between visual detection and underlying control, blocking call methods severely slow down the high-frequency (e.g., above 30Hz) underlying control loops, resulting in lag in chassis control response. Simultaneously, existing equipment lacks a smooth transition mechanism for human-machine interaction control, preventing seamless manual takeover in automatic operation modes. In special circumstances, the system often has to be interrupted by triggering an emergency stop, directly impacting the continuity of operation under complex conditions. Summary of the Invention

[0006] The technical problem solved by this invention is that, when existing underwater robots perform attachment cleaning tasks, there is a frequency difference between the inference time of the visual perception module and the operation cycle of the underlying control module. Using synchronous communication can easily lead to system blockage and control delay. At the same time, the fuzzy target features in the underwater environment can easily cause fluctuations in detection confidence. Relying on a single coordinate feedback for control can easily cause chassis movement vibration or target loss. In addition, existing systems lack a relocation mechanism after target loss, which can easily lead to operation interruption or system deadlock.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a barnacle cleaning visual servo system based on confidence-adaptive architecture, comprising: a sensing device, an edge device, a control device, and a chassis; the sensing device establishes a video data stream transmission connection with the edge device, the sensing device being used to acquire external environmental image data and send it to the edge device; the edge device establishes an electrical connection with the control device via a serial communication bus, and the signal output terminal of the control device is connected to the chassis drive motor; the edge device is internally configured with middleware, and the system software layer includes sensing nodes, decision nodes, and communication nodes, and shared memory is allocated in the edge device's memory space; both the sensing nodes and decision nodes establish a memory read-write mapping relationship with the shared memory to complete asynchronous data interaction under different running thread frequencies; the sensing node acquires... The system acquires external environment image data and executes target detection algorithm inference, outputting the target bounding box center coordinates and detection confidence score, and writes the extracted target bounding box center coordinates and detection confidence score into shared memory. The decision node reads the latest target bounding box center coordinates and detection confidence score from the shared memory in a non-blocking manner at a fixed control loop frequency. The decision node calculates the lateral spatial deviation based on the read target bounding box center coordinates and image center coordinates, and performs interpolation mapping calculations on the preset starting linear velocity and preset steering gain coefficient according to the detection confidence score value, generating linear velocity control commands and angular velocity control commands. The communication node receives the linear velocity control commands and angular velocity control commands and sends them to the control device, which outputs pulse width modulation signals to the chassis to control the chassis movement state.

[0008] Furthermore, the shared memory is physically divided into two independent data blocks of identical size, each with an independent one-byte unsigned integer atomic status flag. The perception node, based on the arithmetic result of modulo-2 on the incrementally increasing version sequence number integer value, alternately switches between the two independent data blocks to perform data overwrite operations. Each write includes the target bounding box center coordinates, detection confidence, and absolute physical timestamp, and uses atomic instructions to update the atomic status flag to record the physical index number of the latest independent data block. At the start of the control cycle, the decision node reads the atomic status flag and locates the corresponding independent data block based on the recorded physical index number to extract visual perception data. When the decision node determines that the difference between the current system absolute physical time and the absolute physical timestamp exceeds the preset physical upper limit of perception delay, it determines that the data is expired and invalidates, forcibly resetting the target detection confidence variable and lateral spatial deviation in the internal computing environment to zero.

[0009] Furthermore, an independent reverse control block is allocated within the shared memory. When the decision node determines that barnacle targets are continuously detected and the detection confidence reaches a stable threshold, it generates a perception mode control flag in a logically true state and writes it into the reverse control block. Before each inference cycle, the perception node reads the perception mode control flag in the reverse control block. If it is in a logically false state, it maintains the global full-size image input mode. If it is in a logically true state, it switches to the local region of interest mode. Using the center point of the target bounding box detected in the previous frame as a reference and combining the size of the bounding box in the previous frame, it expands outward and performs numerical clamping and cropping operations on the external environment image data to extract local image blocks. After scaling the local image blocks to a fixed input dimension, the target detection algorithm inference is performed.

[0010] In a preferred embodiment of the present invention, in the local region of interest mode, the perception node performs a geometric affine transformation mapping the coordinates of the center point of the target bounding box in the relative coordinate system of the inference output back to the global coordinate system. The perception node subtracts the unidirectional grayscale fill pixel width introduced in the horizontal or vertical direction when the edge grayscale fill strategy is executed in the preprocessing stage from the coordinates of the center point of the target bounding box in the relative coordinate system. Combined with the inverse scaling restoration ratio value of the local image block to the fixed input dimension and the positioning offset of the upper left corner reference point of the local image block in the original global image pixel coordinate system, the absolute x-coordinate and absolute y-coordinate of the center point of the target bounding box after mapping in the original global image pixel coordinate system are calculated and written to shared memory after performing a rounding operation.

[0011] Furthermore, the nonlinear variable mapping calculation process for generating linear velocity control commands and angular velocity control commands based on the detection confidence value by the decision node includes: when the detection confidence is between the effective lower confidence threshold and the confidence saturation upper confidence threshold, the decision node calculates the desired translational linear velocity as the linear velocity control command by positively proportionally amplifying the difference between the detection confidence and the effective lower confidence threshold based on the preset physical lower and upper limits of linear velocity; the decision node weights the detection confidence based on the adjustment sensitivity factor and subtracts it from the unit value, multiplying the difference result by the basic proportional gain constant to obtain the dynamic angular velocity proportional control gain coefficient; the decision node multiplies the normalized lateral spatial deviation, the dynamic angular velocity proportional control gain coefficient, and the preset sign mapping coefficient together to calculate the desired rotational angular velocity as the angular velocity control command.

[0012] Furthermore, the decision node integrates an anti-deadlock timing state machine and allocates a continuous frame loss counter in system memory. When the visual perception data read by the decision node indicates no target and the version sequence number of the visual perception data is incremented, the continuous frame loss counter performs an accumulation operation, and the decision node continuously outputs linear velocity control commands and angular velocity control commands with zero values. When the value of the continuous frame loss counter reaches the preset loss tolerance threshold, the anti-deadlock timing state machine triggers a reverse transition. The decision node rewrites the perception mode control flag to a logical false value state to trigger the perception node to restore the global full-size image input mode. At the same time, the decision node forces the bypass target undetected parameter to return to zero safety isolation mechanism and sends a preset low-speed in-situ spin search angular velocity command to the lower layer to control the chassis to enter the active seek rotation state.

[0013] Furthermore, when the sensing node re-outputs a detection confidence score greater than the effective lower limit threshold in the global full-size image input mode, the anti-deadlock timing state machine exits the active search rotation state, the decision node clears the continuously lost frame counter, resumes the interpolation mapping calculation, and resumes closed-loop tracking based on the latest extracted lateral spatial deviation.

[0014] Furthermore, the middleware configures an independent circular message queue for each data topic establishing data exchange. The maximum physical length of the independent circular message queue is determined by multiplying the average publishing frequency of data packets pushed into the middleware network buffer by the publishing node, the maximum execution time of the subscribing node to extract data packets from the network buffer and complete a single business logic calculation, and a preset dimensionless safety redundancy coefficient. When the independent circular message queue reaches its maximum physical length, the middleware slides the read head pointer of the independent circular message queue backward by the physical memory address span of one data frame and overwrites the historical data frame with the oldest timestamp, ensuring that the subscribing node prioritizes processing the sensor data with the latest timestamp.

[0015] Furthermore, the control equipment internally runs inverse kinematics calculation logic, receives the desired translational linear velocity and desired rotational angular velocity, and combines them with the physical span between the equivalent contact points of the drive wheels on the left and right sides of the chassis to calculate the desired target linear velocity of the left track and the desired target linear velocity of the right track. Based on the physical mapping relationship of the bus DC voltage, the control equipment converts the desired target linear velocities of the left track and the right track into the duty cycle percentage values ​​of the pulse width modulation signal of the hardware timer channel, and outputs them to the motor drive bridge circuit built into the chassis to drive the physical motor.

[0016] Furthermore, the communication node maintains the listening state of the network socket port in an independent background thread and allocates a Boolean address block in memory as a takeover status flag variable. When the communication node receives a manual control data packet sent by the external host computer, it rewrites the takeover status flag variable to a logical true value, forcibly blocks and discards the automatic control data stream transmitted by the middleware, and parses and distributes the manual control data packet to the control device. When the external host computer disconnects, the communication node rewrites the takeover status flag variable to a logical false value, forcibly sends a single zero-speed safety data packet to the control device, and issues a state machine reset event to the decision node to clear the historical control state cache.

[0017] The sensing device acquires external environment image data and sends it to the sensing node. The sensing node executes the target detection algorithm to infer and extract the center coordinates of the target bounding box and the detection confidence score. The perception node asynchronously writes the center coordinates of the target bounding box and the detection confidence score into shared memory through a double buffering mechanism. The decision node extracts the latest perception data from shared memory in a non-blocking manner at a fixed frequency and calculates the lateral spatial deviation between the center of the target bounding box and the center of the image. The decision node performs nonlinear interpolation mapping calculation on the linear velocity and angular velocity ratio gain based on the extracted detection confidence value, generates chassis motion control commands and issues them to drive the chassis to perform adaptive servo tracking of the target.

[0018] This invention provides a vision servoing system for barnacle cleaning based on confidence-adaptive principles. It offers the following advantages:

[0019] 1. This invention allocates shared memory in the edge device's memory space, divides it into two independent data blocks, and attaches atomic state flags, enabling the sensing node and decision node to alternately perform asynchronous read and write operations between the two data blocks. This non-blocking data interaction method decouples the time-consuming visual target detection algorithm from the high-frequency chassis-level control logic, solving the technical problem of mismatch between underwater visual inference frequency and motion control cycle, avoiding cross-thread data waiting and interlocking deadlocks between processes, and effectively reducing system servo control latency.

[0020] 2. This invention incorporates visual detection confidence into the interpolation mapping calculation of chassis motion control, dynamically adjusting linear velocity and angular velocity control commands based on the confidence value. When target features are clear and confidence is high, the system outputs a higher linear velocity to improve approach efficiency; when environmental interference causes confidence to decrease, the system automatically reduces translational linear velocity and correspondingly increases the dynamic angular velocity proportional control gain. This mechanism allows the chassis to prioritize course adjustment during target blurring to maintain the target in the image center, avoiding chassis motion oscillations and target loss problems caused by traditional fixed-gain control methods in complex underwater environments.

[0021] 3. This invention integrates an anti-deadlock timing state machine and a continuous frame loss counter within the decision node, and uses a sensing mode control flag to achieve closed-loop linkage between the image detection range and the chassis motion state. When a target is stably detected, the sensing node switches to a local region of interest mode to reduce the computational load of the algorithm inference; when the target is continuously undetected and the count reaches a preset threshold, the anti-deadlock timing state machine controls the sensing node to resume the global image input mode, and simultaneously sends a preset in-situ spin command to the control device. This design enables the system to actively perform a search action after the target is lost and automatically resume closed-loop tracking after the target is recaptured, avoiding unexpected interruptions in the operation process. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall system architecture of the present invention; Figure 2 This is the main flowchart of the visual servoing method of the present invention; Figure 3 This is a schematic diagram of the underlying device control and communication data flow process of the present invention; Figure 4 This is a schematic diagram of the interaction process between the core algorithm node and shared memory in this invention.

[0023] Among them, 110 is a sensing device; 120 is an edge device; 121 is a sensing node; 122 is a decision node; 123 is a communication node; 124 is a shared memory; 130 is a control device; 140 is a chassis; and 150 is middleware. Detailed Implementation

[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] See attached document Figure 1 The present invention provides a barnacle cleaning visual servo system based on confidence-adaptive design, which may include: a sensing device 110, an edge device 120, a control device 130, and a chassis 140.

[0026] Sensing device 110 establishes a video data stream transmission connection with edge device 120. Sensing device 110 is used to acquire external environmental image data and send it to edge device 120. Edge device 120 establishes an electrical connection with control device 130 through a serial communication bus. The signal output terminal of control device 130 is connected to the drive motor of chassis 140, and control device 130 is used to output pulse width modulation signals to chassis 140 to control the movement state of chassis 140.

[0027] The edge device 120 is internally configured with middleware 150. Middleware 150 is used to integrate and manage the various independently running software function nodes. The system software layer mainly includes sensing node 121, decision node 122, and communication node 123.

[0028] The edge device 120 has a shared memory 124 allocated in its memory space. Both the sensing node 121 and the decision node 122 establish a memory read-write mapping relationship with the shared memory 124. The sensing node 121 and the decision node 122 complete asynchronous data interaction at different running thread frequencies through the shared memory 124.

[0029] Perception node 121 continuously acquires image data collected by perception device 110. Perception node 121 performs target detection algorithm inference on the image data and outputs the center coordinates of the target bounding box and the detection confidence score. Perception node 121 writes the extracted target center coordinates and detection confidence score into shared memory 124.

[0030] Decision node 122 reads the latest target center coordinates and detection confidence from shared memory 124 in a non-blocking manner at a fixed control loop frequency. Decision node 122 calculates the lateral spatial deviation based on the read target center coordinates and image center coordinates.

[0031] Decision node 122 performs interpolation mapping calculations on the preset starting linear speed and preset steering gain coefficient based on the detection confidence value, generating linear speed control commands and angular speed control commands. Decision node 122 sends the linear speed control commands and angular speed control commands to communication node 123. Communication node 123 encapsulates the control commands into serial protocol data packets supported by control device 130 and sends them to control device 130.

[0032] Decision node 122 integrates a sequential state machine. Decision node 122 compares the currently read detection confidence level with a preset lower threshold, and updates the built-in continuous lost frame counter based on the comparison result.

[0033] Decision node 122 generates a corresponding perception mode control flag based on the current value of the consecutively lost frame counter. Decision node 122 feeds back the perception mode control flag to perception node 121. Perception node 121 parses the perception mode control flag and switches between the global inference state of the full-size original image and the local region cropping inference state based on the target's historical coordinates.

[0034] See attached document Figure 2 This invention provides a confidence-adaptive visual servoing system for barnacle cleaning, comprising the following steps: S10, Initialize device communication connection and memory mapping allocation, and establish an asynchronous decoupling communication mechanism based on shared memory 124 between the sensing node 121 and the decision node 122 with different thread frequencies. S20: Acquire external environment image data to perform target detection inference, extract target features and write them to shared memory 124, and perform mode switching between global frame inference and local frame cropping inference based on the perception mode control flag bit fed back by decision node 122. S30 calculates the spatial deviation based on the target features read from shared memory 124, determines the target recognition status according to the detection confidence level, triggers the anti-deadlock timing state machine to send the perception mode control command back to the perception node 121, and performs adaptive interpolation mapping calculation on the linear velocity based on the confidence level value. S40 calculates the variable gain coefficient based on the detection confidence level to generate an angular velocity control command, sends the control quantity and converts it into a pulse width modulation signal to drive the chassis 140 to perform tracking actions.

[0035] See attached document Figure 3 The present invention provides a barnacle cleaning visual servo system based on confidence-adaptive technology. The sensing device 110 establishes a data transmission connection with the edge device 120 through a physical bus interface. The sensing device 110 is responsible for acquiring optical images of the underwater operation area and converting the captured external underwater environmental optical signals into a digital image matrix and transmitting them to the edge device 120.

[0036] The optical lens assembly and internal photosensitive component packaging structure of the sensing device 110 can be implemented using the conventional sealing manufacturing process of existing underwater high-resolution serial bus cameras. During the initialization configuration phase, the sensing device 110 presets the pixel matrix size of the global original image according to the low-level hardware configuration instructions issued by the edge device 120. The pixel matrix size of the global original image is limited by the number of pixel columns in the horizontal direction and the number of pixel rows in the vertical direction. The sensing device 110 can push the complete pixel matrix data packet onto the bus at a fixed hardware sampling frequency to provide a basic data source for subsequent visual feature extraction and inference algorithms at the physical link layer.

[0037] In the specific data transmission implementation process, the workflow of sensing device 110 includes the following sub-steps: S111, after the system is powered on, sensing device 110 completes the internal register initialization configuration according to the underlying driver protocol, activates the photosensitive array, and activates the corresponding photosensitive area according to the preset full-size resolution parameters; S112, sensing device 110 performs analog-to-digital conversion on the analog electrical signal captured by the photoelectric conversion array to generate a discrete digital image matrix containing color channel feature parameters; S113, sensing device 110 splits the discrete digital image matrix into a continuous data packet sequence according to the Universal Serial Bus protocol specification, and continuously sends the data packet sequence to the communication interface buffer register corresponding to edge device 120 through the underlying hardware data link.

[0038] Typically, to establish an accurate spatial coordinate reference system and define the data structure of the environment image in subsequent calculations, the environment image frame is abstractly defined in memory as a multi-dimensional digital matrix. The spatial distribution and numerical mapping relationship of the pixel array of the multi-dimensional digital matrix are expressed by the following formula. In the formula, This represents the global environment image matrix output by sensing device 110. This represents the feature value of a pixel at a specific spatial coordinate position in the image matrix. Specifically, the feature value is an integer value of the brightness of each color channel, and the value range is set between 0 and 255. This represents the total number of columns of pixels in the horizontal direction of the image matrix. The total number of rows and columns of pixels in the vertical direction of the image matrix is ​​represented by the total number of rows and columns of pixels. The specific values ​​of these values ​​are determined by the physical specifications of the photosensitive components of the sensing device 110.

[0039] The control device 130 is configured with a microprocessor core, establishing an electrical connection with the edge device 120 via a serial communication bus, and a high-frequency electrical connection with the power input terminal of the chassis 140. The microcontroller hardware timer channel and peripheral isolated drive circuit structure contained within the control device 130 can be implemented according to existing conventional electrical control design specifications for underwater robots.

[0040] In the specific control signal conversion and drive implementation process, the workflow of control device 130 includes the following sub-steps: S131, control device 130 receives a specific serial protocol data frame sent by communication node 123 and performs protocol unpacking, extracts the signed integer load data corresponding to the expected translation linear velocity command and expected rotation angular velocity command generated by the top-level decision algorithm, and strictly performs a division inverse restoration operation according to the physical scaling multiplier preset at the communication node 123 end, restoring it to a floating-point expected translation linear velocity command and expected rotation angular velocity command with real physical dimensions; S132 Based on the built-in chassis inverse kinematics model, the control device 130 calculates and converts the desired translational linear velocity command and the desired rotational angular velocity command into the desired target linear velocities corresponding to the left and right tracks of the chassis 140, and further calculates the duty cycle percentage value of the corresponding hardware timer channel pulse width modulation signal according to the physical mapping relationship; S133, the control device 130 outputs the generated pulse width modulation signal and direction control level to the motor drive bridge circuit built into the chassis 140 through the hardware timer channel, and controls the high-frequency on / off state of the power switching transistor inside the motor drive bridge circuit.

[0041] To achieve the decoupling mapping from the desired control quantity at the top level to the speeds of the two actuators, the inverse kinematics solution logic of the control device 130 follows the formula below: In the formula, This represents the expected translational linear velocity of the chassis at 140. This represents the desired rotational angular velocity of chassis 140 (output by decision node 122). This represents the desired target linear velocity of the right track. This represents the desired target linear velocity of the left track. This indicates the physical distance between the equivalent contact points of the drive wheels on both sides of the chassis (140).

[0042] After acquiring the desired target linear velocity on both sides, the average effective voltage across the stator winding of the drive motor output by the control device 130 to the chassis 140 is related to the duty cycle of the pulse width modulation signal output by the timer channel of the underlying microcontroller of the control device 130 according to the following formula: In the formula, This represents the average effective voltage across the stator windings of the chassis 140 drive motor. This indicates the DC bus voltage of the chassis 140 power supply system. The specific value of the DC bus voltage is determined by the rated voltage parameters of the battery pack configured in the system. This indicates the percentage value of the duty cycle of the pulse width modulation signal output from the hardware timer channel of controller device 130. The duty cycle can be adjusted... The control device 130 drives the independent drive motors on both sides of the chassis 140 to generate corresponding electromagnetic torque, thereby continuously responding to the tracking commands issued by the system level to complete the spatial attitude adjustment closed loop of the actuator.

[0043] To achieve the final physical response of the actuator, the chassis 140 serves as the physical platform for the vision servo system and is in direct contact with the external working surface. The drive bridge circuit inside the chassis 140 is electrically connected to the signal output port of the control device 130 through a physical wiring harness.

[0044] The metal hull sealing structure and track tensioning mechanism of chassis 140 can be implemented using existing conventional pressure-resistant and waterproof manufacturing processes for underwater robots. The chassis 140 hull is equipped with a left drive motor and a right drive motor symmetrically installed inside. The main shafts of the left drive motor and the right drive motor are rigidly connected to the external track drive wheel through a reduction gearbox. The left drive motor and the right drive motor work together to ultimately convert the digital control commands into a displacement response in physical space.

[0045] In the specific mechanical response and motion execution process, the workflow of chassis 140 includes the following sub-steps: S141, the drive bridge circuit inside chassis 140 receives the pulse width modulation signal continuously sent by control device 130, and controls the on and off states of the power switching transistor inside the drive bridge circuit according to the duty cycle percentage value of the pulse width modulation signal, converting the energy of the external DC power supply into an average excitation current flowing through the stator windings of the left drive motor and the stator windings of the right drive motor; S142, the rotors of the left drive motor and the right drive motor move in the left drive... The stator windings of the drive motor and the stator windings of the right drive motor generate electromagnetic torque under the action of the magnetic field. The electromagnetic torque output by the left and right drive motors is amplified by the reduction gearbox and the speed is reduced before being transmitted to the track drive wheels on both sides of the chassis 140 to overcome the fluid resistance of the underwater environment and the frictional resistance of the mechanical transmission system. S143, the track drive wheels on both sides of the chassis 140 drive the track to move relative to the underwater working surface. The macroscopic longitudinal traction force and lateral yaw torque are generated by the physical adhesion between the external tooth structure of the track and the underwater working surface.

[0046] The differential steering trajectory response of chassis 140 in physical space depends on the speed difference between the left and right drive motors. To accurately describe the spatial motion attitude of chassis 140 on the two-dimensional working reference plane, a kinematic model is typically used to map the local velocity variables of the dual drive wheels to the global rotational state of the chassis. The instantaneous turning radius and instantaneous angular velocity response logic of chassis 140 are expressed by the following formula: In the formula, This represents the instantaneous turning radius of chassis 140 on the two-dimensional working reference plane. This indicates the actual linear speed of the right track of chassis 140. This represents the actual linear velocity of the left track of chassis 140. The actual linear velocities of the right and left tracks of chassis 140 are derived and calculated by the system based on the output shaft speed of the drive motor and the physical radius of the drive wheel. This indicates the physical wheelbase between the geometric center axes of the tracks on both sides of the chassis (140). This represents the instantaneous rotational angular velocity of the chassis 140 as a whole. The magnitude and direction difference between the actual linear velocity of the right track and the actual linear velocity of the left track of the chassis 140 together determine whether the chassis 140 is in a straight-line driving state, a stationary spinning state, or a curved yaw state. To avoid an absolute dead zone where the denominator in the formula equals zero, when the absolute value of the difference between the actual linear velocity of the right track and the actual linear velocity of the left track of the chassis 140 is less than a preset minimum constant, the system determines that the chassis 140 is in an ideal straight-line driving state and forcibly sets the instantaneous turning radius to infinity. In the underlying control logic, the angular velocity calculation branch is directly bypassed to output a pure translation control signal. The preset minimum constant is determined by the physical limit of the floating-point arithmetic precision of the underlying microprocessor.

[0047] Edge device 120 serves as the top-level decision-making core of the entire vision servo system, carrying highly complex visual perception algorithms and multiple concurrent control threads. The basic printed circuit board wiring structure and underlying power management module of edge device 120 can be implemented according to the existing industrial-grade embedded computing platform conventional manufacturing specifications.

[0048] To ensure the stable operation of high-performance inference models and multiple concurrent threads, the edge device 120 integrates a multi-core central processing unit based on a reduced instruction set architecture and a graphics processing unit (GPU) for tensor operation acceleration. The CPU and GPU of the edge device 120 establish a data exchange channel with the system main memory of the edge device 120 through a high-speed system bus. The high-speed system bus of the edge device 120 dynamically allocates memory read and write bandwidth according to a preset hardware arbitration mechanism.

[0049] In the specific system environment initialization and resource scheduling implementation process, the workflow of edge device 120 includes the following sub-steps: S211, after the edge device 120 is powered on, the central processing unit of the edge device 120 executes the hardware boot program to complete the hardware status self-test, initializes the memory management unit of the central processing unit of the edge device 120, and establishes the memory page table mapping relationship from virtual address to physical address; S212, the central processing unit of the edge device 120 activates the multi-core parallel mechanism according to the system task scheduling configuration file, and hard-binds the high-frequency motion control thread and the underlying communication interrupt response program to a specific internal core of the central processing unit of the edge device 120. The physical core prevents high-frequency motion control threads from being preempted by low-frequency perception tasks, thus preventing execution timing jitter; S213, the edge device 120 loads the hardware driver of the graphics processor of the edge device 120 for the dense matrix operation requirements of the visual perception node, and completely moves the pre-trained convolutional neural network visual inference model weight file from the non-volatile storage medium of the edge device 120 to the system main memory of the edge device 120 for high-frequency calling during the inference operation stage. The pre-trained convolutional neural network visual inference model weight file contains feature extraction convolution kernel parameters for processing three-dimensional tensor input data to support the allocation of underlying video memory resources.

[0050] In the underlying logic of memory allocation and address bus mapping performed by the edge device 120, the memory management unit of the central processing unit of the edge device 120 divides the system main memory of the edge device 120 into contiguous physical page frames of fixed length. The calculation logic of the absolute physical memory address requested by each concurrent thread follows the formula below. In the formula, This represents the actual absolute physical memory address allocated to edge device 120 for a specific concurrent thread. This indicates the default base memory page frame length in bytes. The specific value of the base memory page frame length is physically determined by the default page size of the CPU architecture of the edge device 120. This indicates the physical page index number assigned to a specific concurrent thread by the memory management unit of the central processing unit of edge device 120. This represents the relative addressing offset of data within a continuous physical page frame. The range of values ​​for the relative addressing offset is strictly limited by the underlying system to a physical interval greater than or equal to zero and less than the length of the basic memory page frame in bytes. Edge device 120 ensures physical isolation between memory spaces and data read / write safety between concurrent threads through underlying hardware-level memory paging and address translation mechanisms.

[0051] The middleware 150 is deployed on top of the operating system kernel of the edge device 120 as a basic configuration platform to support inter-process network communication between various independent running nodes. The construction of the underlying network protocol stack and the encapsulation of general interfaces of the middleware 150 can be implemented according to the existing robot operating system standard network communication specifications.

[0052] In the specific software environment initialization and topology network construction implementation process, the workflow of middleware 150 includes the following sub-steps: S221, the operating system kernel of edge device 120 starts the master node process of middleware 150 according to the standard underlying protocol specification and completes the system main memory resident for node discovery service; S222, the master node process of middleware 150 performs port listening and node communication topology dictionary establishment and configuration work under the standard protocol framework; S223, the master node process of middleware 150 prompts the absolute network address of the publishing end node and the absolute network address of the subscribing end node to complete the matching and establish a point-to-point transmission control protocol connection channel for underlying data packet flow.

[0053] In the network communication architecture supporting high-frequency data exchange in Middleware 150, to prevent memory overflow and crashes caused by data accumulation in the network buffer due to computing power bottlenecks or sudden task preemption by the subscribing nodes, Middleware 150 needs to configure an independent circular message queue for each data topic establishing a Transmission Control Protocol (TCP) connection channel. Middleware 150 balances memory consumption and loss prevention redundancy based on queue length. The depth calculation logic of the independent circular message queue follows the formula below. In the formula, This represents the maximum physical length of the independent circular message queue configured by middleware 150 for a specific data topic, in frames. This represents the average frequency at which the publishing node pushes data packets into the middleware's network buffer 150, measured in Hertz. This represents the maximum execution time, in seconds, for a subscriber node to retrieve data packets from the network buffer and complete a single business logic calculation. The dimensionless safety redundancy coefficient represents the degree of response to sudden network jitter and the operating system kernel threads of edge device 120. The value of the dimensionless safety redundancy coefficient is strictly limited to between 1.2 and 1.5 to ensure a minimum balance between system fault tolerance and physical memory overhead. The specific value of the dimensionless safety redundancy coefficient is obtained by physical calibration based on the extreme response latency of the edge device 120 central processing unit under full load stress test.

[0054] Middleware 150 has a data lag truncation protection logic during the operation of the node communication topology dictionary. When the rate at which the publishing node pushes in new data packets is consistently higher than the rate at which the subscribing node extracts them, causing the independent circular message queue for a specific data topic to reach its maximum physical length, middleware 150 will skip the conventional memory data block moving and erasing process and directly slide the read head pointer of the independent circular message queue backward by the physical memory address span of one data frame. This physically overwrites the discarded historical data frame with the oldest timestamp and pushes the latest arriving data frame into the physical memory address pointed to by the write tail pointer of the independent circular message queue. The underlying memory protection strategy ensures that when the edge device 120 faces instantaneous computing power overload, it can force the subscribing node to prioritize the processing of sensing data with the latest timestamp, completely avoiding the serious loss of control dead zone caused by the underlying communication channel being filled with old and lagging data.

[0055] Communication node 123 runs in the system main memory of edge device 120 as a data interaction interface between the connecting software algorithm model and the underlying physical control hardware. The underlying implementation of the network socket port listened to by communication node 123 and the asynchronous event-driven architecture can be implemented according to the existing embedded operating system network socket programming interface specifications.

[0056] In the specific implementation of protocol encapsulation and data link distribution, the workflow of communication node 123 includes the following sub-steps: S231, communication node 123 subscribes to floating-point kinematic control variables generated by the top-level decision algorithm through middleware 150. The floating-point kinematic control variables include the expected translational linear velocity and expected rotational angular velocity of chassis 140; S232, communication node 123 maps the floating-point kinematic control variables to signed integer data according to a preset physical scaling multiplier. Based on the underlying binary two's complement mechanism, the positive and negative directional characteristics of the kinematic control quantities are preserved. The preset physical scaling multiplier is limited to an integer multiple between one hundred and one thousand to ensure the floating-point kinematic control variables are properly sized. When a decimal number is converted into a signed integer after multiplication, the minimum physical precision recognizable by the underlying hardware is not lost. The signed integer data generated by the communication node 123 is decomposed into independent byte sequences bit by bit. The frame header identifier and frame tail identifier are assembled according to the preset underlying serial communication protocol framework. At the same time, the data check byte of the specific serial protocol data frame is calculated. In S233, the communication node 123 sends the specific serial protocol data frame to the control device 130 through the physical interface of the universal asynchronous transceiver of the edge device 120. At the same time, the communication node 123 maintains the listening state of the network socket port in an independent background thread to receive manual takeover instructions issued by the external host computer.

[0057] To avoid abnormal physical movement caused by differences in byte endianness parsing between different microprocessor architectures, communication nodes 123 enforce data type conversion and redundancy check mechanisms during the protocol encapsulation phase. The calculation logic for the data check byte within a specific serial protocol data frame follows the formula below. In the formula, This represents the data check byte value within a specific serial protocol data frame. This indicates the total length of the payload data bytes contained in a specific serial protocol data frame. This indicates the first byte in the payload data byte sequence. A hexadecimal unsigned integer consisting of 1 data byte.

[0058] This indicates that an arithmetic truncation operation modulo 256 is performed on the result of the summation of the payload data byte sequence. Communication node 123 physically compresses the multi-byte summation result of the payload data byte sequence into a single-byte storage space through the arithmetic truncation operation to strictly meet the physical bandwidth constraints of the underlying general asynchronous transceiver controller.

[0059] In the operational logic of the external human-machine interaction link, communication node 123 allocates a specific Boolean address block in memory space as a takeover status flag variable. When the network socket port bound to communication node 123 receives a manual control data packet sent by the external host computer, communication node 123 triggers a network receive callback function and rewrites the takeover status flag variable to a logical true value. During the time interval during which the takeover status flag variable maintains a logical true value, communication node 123 forcibly blocks and discards the automatic control data stream transmitted by middleware 150. Communication node 123 then transmits the manual control data... The data packets are parsed and converted into specific serial protocol data frames and directly distributed to the control device 130. When the external host computer disconnects or issues a release control command, the communication node 123 rewrites the takeover status flag variable to a logical false value and triggers a control state machine reset, forcibly sending a single zero-speed safety data packet to the control device 130. Simultaneously, the communication node 123 publishes a state machine reset event to the decision node 122 through the middleware 150 to clear the historical control state cache of the decision node 122's internal control algorithm, preventing the chassis 140 from undergoing sudden movements based on legacy automatic control commands. The underlying data truncation and interception logic executed by the communication node 123 establishes the absolute execution priority of manual operation by the external host computer and completely avoids mechanical loss of control of the chassis 140 under the alternating conflict between automatic control commands and manual control commands issued by the external host computer. Subsequently, the communication node 123 re-establishes an asynchronous subscription and listening mechanism with the middleware 150 and waits to receive the latest automatic kinematic control variables containing a valid timestamp to restore the autonomous tracking mode of the visual servo system, thereby ensuring a safe and smooth switch between manual takeover and autonomous tracking modes.

[0060] See attached document Figure 4 This invention provides a barnacle cleaning visual servoing system based on confidence-adaptive technology. Shared memory 124 is allocated within the physical space of the system main memory of the edge device 120 and serves as the data exchange core between the low-frequency visual perception thread running on the perception node 121 and the high-frequency motion decision thread running on the decision node 122. The underlying system calls and cross-process memory mapping mechanism of shared memory 124 can be implemented according to the existing operating system standard memory management specifications.

[0061] In the specific implementation of timing decoupling and data exchange, the workflow of shared memory 124 includes the following sub-steps: S311, the operating system kernel of edge device 120 allocates a continuous physical memory block with a fixed byte capacity outside the kernel space of the system main memory of edge device 120 as shared memory 124, and the operating system kernel of edge device 120 allocates a globally unique inter-process communication key value for shared memory 124; S312, the sensing node 121 and the decision node 122 respectively pass the globally unique inter-process communication key value through the underlying system call interface to complete the physical mapping of shared memory 124 to the virtual memory address spaces of the sensing node 121 and the decision node 122, the operating system kernel of edge device 120 grants the sensing node 121 physical write permission to the main data block of shared memory 124, and the operating system kernel of edge device 120 grants the decision node 122 physical read permission to the main data block of shared memory 124; at the same time, the operating system kernel of edge device 120 additionally opens an independent reverse control block inside shared memory 124, and grants the decision node 122 permission to access the reverse control block. The physical write permissions of the block and the physical read permissions of the sensing node 121 to the reverse control block are used to establish a non-blocking data link for the decision node 122 to feed back the sensing mode control flag bit to the sensing node 121. During the system power-on initialization phase, the operating system kernel of the edge device 120 forces the initialization of the sensing mode control flag bit in the reverse control block to a logical false value to ensure that the visual servo system is in the global full-size image input mode when it is first started. In S313, the sensing node 121 and the decision node 122 concurrently access the shared memory 124 based on the lock-free double buffer mechanism. After the sensing node 121 completes a single low-frequency visual inference calculation, it directly overwrites the data block inside the shared memory 124. A single visual perception data structure encapsulates the absolute coordinates of the target center pixel, the width of the bounding box pixel, the height of the bounding box pixel, the floating-point value of the target detection confidence, and the absolute physical timestamp when the data frame is generated in memory in a fixed order. During the high-frequency motion control cycle, the decision node 122 continuously extracts the visual perception results from the data block inside the shared memory 124 in a non-blocking mode.

[0062] To achieve physical timing decoupling between the low-frequency visual perception thread running on perception node 121 and the high-frequency motion decision-making thread running on decision node 122, and to avoid thread blocking issues caused by cross-process read-write locks, the shared memory 124 is physically divided into two independent data blocks of identical size, with an independent one-byte unsigned integer atomic status flag added. The calculation logic for the absolute address of the physical write block when perception node 121 performs a data overwrite operation follows the following formula. In the formula, This indicates the absolute address of the physical block being written to by sensing node 121 during the current data overwrite operation. This indicates the base physical starting address allocated by edge device 120 to shared memory 124. This indicates that the perception node 121 increments the version sequence number integer value each time it outputs new visual perception data to the shared memory 124. The version sequence number data type is specified as unsigned integer. When the version sequence number integer value reaches the physical maximum storage boundary value of unsigned integer, a forced wrap-around to zero occurs. The physical phenomenon of numerical wrap-around does not change the mathematical operation law of alternating parity of the version sequence number integer value. mod2 indicates that the modulo 2 arithmetic operation is performed on the incrementing version sequence number integer value to cause the perception node 121 to alternately switch the writing target between two independent data blocks. This represents the total length of physical bytes occupied by a single visual perception data structure.

[0063] In the non-blocking read operation logic, after completing the data overwrite to the absolute address of the current physical write block, the perception node 121 uses low-level atomic instructions to forcibly update the independent one-byte unsigned integer atomic status flag to record the physical index number of the latest independent data block containing complete visual perception data. At the beginning of each high-frequency motion control cycle, the decision node 122 directly reads the independent one-byte unsigned integer atomic status flag and locates the corresponding and complete independent data block for data extraction based on the physical index number recorded by the independent one-byte unsigned integer atomic status flag. Since the high-frequency read cycle of the decision node 122 is strictly shorter than the low-frequency write cycle of the perception node 121, the double-buffered access mechanism of the shared memory 124, which is physically isolated, combined with this absolute timing suppression characteristic, ensures that the operation of the decision node 122 in reading visual perception data effectively prevents data overwrite with that of the perception node 121. Write operations generate memory address collisions, thus preventing decision node 122 from missing fixed hardware control output timings while waiting for memory data to be unlocked. Simultaneously, after extracting visual perception data, decision node 122 immediately extracts the timestamp information attached to the visual perception data. When the difference between the current system absolute physical time and the timestamp information attached to the visual perception data exceeds the preset physical upper limit of perception delay, decision node 122 determines that the visual perception data inside the current shared memory 124 has entered an expired state and immediately forces the target detection confidence variable and normalized lateral spatial deviation in the internal computing environment to zero. Then, communication node 123 issues a zero-speed braking command to control device 130, thereby triggering the braking protection program of chassis 140. The specific value of the physical upper limit of perception delay is obtained based on the physical calibration of the maximum inference cycle time of the lightweight target detection model on the graphics processor inside edge device 120.

[0064] The perception node 121 runs in the graphics processor computing environment of the edge device 120 to perform target feature quantization extraction and inverse frame adjustment technology. The underlying image processing function library called by the perception node 121 can be implemented based on existing open-source computer vision software libraries.

[0065] In the specific implementation of visual reasoning and frame adjustment, the workflow of perception node 121 includes the following sub-steps: S321, perception node 121 acquires raw image data from perception device 110 and performs frame cropping and tensor dimension preprocessing according to the perception mode control flag bit fed back by decision node 122; S322, perception node 121 inputs the preprocessed image tensor into the lightweight object detection model for forward reasoning, and the lightweight object detection model outputs a multi-dimensional tensor containing the bounding box pixel coordinates of the barnacle target and the quantized detection confidence; S323, perception node 121 performs geometric affine transformation on the local center pixel coordinates output by the lightweight object detection model in the local region of interest mode to map back to the global coordinate system, and encapsulates the target parameters in the global coordinate system and writes them into shared memory 124 for decision node 122 to call.

[0066] For the lightweight object detection model's network structure and training mechanism, the perception node 121 adopts a single-stage object detection neural network based on a cross-stage local network. The internal hierarchical structure of this single-stage object detection neural network includes a cross-stage local backbone feature extraction network for extracting low-level edge and texture features from the image, a path aggregation feature pyramid network for fusing multi-scale semantic features, and a decoupled detection head module for outputting prediction results. Image tensor data flows sequentially through the cross-stage local backbone feature extraction network and the path aggregation feature pyramid network within the single-stage object detection neural network before finally being fed into the decoupled detection head module. During the model construction and training phases, training samples are derived from a dataset of real barnacle attachment images collected by an underwater robot, and sample labels are manually annotated with real edges. The training steps of a single-stage object detection neural network, which consists of bounding box pixel coordinates and barnacle category indexes, involve inputting shuffled training samples into the network and updating its weights via backpropagation. During the training phase, an adaptive moment estimation optimization algorithm is used to update the gradients of the network weights. The initial learning rate is set to 0.001 and iteratively decays using cosine annealing. Key parameters and loss functions used in the training phase include the full intersection-over-union (CIU) loss function for regressing bounding box coordinates and the binary cross-entropy loss function for evaluating classification accuracy. Technicians can reproduce the entire training process of weight convergence in the single-stage object detection neural network based on the CIU loss function and the binary cross-entropy loss function.

[0067] In the actual inference deployment scenario of perception node 121, the input data preprocessing logic of the lightweight object detection model includes uniformly scaling the original image data or the cropped local image to a fixed width and height pixel dimension using bilinear interpolation. The fixed width and height pixel dimensions are uniformly set to 640 pixels by 640 pixels. Perception node 121 converts the input data channel arrangement into a red, green and blue three-channel format and then performs a pixel value normalization operation by dividing all red, green and blue pixel grayscale values ​​by 255 to generate a four-dimensional floating-point image tensor containing a single batch number, three color channels, and a width and height of 640. The specific physical state output by the decoupled detection head module of the lightweight object detection model includes the x and y coordinates of the center point of the bounding box representing the spatial distribution state of barnacles, as well as the width and height of the bounding box. The decoupled detection head module also outputs the object detection confidence, which represents the degree of certainty of the algorithm's determination of the probability of barnacles in the current field of view. The object detection confidence provides quantitative perceptual quality feedback for the adaptive decision-making of subsequent visual servoing.

[0068] To improve the detection stability of long-distance, small-scale barnacles under the limited computing power of edge device 120, perception node 121 performs reverse frame adjustment technology. Before the start of each inference cycle, perception node 121 reads the perception mode control flag bit fed back by decision node 122. When the perception mode control flag bit is in the logical false state, perception node 121 maintains the global full-size image input mode to perform macro-target search. When decision node 122 determines that the barnacle target is continuously detected and the target detection confidence reaches the stable threshold, decision node 122 rewrites the perception mode control flag bit to the logical true state. The value range of the stable threshold is set between 0.7 and 0.85 to distinguish between accidental false detection noise and the real continuous tracking state. After parsing the perception mode control flag bit in the logical true state, perception node 121 switches to the local region of interest mode. Based on the center point of the barnacle bounding box detected in the previous frame image, perception node 121 expands outward by a preset ratio in combination with the length and width of the bounding box of the previous frame, and crops a local image block with dynamic pixel size on the original image data. To prevent memory array addressing out-of-bounds crashes caused by local image blocks approaching the physical boundaries of the original image, the perception node 121 performs a bidirectional numerical clamping operation on the four vertex pixel coordinates of the cropping bounding box before performing image cropping. When the vertex pixel coordinates are less than zero, they are forcibly clamped to the zero-point reference pixel coordinates. When the vertex pixel coordinates exceed the resolution limit, a numerical clamping operation based on the extreme values ​​of the original image width and height is forcibly performed. Subsequently, the perception node 121 uniformly processes the clamped local image blocks using an edge grayscale filling strategy that maintains the original aspect ratio and a bilinear interpolation scaling operation to the tensor dimension required by the fixed input dimension of the lightweight object detection model. This completely avoids the barnacle texture distortion caused by forced image stretching, and only the preprocessed local image blocks are input into the lightweight object detection model for inference. The dynamic image cropping and scaling switching logic executed by the perception node 121 not only improves the resolution ratio of local features but also significantly reduces background interference.

[0069] In the Local Region of Interest (ROI) mode, the coordinates of the bounding box center point output by the lightweight object detection model belong to the relative coordinate system of the local image block. Perception node 121 must perform an affine transformation to map back to the global coordinate system to guide the chassis 140 in performing accurate spatial motion. Before performing geometric mapping calculations, perception node 121 first extracts the unidirectional grayscale fill pixel width introduced in the horizontal or vertical direction during the edge grayscale fill strategy in the preprocessing stage. This unidirectional grayscale fill pixel width is strictly calibrated to the feature scale under the fixed input dimension of the lightweight object detection model. The relative coordinates of the bounding box center point output by the lightweight object detection model are then subtracted from this unidirectional grayscale fill pixel width to eliminate the physical translation of the zero-point coordinates caused by maintaining the aspect ratio of the image. After coordinate correction, the geometric calculation steps for global coordinate mapping follow the following formula:

[0070] In the formula, This represents the absolute x-coordinate of the center point of the barnacle bounding box in the original global image pixel coordinate system after mapping. This represents the absolute ordinate of the center point of the barnacle bounding box in the original global image pixel coordinate system after mapping. This represents the horizontal coordinate offset of the top-left corner reference point of a local image block in the original global image pixel coordinate system. This represents the offset of the top-left corner reference point of a local image block in the original global image pixel coordinate system. This represents the effective x-coordinate of the center point of the barnacle bounding box output by the lightweight object detection model within the local image block after removing the grayscale fill offset. This represents the effective ordinate of the barnacle bounding box center point output by the lightweight object detection model within the local image block after removing the grayscale fill offset. This represents the inverse scaling ratio of a local image patch scaled to the fixed input dimension of the lightweight object detection model. The inverse scaling ratio is calculated by dividing the actual physical pixel length of the local image patch by the pixel width of the fixed input dimension of the lightweight object detection model, ensuring a strictly consistent mapping ratio while maintaining the original aspect ratio. Perception node 121 performs rounding operations on the floating-point absolute x-coordinates and absolute y-coordinates calculated after affine mapping to match the discrete integer index attributes of the global original image pixel coordinate system. Perception node 121 ensures, through the affine transformation formula and rounding operation, that the barnacle space coordinate reference obtained by decision node 122 remains consistent and conforms to the underlying memory addressing specifications regardless of the frame mode switching.

[0071] The decision node 122 runs in the main memory physical space of the edge device 120 to perform nonlinear variable mapping and anti-deadlock state machine control technology. For the underlying proportional regulation automatic control algorithm called by the decision node 122, those skilled in the art can implement the underlying automatic control algorithm based on existing industrial control theory.

[0072] In the specific implementation of adaptive decision-making and state machine transition, the workflow of decision node 122 includes the following sub-steps: S331, decision node 122 extracts visual perception data from shared memory 124 and calculates the lateral spatial deviation; when the target is not detected, decision node 122 triggers the safety isolation mechanism of zeroing the underlying parameters; S332, decision node 122 performs positive interpolation mapping of the expected translation linear velocity of chassis 140 and negative adjustment calculation of the dynamic angular velocity proportional control gain coefficient of chassis 140 based on the target detection confidence; at the same time, when decision node 122 continuously loses target features, it triggers the anti-deadlock state machine and sends a reverse mode switching command to perception node 121, thereby breaking the local frame deadlock closed loop.

[0073] In the physical calculation logic for calculating the lateral spatial deviation, decision node 122 extracts the absolute x-coordinate of the barnacle bounding box center point in the original global image pixel coordinate system. The mathematical logic for the normalization calculation performed by decision node 122 follows the formula below. In the formula, This represents the normalized horizontal spatial deviation floating-point value. This represents the absolute x-coordinate of the center point of the barnacle bounding box in the original global image pixel coordinate system after mapping. This represents half the physical width pixel value of the original global image. The normalization operation, which converts the absolute pixel horizontal coordinate into a dimensionless proportional value with alternating positive and negative values, ensures that the underlying control gain parameters subsequently set by decision node 122 are not physically limited by the resolution differences between different camera models. To prevent the chassis 140 from uncontrolled rotation based on the last valid lateral spatial deviation at the moment the barnacle target is lost, decision node 122, when determining that the visual perception data indicates the target was not detected or that the visual perception data has timed out, overwrites the normalized lateral spatial deviation floating-point value to zero and simultaneously forces the target detection confidence variable to be cleared. This triggers the safety isolation mechanism of clearing the underlying control parameters of chassis 140, ensuring that the desired rotational angular velocity and desired translational linear velocity of chassis 140 are synchronously truncated under abnormal conditions. Simultaneously, when the trigger timed out is determined, decision node 122 will synchronously force the clearing of the built-in continuous lost frame counter to completely prevent erroneous deadlock state machine jumps caused by residual historical dirty data after the network or perception process recovers from a apparent deadlock.

[0074] The decision node 122 executes control commands based on the target detection confidence level using a nonlinear variable mapping mechanism. The forward interpolation calculation logic for the expected translational linear velocity of the chassis 140 follows the formula below. In the formula, This indicates the desired translation speed of the chassis at 140. This represents the preset physical lower limit of linear velocity. This indicates the preset physical upper limit of linear velocity. This represents the target detection confidence score output by the lightweight target detection model. This represents the effective lower limit threshold for confidence. The confidence saturation upper threshold is defined as the upper threshold, the confidence effective lower threshold is defined as the lower threshold is defined as the upper threshold is defined as the lower threshold is defined as the upper threshold is defined as the upper threshold is defined as the lower threshold. The preset physical lower limit and the preset physical upper limit of linear velocity are obtained through actual dynamic calibration based on the extreme power of the underwater thruster of chassis 140. When the target detection confidence is lower than the effective lower threshold, decision node 122 directly truncates the mathematical calculation and outputs a zero-speed braking command. In order to avoid the chassis 140 from mechanically jerking due to the instantaneous step drop of the speed command to zero, this zero-speed braking... The instructions are smoothed and filtered by a preset deceleration slope function within the underlying microcontroller before driving the chassis 140. This reduces the risk of physical collisions in the barnacle cleaning vision servo system under blurred vision. To avoid a logical dead zone where the calculation results violate physical limits due to the target detection confidence exceeding the interpolation range, the decision node 122 directly sets the expected translational linear velocity of the chassis 140 to a preset physical upper limit when the target detection confidence is greater than or equal to the confidence saturation upper limit threshold. Simultaneously, the negative adjustment logic of the dynamic angular velocity proportional control gain coefficient follows the formula below: In the formula, This represents the basic proportional gain constant, which is obtained by physical calibration of the overshoot of the chassis 140's limit steering angular velocity under full-load test of the system closed-loop performance using proportional-integral-derivative control theory. This indicates the sensitivity factor, whose value range is set between 0.1 and 0.3 to limit the physical amplitude of gain attenuation controlled by the dynamic angular velocity ratio.

[0075] Furthermore, the underlying desired rotational angular velocity command output by decision node 122 follows the formula below: In the formula, This represents the desired rotational angular velocity of the chassis at 140 degrees. This represents the symbolic mapping coefficient used to adapt the wiring polarity of the physical motor of the chassis 140 to the positive direction of the image coordinate system of the sensing device 110. The value of this symbolic mapping coefficient is strictly limited to positive one or negative one. The specific value is determined by the visual servo system in the closed-loop negative feedback polarity calibration test before leaving the factory to ensure that the system has a physical yaw tendency to move toward the center of the target. This represents the dynamic angular velocity proportional control gain coefficient. This basic proportional gain constant is obtained by physical calibration of the overshoot of the chassis 140's limit steering angular velocity under full-load test of the system closed-loop performance using proportional control theory. This represents the normalized lateral spatial deviation floating-point value. The nonlinear variable mapping mechanism executed by decision node 122 causes chassis 140 to amplify its steering gain when the target detection confidence is low in order to quickly center the barnacle target in the visual field, and to reduce the steering gain of chassis 140 when the target detection confidence is high in order to reduce the mechanical overshoot oscillation phenomenon of chassis 140.

[0076] During the control timing flow of the anti-deadlock state machine, decision node 122 first performs forward tracking judgment. When the target detection confidence extracted by decision node 122 in multiple consecutive high-frequency motion control cycles reaches the preset stability threshold, decision node 122 determines that the system has entered the stable tracking state and rewrites the perception mode control flag bit written to the reverse control block of shared memory 124 to the logical truth value state to trigger perception node 121 to enter the local region of interest mode. The value range of the stability threshold is set between 0.7 and 0.85. At the same time, decision node 122 allocates an independent integer variable in system memory as a continuous lost frame counter. When decision node 122 reads a valid and non-expired barnacle bounding box coordinate from shared memory 124 in a single high-frequency motion control cycle and the target detection confidence is greater than the effective lower limit threshold of the confidence, decision node 122 physically clears the continuous lost frame counter to prevent logical misjudgment caused by historical accumulated values.

[0077] When the visual perception data read by decision node 122 indicates no target and the version sequence number of the visual perception data has been updated compared to the previous read, decision node 122 causes the continuous loss frame counter to increment by one. During the counting period, it continuously maintains the zero values ​​of the desired translational linear velocity and the desired rotational angular velocity to ensure physical safety braking. When the value of the continuous loss frame counter reaches the preset loss tolerance threshold, decision node 122 determines that the current barnacle clearing visual servo system has entered a deadlock state in the partial frame mode where the barnacle target has completely moved out of the field of view boundary. The specific value is set between ten and fifteen perception frames to balance the anti-interference tolerance and deadlock response delay of the barnacle clearing visual servo system. Decision node 122 then triggers the reverse state machine transition and rewrites the perception mode control flag bit written to the reverse control block of shared memory 124 to a logical false value state, completely clears the historical motion control cache space and resets the state machine transition sequence. At this time, perception node 121 receives the perception mode control flag bit in the logical false value state by reading the reverse control block, exits the local region of interest mode and restores the global full-size image mode for macroscopic search. Meanwhile, to prevent the target from completely moving out of the physical field of view and causing the system to permanently stop, after switching back to the global full-size image mode, decision node 122 will forcibly bypass and override the aforementioned target undetected parameter zeroing safety isolation mechanism within the state machine. During this specific search phase, the output of the zero-speed braking command will be paused, and the linear velocity interpolation and angular velocity gain calculation modules executed based on the target detection confidence level will be forcibly blocked in the internal computing environment. Instead, a preset low-speed in-situ spin search angular velocity command will be directly sent to the lower layer, forcing the chassis 140 to enter an active seeking rotation state until the perception node 121 recaptures the target features that meet the effective lower limit threshold of the confidence level in the global image. At this time, decision node 122 responds to the state machine transition, exits the active seeking rotation state, and synchronously restores the aforementioned target undetected parameter zeroing safety isolation mechanism and interpolation calculation module. Based on the latest extracted lateral spatial deviation, the autonomous tracking closed loop of the system is restored. The timing state machine transition process executed by decision node 122 effectively breaks the continuous visual loss deadlock closed loop caused by the mutual clamping of local detection boxes and ensures the autonomous escape capability of the physical actuator.

[0078] To aid in understanding the technical solution of this invention, a specific application example of underwater bridge pier base barnacle cleaning operation is provided below.

[0079] In this application embodiment, the barnacle cleaning visual servo system is deployed on an underwater tracked cleaning robot platform. The sensing device uses an underwater global shutter high-resolution camera, and the edge device uses an embedded computing board integrating a reduced instruction set architecture central processing unit and a dedicated graphics processing unit. After the system is powered on, the sensing device acquires environmental image data of the underwater bridge pier surface at a fixed frequency of 30 frames per second, and the middleware inside the edge device establishes the data exchange communication topology for each computing node.

[0080] In the initial stage, the perception node performs forward feature inference on the environmental image data in a global full-size mode. When the edge device detects barnacles attached to the bridge pier surface, and the barnacle detection confidence score is above a stable threshold of 0.75 for three consecutive frames, the decision node sets the perception mode control flag to ground truth. After reading the ground truth, the perception node crops a local image block by extending it outward by 200 pixels based on the center coordinates of the barnacle bounding box in the previous frame. This local image block is then rescaled and input into the neural network for inference. After performing local cropping, the single-frame tensor computation time of the edge device is significantly reduced.

[0081] The decision node synchronously performs interpolation mapping calculations based on the detection confidence level. When the detection confidence level reaches 0.88, the decision node proportionally increases the desired translational linear velocity and decreases the desired rotational angular velocity gain, sending a tracking control command to the chassis. After receiving the command, the chassis performs a straight-line movement towards the barnacle target at a relatively fast linear velocity. When external water flow disturbances cause the barnacle to briefly move out of the field of view, the continuous frame loss counter inside the decision node performs an accumulation operation. If the counter records 12 consecutive frames lost, the decision node determines that the system has entered a deadlock state and forcibly resets the perception mode control flag to a false value. The perception node restores the global full-size environmental image inference mode based on the false value, and the decision node simultaneously sends a fixed low-speed in-place spin angular velocity command to the chassis. The chassis performs an in-place rotation active search action until the perception node re-outputs the barnacle target coordinates with a valid confidence level in the global image, and the system re-enters the closed-loop tracking state.

[0082] Table 1. Comparison of underwater operation performance test data between visual servo system and traditional methods for barnacle cleaning. According to the data in Table 1, the confidence-adaptive visual servoing method for barnacle cleaning proposed in this invention demonstrates significant technical advantages in multiple core parameter indicators in underwater high-frequency dynamic operating environments. The specific analysis and discussion are as follows:

[0083] In terms of system computing resource utilization and operating frequency, the average processing frame rate of traditional fixed-gain global inference methods ranges from 13.85Hz to 14.51Hz. The confidence-adaptive visual servoing method of this invention increases the overall average processing frame rate to between 27.96Hz and 29.13Hz. The underlying hardware logic behind this significant frame rate increase lies in the asynchronous read / write communication decoupling mechanism based on shared system memory between the sensing and decision nodes, eliminating data blocking and waiting time between processes. Simultaneously, the dynamic initiation of the local image cropping inference mode effectively eliminates redundant background pixel calculations in the global environment image, completely resolving the processing latency problem at the sensing end caused by the computing power bottleneck of underwater embedded edge devices.

[0084] In terms of closed-loop control accuracy and tracking response of the actuator, the root mean square value of spatial tracking deviation of the traditional method remains at a relatively high level of 47.81 pixels to 51.23 pixels. The method of this invention significantly reduces the root mean square value of spatial tracking deviation to the range of 18.42 pixels to 21.15 pixels. Moreover, the average translational linear velocity output by the method of this invention (0.245 m / s to 0.273 m / s) is approximately twice that of the traditional method (0.117 m / s to 0.131 m / s). The relevant data differences directly verify the physical effectiveness of the model that performs positive interpolation of linear velocity and negative adjustment of angular velocity gain based on the detection confidence value. The confidence-adaptive vision servo system can output high translational linear velocity and low steering ratio gain when the target features are clear and stable, preventing mechanical overshoot oscillation of the tracked chassis. When the target deviates from the image center, the system can quickly amplify the steering ratio gain according to the decreasing confidence trend, forcing the chassis to quickly complete the centering alignment of the field of view, maintaining the dynamic tracking balance of the entire servo link.

[0085] In terms of system anti-interference and abnormal escape capabilities, traditional methods, limited by fixed control logic, suffer from average re-acquisition times of 1856.4 to 2104.7 milliseconds after being trapped in a localized loss of vision due to random underwater fluid disturbances causing targets to lose their field of vision. Furthermore, these methods have repeatedly caused irregular chassis rotation in place during tests. The target loss recovery time recorded by the method of this invention is concentrated between 405.8 and 518.6 milliseconds. This order-of-magnitude reduction in recovery time is attributed to the anti-deadlock timing state machine logic integrated within the decision node. After determining that the consecutive frame loss counter has reached a preset tolerance threshold, the state machine forcibly interrupts the local search loop and transitions in reverse to the global search mode, while simultaneously actively outputting chassis yaw and spin search commands. This complete anti-deadlock logic effectively avoids the permanent loss of vision caused by mutual clamping of local detection frames, restoring the autonomous operation capability of the servo system.

[0086] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A confidence-adaptive visual servoing system for barnacle cleaning, comprising: Sensing device (110), edge device (120), control device (130), and chassis (140); The feature is that the sensing device (110) establishes a video data stream transmission connection with the edge device (120), and the sensing device (110) is used to collect external environment image data and send it to the edge device (120). The edge device (120) establishes an electrical connection with the control device (130) through a serial communication bus, and the signal output terminal of the control device (130) is connected to the drive motor of the chassis (140); The edge device (120) is configured to run middleware (150), and the system software layer includes a perception node (121), a decision node (122) and a communication node (123). The edge device (120) has shared memory (124) allocated in its memory space. Both the sensing node (121) and the decision node (122) establish a memory read-write mapping relationship with the shared memory (124) to complete asynchronous data interaction under different running thread frequencies; The perception node (121) acquires the external environment image data and performs target detection algorithm inference, outputs the target bounding box center coordinates and detection confidence, and writes the extracted target bounding box center coordinates and detection confidence into the shared memory (124). The decision node (122) reads the latest target bounding box center coordinates and detection confidence from the shared memory (124) in a non-blocking manner at a fixed control loop frequency; the decision node (122) calculates the lateral spatial deviation based on the read target bounding box center coordinates and image center coordinates, and performs interpolation mapping calculation on the preset starting linear velocity and preset steering gain coefficient according to the detection confidence value to generate linear velocity control instructions and angular velocity control instructions; The communication node (123) receives the linear velocity control command and the angular velocity control command and sends them to the control device (130). The control device (130) is used to output a pulse width modulation signal to the chassis (140) to control the motion state of the chassis (140).

2. The barnacle cleaning visual servoing system based on confidence adaptation according to claim 1, characterized in that: The shared memory (124) is physically divided into two independent data blocks of identical size, each with an independent one-byte unsigned integer atomic state flag. The perception node (121) performs data overwrite operations alternately between the two independent data blocks based on the arithmetic result of modulo 2 of the incrementally increasing version sequence number integer value. Each write includes the center coordinates of the target bounding box, the detection confidence, and the absolute physical timestamp. The atomic state flag is updated using atomic instructions to record the physical index number of the latest independent data block. The decision node (122) reads the atomic state flag at the beginning of the control cycle and locates the corresponding independent data block to extract visual perception data based on the recorded physical index number. When the decision node (122) determines that the difference between the current system absolute physical time and the absolute physical timestamp exceeds the preset physical upper limit of perception delay, it determines that the data has expired and is invalid, and forces the target detection confidence variable in the internal computing environment and the lateral spatial deviation to zero.

3. The barnacle cleaning visual servoing system based on confidence adaptation according to claim 1, characterized in that: An additional independent reverse control block is allocated within the shared memory (124); when the decision node (122) determines that the barnacle target is continuously detected and the detection confidence reaches a stable threshold, it generates a perception mode control flag in a logical truth state and writes it into the reverse control block; before each inference cycle, the perception node (121) reads the perception mode control flag in the reverse control block. If it is in a logical false state, it maintains the global full-size image input mode; if it is in a logical truth state, it switches to the local region of interest mode, uses the center point of the target bounding box detected in the previous frame as a reference and expands outward in combination with the size of the bounding box in the previous frame, performs numerical clamping and cropping operations on the external environment image data to extract local image blocks, scales the local image blocks to a fixed input dimension, and then performs target detection algorithm inference.

4. The barnacle cleaning visual servoing system based on confidence adaptation according to claim 3, characterized in that: In the local region of interest mode, the perception node (121) performs a geometric affine transformation mapping the center point coordinates of the target bounding box in the relative coordinate system of the inference output back to the global coordinate system; the perception node (121) subtracts the unidirectional grayscale fill pixel width introduced in the horizontal or vertical direction when the edge grayscale fill strategy is executed in the preprocessing stage from the center point coordinates of the target bounding box in the relative coordinate system, and calculates the absolute horizontal and absolute vertical coordinates of the center point of the target bounding box in the original global image pixel coordinate system after mapping, combined with the reverse scaling restoration ratio value of the local image block scaled to a fixed input dimension and the positioning offset of the upper left corner reference point of the local image block in the original global image pixel coordinate system, and writes them into the shared memory (124) after performing a rounding operation.

5. A barnacle cleaning visual servoing system based on confidence-adaptive design according to claim 1, characterized in that: The nonlinear variable mapping calculation process for generating the linear velocity control command and angular velocity control command based on the detection confidence value of the decision node (122) includes: when the detection confidence is between the effective lower limit threshold and the confidence saturation upper limit threshold, the decision node (122) calculates the expected translational linear velocity as the linear velocity control command by positively proportionally amplifying the difference between the detection confidence and the effective lower limit threshold based on the preset physical lower limit and physical upper limit of linear velocity; the decision node (122) weights the detection confidence based on the adjustment sensitivity factor and subtracts it from the unit, and multiplies the difference result with the basic proportional gain constant to obtain the dynamic angular velocity proportional control gain coefficient; the decision node (122) multiplies the normalized lateral spatial deviation, the dynamic angular velocity proportional control gain coefficient and the preset sign mapping coefficient together to calculate the expected rotational angular velocity as the angular velocity control command.

6. The barnacle cleaning visual servoing system based on confidence adaptation according to claim 1, characterized in that: The decision node (122) integrates an anti-deadlock timing state machine and allocates a continuous loss frame counter in the system memory. When the visual perception data read by the decision node (122) indicates that there is no target and the version sequence number of the visual perception data is incremented, the continuous loss frame counter performs an accumulation operation, and the decision node (122) continuously outputs the linear velocity control command and angular velocity control command with zero values. When the value of the continuous loss frame counter reaches the preset loss tolerance threshold, the anti-deadlock timing state machine triggers a reverse transition. The decision node (122) rewrites the perception mode control flag to a logical false value state to trigger the perception node (121) to restore the global full-size image input mode. At the same time, the decision node (122) forces the bypass target undetected parameter to return to zero safety isolation mechanism and sends a preset low-speed in-situ spin search angular velocity command to the underlying layer to force the chassis (140) to enter the active seek rotation state.

7. A barnacle cleaning visual servoing system based on confidence adaptation according to claim 6, characterized in that: When the sensing node (121) re-outputs a detection confidence greater than the effective lower limit threshold in the global full-size image input mode, the anti-deadlock timing state machine exits the active seek rotation state, the decision node (122) physically clears the continuous lost frame counter, resumes interpolation mapping calculation, and resumes closed-loop tracking based on the latest extracted lateral spatial deviation.

8. The barnacle cleaning visual servoing system based on confidence adaptation according to claim 1, characterized in that: The middleware (150) configures an independent ring message queue for each data topic to establish data exchange. The maximum physical length of the independent ring message queue is determined by multiplying the average publishing frequency of data packets pushed into the middleware (150) network buffer by the publishing node, the maximum execution time of the subscribing node to extract data packets from the network buffer and complete a single business logic calculation, and a preset dimensionless security redundancy coefficient. When the independent ring message queue reaches the maximum physical length, the middleware (150) slides the read head pointer of the independent ring message queue backward by the physical memory address span of one data frame and overwrites the historical data frame with the oldest timestamp at the physical level to ensure that the subscribing node prioritizes the processing of the sensor data with the latest timestamp.

9. A barnacle cleaning visual servoing system based on confidence adaptation according to claim 1, characterized in that: The control device (130) runs inverse kinematics calculation logic, receives the desired translational linear velocity and the desired rotational angular velocity, and calculates the desired target linear velocity of the left track and the desired target linear velocity of the right track by combining the physical span between the equivalent contact points of the drive wheels on the left and right sides of the chassis (140). The control device (130) converts the desired target linear velocity of the left track and the desired target linear velocity of the right track into the duty cycle percentage value of the pulse width modulation signal of the hardware timer channel based on the physical mapping relationship of the bus DC voltage, and outputs it to the motor drive bridge circuit built into the chassis (140) to drive the physical motor.

10. A barnacle cleaning visual servoing system based on confidence-adaptive design according to claim 1, characterized in that: The communication node (123) maintains the listening state of the network socket port in an independent background thread and allocates a Boolean address block in the memory space as a takeover status flag variable. When the communication node (123) receives a manual control data packet sent by the external host computer, it rewrites the takeover status flag variable to a logical true value, forcibly blocks and discards the automatic control data stream transmitted by the middleware (150), and parses and distributes the manual control data packet to the control device (130). When the external host computer disconnects, the communication node (123) rewrites the takeover status flag variable to a logical false value, forcibly sends a single zero-speed safety data packet to the control device (130), and issues a state machine reset event to the decision node (122) to clear the historical control status cache.

Images