Sea cucumber fishing robot based on machine vision and fishing method

Abstract

The invention relates to a sea cucumber fishing robot based on machine vision and a fishing method, and solves the technical problems that in the fishing process of an existing sea cucumber fishing robot, sea cucumbers are prone to being damaged, the working efficiency is low, the production environment of the sea cucumbers is prone to being damaged, accurate fishing is achieved, and intelligence is improved. The device comprises an unmanned underwater vehicle, a first depth camera, a parallel mechanical arm, a hose and a suction pipe, and is used for accurately identifying sea cucumbers, performing graded fishing and performing high-stability motion control in a complex seabed environment, and meanwhile, avoiding damage to the seabed ecological environment.

Description

Technical Field

[0001] This invention relates to the field of sea cucumber harvesting technology, and more specifically, to a sea cucumber harvesting robot and harvesting method based on machine vision. Background Technology

[0002] Oceans cover 70.8% of the Earth's surface and contain abundant mineral and marine biological resources, which are important assets for the sustainable development of human society.

[0003] Sea cucumbers are a highly nutritious seafood product. They feed on seaweed and plankton and are typically farmed in nearshore waters. Currently, sea cucumber harvesting is mainly done manually using simple tools by divers, which is inefficient, costly, and dangerous. Existing technologies also use robotic arms to grasp sea cucumbers, but these arms easily damage the cucumbers, resulting in low efficiency. Furthermore, sea cucumbers of all sizes are caught without differentiation, and smaller ones are not released. Referring to invention patent applications CN118489638A and CN112544577A, underwater harvesting robots with walking mechanisms have also been used. However, these robots damage seaweed and the sea cucumbers' habitat while moving through the seabed, easily crushing some sea cucumbers. In environments with reefs and silt, the walking mechanism is difficult to operate reliably and stably, affecting the robot's normal operation.

[0004] Existing technologies can lead to the accidental capture of juvenile sea cucumbers, seriously threatening resource sustainability.

[0005] Therefore, developing a sea cucumber harvesting robot with accurate identification capabilities, non-contact harvesting methods, and highly stable control is of significant scientific importance and practical economic value. Summary of the Invention

[0006] This application aims to address the technical problems of existing sea cucumber harvesting robots, such as easy damage to sea cucumbers, low operating efficiency, and easy disruption of the sea cucumber production environment, and to provide a machine vision-based sea cucumber harvesting robot and harvesting method based on machine vision and sliding mode predictive control (SMPC).

[0007] The first aspect of this disclosure provides a machine vision-based sea cucumber harvesting robot, including an unmanned underwater vehicle (UUV), a first depth camera, a parallel robotic arm, a hose, and a suction tube. The parallel robotic arm is connected to the bottom of the UUV's hull, and the first depth camera is connected to the front of the UUV. The parallel robotic arm has a top connecting plate with a hose connection hole. The upper part of the hose is fixedly connected to the hose connection hole, and the lower end of the hose is fixedly connected to the end of the parallel robotic arm. The suction tube is fixedly connected to the end of the robotic arm and communicates with the lower end of the hose. The UUV has a hull, and the upper part of the hose passes through the hose connection hole and then passes upward through the bottom of the hull. A collection chamber and a pump are connected to the bottom of the UUV's hull. The collection chamber has an inlet and an outlet. The upper end of the hose is connected to the inlet of the collection chamber, and the outlet of the collection chamber is connected to the inlet of the pump via a pipe. A drain pipe is connected to the outlet of the pump, and a screen is connected to the inner cavity of the collection chamber, with the screen close to the outlet of the collection chamber.

[0008] Preferably, the machine vision-based sea cucumber harvesting robot also includes a second depth camera, which is connected to the end of the parallel robotic arm.

[0009] A second aspect of this disclosure provides a sea cucumber harvesting method using a machine vision-based sea cucumber harvesting robot, characterized by comprising the following steps:

[0010] The first step is to establish a sea cucumber target detection model based on convolutional neural networks;

[0011] The second step involves the first depth camera acquiring images of the seabed environment, obtaining color and depth images, and then inputting the color and depth images into the controller.

[0012] The third step is for the controller to preprocess the color image;

[0013] The fourth step is to input the preprocessed color image into the sea cucumber target detection model to identify the sea cucumber target area and obtain the detection result image.

[0014] The fifth step is to perform pixel-level registration between the detection result image and the depth image obtained in the second step, based on the sea cucumber detection box in the detection result image, and extract the three-dimensional point cloud data corresponding to the target area.

[0015] Step 6: Estimation of quantitative indicators based on 3D convex hull;

[0016] The seventh step is to grade the sea cucumbers according to quantitative indicators;

[0017] The eighth step is to harvest the sea cucumbers.

[0018] Preferably, the sea cucumber target detection model based on convolutional neural networks is an improved YOLOv8 target detection model that includes a CABM module.

[0019] Preferably, the improved YOLOv8 target detection model also includes a Mask detection module.

[0020] Preferably, the improved YOLOv8 object detection model is established in the following way:

[0021] Step (1): Create an underwater image dataset for training the target detection model;

[0022] Step 1): Obtain the GroundTruth dataset of real underwater images from a public online database;

[0023] Step 2): For all images in the GroundTruth dataset, underwater imaging characteristics simulation processing is performed. After spectral absorption, backscattering, forward scattering blurring, and camera image signal processing pipeline, the sample image Isim is obtained.

[0024] Step 3): Restore the simulated sample images using a lightweight Visual Transformer model;

[0025] Step 4): Select images from the GroundTruth dataset according to a certain ratio, and together with the restored sample images, form the underwater image dataset Train for training the target detection model;

[0026] Step (2) Semantic segmentation and annotation of sea cucumber targets in the underwater image dataset Train are performed, sea cucumber edge features are extracted, and a semantic segmentation training set is constructed.

[0027] Step (3) train the improved YOLOv8 object detection model using the semantic segmentation training set.

[0028] Preferably, step six:

[0029] For point cloud data P in the target area f ={P i ∈R 3 A 3D convex hull calculation is performed to obtain the 3D convex hull structure of the sea cucumber. The surface of the 3D convex hull structure is a closed triangular mesh T with uniform normals. The triangular mesh T contains several triangular faces, and the vertices of each triangular face f∈T are (v0,v1,v2), with an outward normal of n. f The plane offset is d f The centroid of the triangular mesh T is P0;

[0030] Calculate volume V hullUsing the centroid P0 of the mesh as a reference point, and combining each triangular facet (v0, v1, v2) of the triangular mesh T to form a tetrahedron (P0, v0, v1, v2), volume accumulation is performed:

[0031]

[0032] Calculate the approximate volume V cap :

[0033] For point cloud data P in the target area f ={P i ∈R 3 To perform principal component analysis, calculate the covariance ∑ using the following formula:

[0034]

[0035] In the formula, μ is the centroid;

[0036] Perform eigenvalue decomposition on ∑, obtaining three orthogonal principal axes e1, e2, and e3 in descending order of eigenvalues. e1 is the length axis, e2 is the principal diameter axis, and e3 is the principal diameter axis. Then, for each three-dimensional point P... i Perform a one-dimensional projection on the three principal axes e1, e2, and e3:

[0037] k = 1, 2, 3, so {t} 1，i} is a series of scalar coordinates along the length axis; {t 2，i},{t 3，i} represents scalar coordinates on two cross-sectional axes, and quantile extrema are used to reduce noise.

[0038] L = Q 98 (t1)-Q2(t1)

[0039] D1 = Q 98 (t2)-Q2(t2)

[0040] D2 = Q 98 (t3)-Q2(t3);

[0041] Let D = (D1 + D2) / 2 as the equivalent diameter D;

[0042] Approximate volume V cap :

[0043]

[0044] Calculate the fusion volume

[0045]

[0046] In the formula, ω is a coefficient, ω∈[0,1];

[0047] Calculate the projected area A:

[0048] The sea cucumber target region is extracted from the detection result image. A small local image is cropped from the color image according to the detection box. Then, a small depth block is cropped from the depth image based on the same pixel coordinates. Non-sea cucumber pixels within the local image are set to zero or removed to obtain a clean mask M(u,v). The projected micro-area of ​​each pixel (u,v) within the mask at the corresponding depth Z(u,v) is calculated.

[0049]

[0050] Then, by summing up, we obtain the projected area A:

[0051]

[0052] Calculate the cross-sectional area R of the sea cucumber:

[0053]

[0054] Where D is the equivalent diameter.

[0055] A third aspect of this disclosure provides a control method for a machine vision-based sea cucumber harvesting robot. The control method is a sliding mode predictive control (SMPC) method that integrates MPC and sliding mode control. The SMPC method has a sliding mode prediction model, utilizes the future prediction information of sliding variables, and combines the rolling optimization mechanism in MPC to obtain the optimal control input by solving an optimization problem with constraints.

[0056] Preferably, the sliding mode predictive control (SMPC) method includes the following steps:

[0057] Step 1: First, based on the pose error information obtained from the vision system, the desired acceleration and angular velocity inputs are determined using the SMPC algorithm;

[0058] Step 2: Solve the control input using the robot hydrodynamic model and motor power feedback information to obtain the actual motor output;

[0059] Step 3: Use IMU measurement data to estimate and optimize the inertial motion state, further improving the accuracy of pose calculation;

[0060] Step 4: Finally, the attitude angle information is combined to complete the real-time coordinate transformation, thereby achieving accurate tracking of the target trajectory.

[0061] A fourth aspect of this disclosure provides a method for grading sea cucumbers, comprising the following steps:

[0062] The first step is to establish a sea cucumber target detection model based on convolutional neural networks;

[0063] The second step involves using a depth camera to capture images of the seabed environment, obtaining color and depth images.

[0064] The third step is to preprocess the color image;

[0065] The fourth step is to input the preprocessed color image into the sea cucumber target detection model to identify the sea cucumber target area and obtain the detection result image.

[0066] The fifth step is to perform pixel-level registration between the detection result image and the depth image obtained in the second step, based on the sea cucumber detection box in the detection result image, and extract the three-dimensional point cloud data corresponding to the target area.

[0067] Step 6: Estimation of quantitative indicators based on 3D convex hull;

[0068] The seventh step is to grade the sea cucumbers according to quantitative indicators.

[0069] The beneficial effects of this disclosure are: it enables automated harvesting of sea cucumbers using robots, replacing manual labor and resulting in high harvesting efficiency; the entire suction process does not damage the sea cucumbers, and the harvesting accuracy is high, targeting only the sea cucumbers; because the robot hovers above the seabed, it does not damage the seabed or the seaweed and other sea cucumber growth environments, making it suitable for seabeds with reefs and silt. The use of a non-contact suction device achieves flexible harvesting of sea cucumbers, avoiding damage during the harvesting process.

[0070] The machine vision system identifies target sea cucumbers and can classify them for targeted harvesting. It can also select and catch larger sea cucumbers for precise harvesting.

[0071] The robot's stable hovering in a specific position ensures accurate suction and improves harvesting efficiency. This high-precision hovering operation in the water avoids the damage to the seabed ecosystem caused by traditional walking robots.

[0072] By utilizing sliding mode predictive control (SMPC) technology, the robot maintains strong anti-interference capabilities and stable motion performance in complex environments (such as reefs, silt, and dynamic water flow disturbances), significantly improving fishing efficiency and success rate.

[0073] It can adapt to underwater environments with reefs and silt.

[0074] Further features and aspects of this disclosure will be clearly described in the following detailed description with reference to the accompanying drawings. Attached Figure Description

[0075] Figure 1 This is an isometric drawing of a sea cucumber harvesting robot;

[0076] Figure 2 yes Figure 1 A top view of the sea cucumber harvesting robot shown.

[0077] Figure 3 yes Figure 1 The rear view of the sea cucumber harvesting robot shown.

[0078] Figure 4 yes Figure 1 The image shows a bottom view of the sea cucumber harvesting robot.

[0079] Figure 5 yes Figure 1 The image shows a front view of a sea cucumber harvesting robot.

[0080] Figure 6 yes Figure 1 The right view of the sea cucumber harvesting robot shown;

[0081] Figure 7 yes Figure 1 Left view of the sea cucumber harvesting robot shown;

[0082] Figure 8 This is an isometric view of the rear of the sea cucumber harvesting robot from a downward perspective.

[0083] Figure 9 This is an axonometric view of the rear top view of the sea cucumber harvesting robot;

[0084] Figure 10 This is an axonometric view of the front-facing top view of the sea cucumber harvesting robot;

[0085] Figure 11 This is an isometric drawing of a sea cucumber harvesting robot;

[0086] Figure 12 This is a schematic diagram of the structure on which the flexible hose is installed on the sea cucumber harvesting robot;

[0087] Figure 13 This is a schematic diagram of a collection chamber and pump installed on the hull of an unmanned underwater vehicle.

[0088] Figure 14 It is a control process flowchart based on the SMPC method;

[0089] Figure 15 This is the control flowchart for an unmanned underwater vehicle;

[0090] Figure 16 It is an information processing flow based on machine vision;

[0091] Figure 17 These are images captured by a depth camera;

[0092] Figure 18 Yes Figure 17 The image shown is the preprocessed image;

[0093] Figure 19 This is a visual recognition process. After the recognition process, Figure (b) is obtained from Figure (a). Figure (c) is the result of semantic segmentation of Figure (b).

[0094] Figure 20 This is a visual recognition process. After the recognition process, Figure (b) is obtained from Figure (a). Figure (c) is the result of semantic segmentation of Figure (b).

[0095] Figure 21 This is a visual recognition process. After the recognition process, Figure (b) is obtained from Figure (a). Figure (c) is the result of semantic segmentation of Figure (b).

[0096] Figure 22 This is a flowchart of the hierarchical processing procedure;

[0097] Figure 23 This is a flowchart related to sliding mode predictive control;

[0098] Figure 24 It is the original YOLOv8 object detection model;

[0099] Figure 25 It is the YOLOv8 object detection model improved with the addition of CABM;

[0100] Figure 26 It is the YOLOv8 object detection model improved by adding a mask.

[0101] Explanation of symbols in the diagram:

[0102] 100. Unmanned underwater vehicle; 100-1. Hull; 200. First depth camera; 300. Parallel robotic arm; 301. End effector; 302. Top connecting plate; 302-1. Hose connection hole; 400. Second depth camera; 1. Hose; 2. Suction pipe; 3. Collection chamber; 4. Screen; 5. Pump; 6. Pipe; 7. Drain pipe. Detailed Implementation

[0103] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0104] The specific embodiments described below are merely preferred embodiments of this application, and the scope of protection of this application is not limited thereto. Those skilled in the art can make modifications or variations based on the principles, concepts, and spirit of this application, and the resulting technical solutions should all be covered within the scope of protection of this application.

[0105] like Figures 1-10 As shown, the machine vision-based sea cucumber harvesting robot includes an unmanned underwater vehicle 100, a first depth camera 200, a parallel robotic arm 300, and a second depth camera 400. The parallel robotic arm 300 is connected to the bottom of the hull of the unmanned underwater vehicle 100, and is positioned below the unmanned underwater vehicle 100. The first depth camera 200 is mounted on the front of the unmanned underwater vehicle 100, and the second depth camera 400 is mounted on the end cap 301 of the parallel robotic arm 300.

[0106] The 100 unmanned underwater vehicle is a specific conventional product, which can be an autonomous underwater vehicle (AUV) or a remotely operated vehicle (ROV).

[0107] like Figure 10 , 12 As shown in Figure 13, the top of the parallel robotic arm 300 is a top connecting plate 302. The top connecting plate 302 is provided with a hose connection hole 302-1. The upper part of the hose 1 is fixedly connected to the hose connection hole 302-1, and the lower end of the hose 1 is fixedly connected to the end 301 of the parallel robotic arm 300. The suction tube 2 is fixedly connected to the end 301, and the suction tube 2 communicates with the lower end of the hose 1 (see reference). Figure 10 The suction tube 2 and the lower end of the hose 1 can be connected through the central through hole at the end 301. The unmanned underwater vehicle 100 is equipped with a shell 100-1. The upper part of the hose 1 passes through the hose connection hole 302-1 and then passes upward through the bottom of the shell 100-1. A collection chamber 3 and a pump 5 are installed at the bottom of the shell 100-1. The collection chamber 3 has an inlet and an outlet. The upper end of the hose 1 is connected to the inlet of the collection chamber 3. The outlet of the collection chamber 3 is connected to the inlet of the pump 5 through a pipe 6. The drain pipe 7 is connected to the outlet of the pump 5. A screen 4 is installed in the inner cavity of the collection chamber 3, near the outlet of the collection chamber 3. The lower end of the hose 1 is connected to the suction tube 2 for sucking up the target sea cucumber. The screen 4 is used to intercept the sea cucumber.

[0108] The parallel robotic arm 300 can adjust the position of the suction tube 2 to align with the target sea cucumber. During the movement of its end effector (adjusting the position of the suction tube 2), the special structure of the parallel robotic arm does not affect the overall robot's hovering stability; the robot's hovering position remains unchanged. Compared to using a serial robotic arm, this effectively ensures the stability of the robot's hovering. When the pump 5 operates, the hose 1 generates suction, drawing seawater from the suction tube 2. The seawater flows through the hose 1 into the collection chamber 3, then exits from the outlet of the collection chamber 3 through the pipe 6, and finally is discharged from the drain pipe 7.

[0109] When the sea cucumber harvesting robot is working, the controller controls the unmanned underwater vehicle 100 to move in the sea to the seabed near the sea cucumber area. The unmanned underwater vehicle 100 hovers above the sea cucumber to be harvested and collects images of the seabed through the first depth camera 200. The controller identifies the sea cucumber based on the collected images, and then controls the parallel robotic arm 300 to move the end of the suction tube 2 close to the target sea cucumber. Then the pump 5 is activated, and the target sea cucumber is sucked into the hose 1 along with the seawater through the suction tube 2. The target sea cucumber then enters the collection chamber 3, and the collection chamber 3 outputs the sea cucumber from its outlet. Due to the action of the screen 4, the sea cucumber is intercepted in the inner cavity of the collection chamber 3, thereby realizing the harvesting of the sea cucumber.

[0110] As can be seen, the entire suction process does not damage the sea cucumbers, and the harvesting accuracy is high, targeting only sea cucumbers. Because the robot hovers above the seabed, it does not damage the seabed or the seaweed and other growth environments of sea cucumbers, making it suitable for seabeds with reefs and silt. Furthermore, the harvesting efficiency is high. In addition, machine vision can identify and classify target sea cucumbers, including by size, allowing for selective harvesting of larger sea cucumbers for precise targeting. The robot's stable hovering in one position ensures both accuracy and efficiency in the harvesting process.

[0111] To enable robots to achieve precise and efficient movement in complex underwater environments, this invention employs an improved sliding mode predictive control (SMPC) algorithm that combines model predictive control (MPC) and sliding mode control (SMC) to enhance the performance of underwater robot trajectory tracking and stability control.

[0112] Traditional model predictive control (MPC) methods predict the future state of a system based on a system process model and determine the optimal control input through rolling optimization to achieve high-precision tracking of the desired trajectory by the system output. However, because MPC algorithms require high model accuracy, the uncertainty of model parameters significantly reduces control effectiveness in complex and frequently disturbed underwater environments. Meanwhile, while sliding mode control (SMC) possesses strong robustness and is suitable for uncertainties in underwater environments, its control output is prone to high-frequency chattering, leading to increased energy consumption and accelerated mechanical wear, thus affecting its practical engineering application performance.

[0113] To address the above problems, this invention proposes a sliding mode predictive control (SMPC) method that integrates MPC and sliding mode control. The SMPC algorithm of this invention designs a sliding mode prediction model, utilizes future prediction information of sliding mode variables, and combines it with the rolling optimization mechanism in MPC to obtain the optimal control input by solving a constrained optimization problem. This method combines the model prediction advantages of the MPC algorithm with the anti-interference characteristics of sliding mode control, effectively suppressing the chattering problem in traditional sliding mode control and improving the stability and accuracy of the control system in complex underwater environments.

[0114] The specific implementation process is as follows:

[0115] refer to Figure 14 As shown, after the program starts, it first loads environment variables, mainly including controller parameters and sensor data storage structures (e.g., creating a dictionary to store data from the IMU, sonar, depth sensor, and Doppler velocimeter). Simultaneously, it performs initial calibration and static error compensation for the inertial measurement unit (IMU) to reduce the impact of measurement errors on control accuracy.

[0116] In addition to the IMU and depth sensor, this invention adds a motor power sensor to the robot hardware. By measuring the motor power in real time and feeding it back, the current actual output state of the motor can be calculated. Feeding the motor output power information back to the controller can further correct and optimize the robot's motion model parameters, improving the accuracy of trajectory tracking and pose closed-loop control.

[0117] The specific control implementation process is as follows:

[0118] Step 1: First, based on the pose error information obtained from the vision system, the desired acceleration and angular velocity inputs are determined using the SMPC algorithm;

[0119] Step 2: Solve the control input using the robot hydrodynamic model and motor power feedback information to obtain the actual motor output;

[0120] Step 3: Use IMU measurement data to estimate and optimize the inertial motion state, further improving the accuracy of pose calculation;

[0121] Step 4: Finally, the attitude angle information is combined to complete the real-time coordinate transformation, thereby achieving accurate tracking of the target trajectory.

[0122] The sliding mode predictive control (SMPC) algorithm proposed in this invention not only effectively solves the chattering problem of traditional sliding mode control, but also overcomes the problem of excessively high model accuracy requirements of traditional MPC algorithms. This enables the robot to have better trajectory tracking performance, anti-interference ability and robustness in complex and uncertain underwater environments, and significantly improves the efficiency and reliability of sea cucumber harvesting operations.

[0123] The dynamic equations of the unmanned underwater vehicle involved in the process are as follows:

[0124]

[0125] Vector v = [u,v,w,p,q,r] T This is the velocity vector in the fixed coordinate system of the aircraft, where u, v, w, p, q, and r correspond to sway, heave, roll, pitch, and yaw, respectively. The vector η = [x, y, z, φ, θ, ψ] T It is a vector combining displacement and Euler angles relative to the inertial coordinate system. Vector τ = [X, Y, Z, K, M, N] T This includes forces and torques acting on the organism. Vector τ d This represents disturbances in the underwater environment, including the impact of the fishing equipment on the vessel. Other parameters and variables used are shown in the table below:

[0126]

[0127] The specific calculation formula for the coordinate transformation matrix is ​​as follows:

[0128]

[0129] Assuming the origin of the body coordinate system is located at the body's center of gravity, the restoring force matrix vector operation can be simplified to:

[0130]

[0131] W = mg, These are gravity and buoyancy, respectively (ρ is the density of water). Let (xb, yb, zb) be the volume of fluid displaced by the robot, and (xb, yb, zb) be the coordinates of the buoyancy center in the B coordinate system. Unmanned underwater vehicles are typically controlled by multiple fixed thrusters. To correlate the vector τ with the voltage V applied to each thruster... i Connecting them, we have:

[0132] τ=TK(s)F(V)

[0133] T = [t1, ...,t] r ]

[0134]

[0135] V is the column vector of the thruster voltages, matrix T is the thrust distribution matrix, and column vector ti represents the force F generated by each thruster. i Associated with vector τ; where ε i It represents the unit vector in the direction of thruster i, and is the force F. i The distance between the point of application and the center of the b-coordinate system. The diagonal matrix K(s) contains the unit DC gain transfer function, representing the dynamic relationship between Vi and force Fi. Since voltage is difficult to measure and feedback in actual control, power is used to describe the motor output. Power, as the product of voltage and current, directly reflects the motor's output capability. By monitoring and controlling power, the motor's output state can be indirectly controlled, ensuring system stability and efficiency. Propeller thrust formula: F = K T ·ρ·n 2 ·D 4 K T The thrust coefficient is typically determined through open-water propeller testing. ρ represents the fluid density, and D represents the propeller diameter.

[0136] The relationship between the power and thrust of the motor is as follows: Where K Q η represents the torque coefficient (related to propeller design), and η represents the motor efficiency. We can obtain...

[0137] Here we let You can get

[0138] In sliding mode control, assume the sliding mode function has the following form:

[0139] s(k)=C·Δx(k)

[0140] Assuming the nominal model of the discrete system is used as the prediction model, the sliding mode function value at future time k+p can be predicted. The predicted value of the sliding mode function at future time can be derived from the nominal model. The key to this method is to use the prediction model to evaluate the future trend of the sliding mode function, thereby designing appropriate control inputs to ensure the robustness and stability of the system. Using the nominal model of the discrete system as the prediction model, the predicted sliding mode function value at future time k+p is...

[0141]

[0142] The sliding mode prediction value of time k at time (kp) can be derived as follows:

[0143]

[0144] In light of practical considerations, to correct the predicted sliding mode value s(k+p), an error is defined between the actual value s(k) of the sliding mode variable and its corresponding predicted value s(k / kp). This error is used to obtain the corrected output value of the sliding mode variable.

[0145]

[0146] The simplified formula can be obtained

[0147]

[0148] The optimization cost function is defined as:

[0149]

[0150] s r (k+1) is the sliding mode reference trajectory, and t j and d l These are the weighting coefficients. The model for a sliding mode reference trajectory is typically:

[0151] s r (k+p)=s0(k+p=0 m×1 ,S r (k+1)=0 Nm×1

[0152] However, excessively fast convergence speeds can cause overshoot, so alternative sliding mode reference paths can be chosen to increase design freedom.

[0153]

[0154] By analyzing the objective function J p Take the derivative and set its partial derivative with respect to the control input to zero, i.e. The corresponding optimal control law can be obtained. The control input is then derived through reasoning.

[0155] U(k)=-(Θ T Θ+G) -1 Θ T [Ξx(k)+O p E(k)-S r (k+1)]

[0156] In rolling optimization, only the first element of the current control input signal U(k) is executed, and the remaining elements are not executed, thus obtaining:

[0157] u(k) = -[1, 0, ..., 0]T (Θ T Θ+G) -1 Θ T [Ξx(k)+O p E(k)-S r (k+1)]

[0158] In the next round of optimization, u(k) will be used as the initial value. When the system state reaches the sliding surface, the optimal control input u(k) is provided at each step to minimize chattering by obtaining and analyzing the prediction error.

[0159] The main implementation methods of SMPC (Sliding Mode Predictive Control) in discrete-time systems are listed in the table below.

[0160]

[0161] After the unmanned underwater vehicle 100 enters the seabed operating area, it uses a first depth camera 200 installed at its front end to scan and identify a large target area ahead, and uses a machine vision-based target detection algorithm to determine the position and size of the target sea cucumber. Subsequently, the unmanned underwater vehicle 100 hovers directly above the target sea cucumber, and uses a second depth camera 400 installed at the end of a parallel robotic arm 300 to further pinpoint the target sea cucumber's location. Based on the real-time positioning information obtained from the second depth camera 400, the parallel robotic arm 300 is controlled to adjust the suction tube 2 to precisely approach the target sea cucumber. Once the suction tube 2 is accurately positioned above the target sea cucumber, the pump 5 is activated, and the target sea cucumber, along with seawater, is sucked into the hose 1 through the suction tube 2. The target sea cucumber then enters the collection chamber 3, which outputs the sea cucumber through its outlet. Due to the action of the screen 4, the sea cucumber is intercepted within the cavity of the collection chamber 3, thus achieving the harvesting of the sea cucumber.

[0162] The parallel robotic arm 300 of this invention has the advantages of high precision and fast dynamic response during position adjustment; the unmanned underwater vehicle 100 adopts hovering operation to ensure accurate positioning of sea cucumbers in complex seabed environments (such as reefs, silt, and dynamic currents), avoiding damage to the seabed ecosystem; through real-time recognition and size classification based on machine vision, precise harvesting can be achieved, only harvesting sea cucumbers that meet the size requirements. Therefore, the entire harvesting process does not damage the sea cucumbers, and the harvesting accuracy is high, targeting only sea cucumbers of suitable size.

[0163] In the entire operation of sea cucumber harvesting robots, sea cucumber identification is the prerequisite and core link for achieving automated and intelligent harvesting. Because harvesting operations have strict requirements on the size, quality, and integrity of sea cucumbers, there is an urgent need for an underwater vision system with artificial intelligence capabilities. This system can intelligently identify target sea cucumbers, just like manual selection, accurately obtain their size and spatial location information, and comprehensively judge their quality. By sending harvesting instructions to the executing agency in real time, it ensures that only sea cucumbers meeting the standards are captured, thereby achieving precise and efficient automated harvesting, balancing operational efficiency with sustainable resource utilization.

[0164] For the entire sea cucumber harvesting process using robots, sea cucumber identification is the prerequisite and key to the automation and intelligence of sea cucumber harvesting. Strict requirements are placed on the size, quality, and integrity of the harvested sea cucumbers. There is an urgent need for an underwater vision system that can intelligently select sea cucumbers like humans and effectively guide precise operations. This system can accurately identify sea cucumbers, obtain their size and location, judge their quality, and send harvesting instructions to the execution device, thereby obtaining sea cucumbers that meet the requirements while maintaining high harvesting efficiency. (Reference) Figure 16 .

[0165] Fishing methods include:

[0166] The first step is to establish a sea cucumber target detection model based on convolutional neural networks.

[0167] Step S101: Create an underwater image dataset for training the target detection model.

[0168] Step (1) Obtain the dataset GroundTruth, which contains real underwater images, from public online databases (such as images of nearshore aquaculture areas released by the Underwater RobotPicking Contest).

[0169] Step (2) involves performing underwater imaging characteristic simulation processing on all images in the GroundTruth dataset to reproduce the impact of the underwater environment on image quality.

[0170] The simulations included: simulating the absorption effect of water on light of different wavelengths, resulting in color shift in the image channels; considering the effects of scattering, refraction, and suspended particles on image sharpness, causing image blur; and simulating the attenuation of imaging intensity and reduction in contrast as the shooting depth increases.

[0171] The specific process of simulation processing is: spectral absorption + backscattering → forward scattering blurring → camera image signal processing pipeline.

[0172] Define the image before simulation processing as i gt The pixel depth is Z(u,v), and the channel attenuation coefficient is c.λ The environmental item is A. λ .

[0173] First, the direct and backscattered light combine (color shift / haze). First, calculate the transmittance using Beer-Lambert's law:

[0174] t λ (u, v) = exp(-c λ Z(u, v))

[0175] Based on this, direct sunlight and fog are combined according to their physical proportions:

[0176] I λ (u,v)=I gt，λ (u,v)t λ (u, v) + A λ (u, v)[1-t] λ (u, v)]

[0177] A λ Estimation method: Given explicit depth information Z(u,v): Distance estimation. In low-contrast regions of the distance, t is approximately 0. The weighted median of pixels in this region, w = 1 - t, is used to obtain A. λ The estimated value. UDCP is used when explicit depth information is lacking. The top-k pixels of intensity from the original image are selected from a very small candidate set of G / B dark channels, and their RGB median is taken as A. λ The estimated value.

[0178] c λ Calibration methods, when there is a reference reflection or a near-range baseline:

[0179]

[0180] Among them, I λ To measure the pixel mean, I 0,λ Baseline values ​​obtained from close-range underwater measurements or from measurements in the air.

[0181] Without reliable I 0,λ When on-site conditions are insufficient to provide a measured baseline, dual-depth calibration is performed:

[0182]

[0183] Secondly, forward scattering causes blurring, and the kernel scale is linearly related to depth:

[0184]

[0185] Gaussian kernels and convolution (spatial variable):

[0186]

[0187] I uw,λ =I λ*sv K λ

[0188] Spatially variable convolution means that in the same image, pixels farther from the camera use a 'larger blur kernel', and pixels closer to the camera use a 'smaller blur kernel'; therefore, the blur intensity changes continuously with the depth Z(u,v), rather than using the same blur level for the entire image.

[0189] Public datasets typically do not provide pixel-wise depth Z(u,v), in which case Z(u,v) is defined as the effective propagation distance in water; when building training data offline, Z(u,v) can be estimated by binocular or multi-view geometry or monocular networks, and when the true depth cannot be obtained, a proxy depth field Z' is used to parameterize the degradation intensity.

[0190] Then, the camera image signal processing pipeline:

[0191] S lin =WB(I uw ), I sim =Γ(clip(S) lin +n shot (S lin )+n read ))

[0192] Where Γ is the camera gamma map, S lin It is Poisson noise (which varies with signal strength), n read It's camera noise (which can be understood as Gaussian noise from the sensor, and doesn't change with signal strength), WB(): white balance first, then exposure normalization, I uw This is the image after forward scattering (before passing through the camera pipeline).

[0193] This yields the simulated sample image Isim.

[0194] It can be seen that the single depth field Z(u,v) of the same pixel simultaneously drives the transmittance (which determines the ratio of "real information / fog") and the blur radius (which determines the intensity of "scattering and diffusion"). Moreover, the proportional constant of the blur radius and the channel absorption coefficient can be artificially coupled to ensure that the physical fact that "the more turbid the water becomes, the more color-biased and blurry it becomes" remains consistent within the model.

[0195] Step (3) restores the simulated sample image by using a lightweight Visual Transformer model.

[0196] A joint correction method is used to address blue-green color shift, contrast reduction, and scattering blur caused by spectral absorption and volume scattering in underwater images.

[0197] A mathematical model is constructed to describe the degradation process of underwater images, and the images are restored by inverting the model.

[0198] This gives us the restored sample image.

[0199] Step (4) Select images (i.e., unprocessed images) from the GroundTruth dataset at a certain ratio (e.g., 20%), and combine them with the restored sample images to form the underwater image dataset Train for training the target detection model.

[0200] Step S102: Use X-AnyLabeling image segmentation software to perform semantic segmentation and annotation on sea cucumber targets in the underwater image dataset Train, extract sea cucumber edge features, and construct a high-quality semantic segmentation training set.

[0201] Step S103: Construct an improved YOLOv8 target detection model for accurate identification of sea cucumbers.

[0202] Building upon the existing YOLOv8 object detection model, this paper introduces a CABM (Convolutional Block Attention Module) to replace the traditional convolutional (Conv) module, thereby enhancing the model's feature extraction capabilities. CABM, as an attention mechanism that integrates both channel and spatial visual cues, improves the model's perception of key regions through a hierarchical feature enhancement strategy. This module consists of a channel attention submodule (CAM) and a spatial attention submodule (SAM) sequentially connected, with its core function being the establishment of a multi-dimensional adaptive feature selection mechanism. Figure 25 As shown.

[0203] A prior mask detection method based on depth map distance gradient is also proposed to improve the localization accuracy of the detection model. In actual underwater scenarios, there is usually a significant depth gradient change between the target sea creature and the background. This characteristic is used to construct a detection prior region (mask). The detection box density is increased within the mask region to form a mask detection module, thereby improving recognition efficiency and accuracy. The specific working method of the mask detection module is as follows: First, the depth map gradient is calculated along the horizontal (x-direction) and vertical (y-direction) using the Sobel or Scharr operator, respectively. Next, based on the gradient magnitude, an appropriate threshold is set to generate a binary mask image, with the region with the larger gradient serving as the prior region for target detection. Then, morphological processing (erosion and dilation) and connected component analysis are performed on the binary mask to remove noise and fill small holes, ensuring the accuracy of the mask. Figure 26 As shown.

[0204] With the help of the Mask detection module, the target region can be located more accurately, the number of redundant candidate boxes can be significantly reduced, and the overall performance of the target detection model can be improved.

[0205] The Mask detection module, acting as a "depth prior branch," runs parallel to the image feature backbone. It generates a prior Mask based on the depth map, and then outputs this prior to the input, features, sampling, and post-processing of the detection network.

[0206] The Mask detection module calculates the Sobel / Scharr gradient of the depth map Z, performs magnitude normalization, adaptive thresholding (Otsu / quantile), morphological cleaning, and connected component filtering to obtain a binary Mask; it can also output a soft prior map (gradient magnitude smoothing map).

[0207] Output destination: M is concatenated with RGB as the 4th channel (RGB+M) and fed into the improved YOLOv8; the first layer convolution automatically learns how to use priors.

[0208] Step S104: Use the semantic segmentation training set obtained in step 102 to train the improved YOLOv8 target detection model to obtain the sea cucumber target detection model.

[0209] The semantic segmentation training set obtained in step 102 is used as input to construct the improved YOLOv8 object detection model. The semantic segmentation training set is divided into a uniform size (e.g., 640×640) and a conventional ratio (e.g., 8:2). In terms of model structure, several convolutional blocks in the Backbone / Neck are replaced or concatenated with attention convolutional blocks that integrate CABM (channel attention CAM, sequential cascaded spatial attention SAM). This allows the network to adaptively strengthen key regions related to sea cucumber contours and textures and suppress background interference during the feature extraction stage. At the same time, the gradient magnitudes in the horizontal and vertical directions are calculated based on the depth map. After thresholding, morphological cleaning, and connected component filtering, a binary prior Mask is obtained and injected into the entire training and inference process. The Mask can be used as an additional channel input along with RGB, or point-by-point gating can be implemented on the feature map of the Neck to improve the feature response of the Mask region. During training, the YOLOv8 backbone loss design is still used (such as bounding box regression with CIoU / SIoU+DFL and classification / targeting with BCE / Focal), and higher weights are given to pixels or candidates within the Mask to guide the allocation of positive samples and the concentration of gradients in the region of abrupt change in the target depth gradient, thereby reducing redundant candidates and false detections.

[0210] The second step involves the robot's first depth camera 200 acquiring images of the seabed environment, including color and depth images, which are then input into the controller for further processing.

[0211] The third step is for the controller to preprocess the color image.

[0212] Due to the varying absorption of different wavelengths of light by seawater, underwater images commonly exhibit blue-green color casts and low contrast. To improve underwater image quality, image preprocessing is performed. The specific preprocessing method involves adjusting the histograms of both the RGB and HSV models of the color image, and then fusing the adjustments to obtain the preprocessed image. Experiments show that this method effectively improves image contrast, significantly reduces the impact of the "blue-green effect," and minimizes over-enhanced or under-enhanced areas in the image.

[0213] It is evident that improving image quality contributes to recognition accuracy.

[0214] Another preprocessing method is UDCP (Underwater Dark Channel Prior), a single-image-based method that does not require training. Underwater environments strongly absorb red light, resulting in a bluish-green tint, low contrast, and distant objects appearing as fog. UDCP constructs the dark channel using only the blue / green channels when estimating the "fog," avoiding the interference of the naturally dark red channel, thus more accurately estimating ambient light and transmittance. Subsequently, the "fog" is subtracted according to the imaging model, and the true colors and brightness are restored, recovering color, improving contrast, and enhancing details. It is fast, easy to implement, and often used as a preprocessing method for underwater vision, significantly improving the stability and accuracy of subsequent detection / segmentation (such as improvements to YOLOv8).

[0215] The fourth step involves inputting the preprocessed color image into the sea cucumber target detection model to identify the sea cucumber target region and obtain the detection result image, which displays the sea cucumber detection bounding box, such as... Figure 19 As shown, Figure (a) is the preprocessed image, and Figure (b) is the detection result. The detection result also includes confidence level.

[0216] The fifth step involves performing pixel-level registration between the detected sea cucumber detection bounding box (edge ​​mask region) and the depth image obtained in the second step to extract the 3D point cloud data corresponding to the target region.

[0217] The RGB image of the detection box region in the detection result image is registered pixel-by-pixel with the depth image, and then based on the camera intrinsic parameters (f x f y c x c y This maps the pixels (u,v,Z) of the registered image to three-dimensional points in the camera coordinate system. Extract the 3D point cloud corresponding to the target region.

[0218] Next, to improve the quality of point cloud data and the accuracy of subsequent size calculations, the 3D point cloud is filtered to remove isolated points, noise points, and abnormal data, resulting in an accurate and reliable point cloud of the target region.

[0219] Step 6: Size estimation based on the 3D convex hull.

[0220] Step S601: The point cloud data P of the target area obtained in step 5 is processed incrementally. f ={P i ∈R 3 A three-dimensional convex hull calculation is performed to obtain the three-dimensional convex hull structure of the sea cucumber. The surface of the three-dimensional convex hull structure is a closed triangular mesh T with uniform normals.

[0221] The triangular mesh T contains several triangular faces, where each triangular face f∈T has vertices (v0, v1, v2) and an outward normal of n. f The plane offset is d f The centroid of the triangular mesh T is P0.

[0222] Step S602, calculate volume V hull .

[0223] Volume V hull It is calculated through tetrahedral decomposition:

[0224] Using the centroid P0 of the mesh as a reference point, and combining each triangular facet (v0, v1, v2) of the triangular mesh T to form a tetrahedron (P0, v0, v1, v2), then perform volume accumulation:

[0225]

[0226] Step S603, calculate the approximate volume V cap .

[0227] The target region point cloud P obtained in step 5 f ={P i ∈R 3 Perform principal component analysis (PCA).

[0228] The covariance ∑ is calculated using the following formula:

[0229]

[0230] In the formula, μ is the center of mass.

[0231] Perform eigenvalue decomposition on ∑ and obtain three orthogonal principal axes e1, e2, and e3 according to the eigenvalues ​​from largest to smallest. e1 is the length axis, e2 is the principal diameter axis, and e3 is the principal diameter axis.

[0232] Subtracting the centroid allows PCA to work on a dispersion "about the geometric center of the object", resulting in principal axes and dimensions consistent with the shape.

[0233] P of each three-dimensional point i (Subtracting the centroid) Perform a one-dimensional projection on the three principal axes e1, e2, and e3:

[0234] k = 1, 2, 3, so {t} 1,i} is a series of scalar coordinates "along the length axis"; {t 2,i},{t 3,i} represents the scalar coordinates on the two cross-sectional axes. Directly using the minimum / maximum values ​​is easily affected by noise, so we use quantile extrema (e.g., 2% and 98%) to mitigate noise.

[0235] L = Q 98 (t1)-Q2(t1)

[0236] D1 = Q 98 (t2)-Q2(t2)

[0237] D2 = Q 98 (t3)-Q2(t3)

[0238] Here Q p () indicates the "p percentile". The unit is meters.

[0239] In the case of sea cucumbers being approximately axisymmetric, we take D = (D1 + D2) / 2 as the equivalent diameter D.

[0240] Taking the calculation of L as an example, the projections of all point clouds along the length axis e1 are taken out to form a series of scalars t1. Arrange them on the number axis and take their 98th percentile - 2th percentile. Here, the empirical quantile in probability statistics means that 98% (or 2%) of the samples are not greater than this value. The maximum and minimum values ​​are not taken to avoid noise interference in the calculation process.

[0241] If we approximate a sea cucumber as a capsule with a "cylindrical center and hemispherical ends", then its approximate volume V cap :

[0242]

[0243] Step S604, calculate the fusion volume

[0244] V cap For point estimation of shape prior, V hull As a geometric upper bound, the two are combined to form a volume squeeze and weighted fusion:

[0245]

[0246] In the formula, ω is a coefficient, ω∈[0,1].

[0247] Step S605: Calculate the projected area.

[0248] Extract the sea cucumber target region (i.e., the detection box region) from the detection result image obtained in step four. Since the coordinates of the top left corner, bottom right corner, and confidence level of the detection box are given, a small local image (ROI, Region of Interest) is cropped from the color image according to this detection box. To prevent the edges from being cut off, the edges can be expanded outward by about 10-15% and confined within the image boundary.

[0249] Then, based on the same pixel coordinates, a small depth patch is cropped from the depth image obtained in the second step.

[0250] By enabling "depth prior Mask / segmentation", non-sea cucumber pixels in the local small image are set to zero or removed to make the target cleaner and obtain a clean mask M(u,v).

[0251] For each pixel (u,v) within the mask, calculate the projected micro-area at the corresponding depth Z(u,v):

[0252]

[0253] Then, by summing up, we obtain the projected area A:

[0254]

[0255] Step S606: Calculate the cross-sectional area of ​​the sea cucumber.

[0256] Because sea cucumbers are approximated as "capsule bodies" (cylinder in the middle + hemispheres at both ends).

[0257] The cross-sectional area R of a sea cucumber can be calculated using the following formula:

[0258]

[0259] Where D is the equivalent diameter.

[0260] Step S606: Estimate the weight of the sea cucumber.

[0261] The sea cucumber's length L, equivalent diameter D, projected area A, and volume V are considered together. hull Approximate volume V cap Form the feature vector s = [L,D,A,V] hull V cap A weight regression model W = f(s) is established based on the pre-collected measured calibration dataset to estimate the weight of the target sea cucumber.

[0262] The process of building a regression model:

[0263] First, prepare n sea cucumber samples. For each sea cucumber sample, acquire color and depth images. Then, process the images to obtain the sea cucumber sample data s = [L, D, A, V]. hull V cap Weigh each sea cucumber sample to obtain its actual weight data W; n s form S. n n W's make up W'' n Thus forming sample information {s n W n}

[0264] Secondly, volume estimation error suppression: calculating the fusion volume

[0265]

[0266] In the formula, ω is a coefficient, and ω∈[0,1].

[0267] Then, calculate the predicted weight:

[0268]

[0269] In the formula, The coefficients to be calibrated are learned using Huber loss (with a threshold δ set to the 90th percentile of the training residuals) and L2 regularization; the training objective loss function is:

[0270]

[0271] regression model coefficient The solution is obtained in one step by minimizing the Huber loss and adding L2 regularization through convex optimization; the solution process and its optimization algorithm (such as IRLS, L-BFGS) are well-known conventional techniques.

[0272] How is the sea cucumber weight estimation model derived: For a single sea cucumber, the model is calculated based on density and volume.

[0273] The effective density of sea cucumber in its water-containing state (kg / m³) 3 ), and changes slowly with the shape and state of the body; V true The actual volume is m_additional; m_additional is a small amount summarizing "surface water film / silt and other adhering substances", which can be approximated as linearly related to the outer surface area.

[0274] The "plump / slender" shape of sea cucumbers alters their water content and tissue firmness, making... It changes slowly around a certain mean. The "shape factor" is dimensionless, and a first-order Taylor expansion is performed at the mean point:

[0275] L / D: Slimness ratio, A / LD (fullness)

[0276]

[0277] A model for estimating sea cucumber weight was obtained:

[0278] m 附加 It is a small amount summarizing "surface water film / silt, etc.", which can be approximately linearly related to the outer surface area and can be expressed as: m 附加 =γ1A+γ0

[0279] Therefore, the regression model for the weight of sea cucumbers can be taken as follows:

[0280]

[0281] Step 7: Grade the sea cucumbers according to their size and / or weight.

[0282] Size and weight are both quantitative indicators.

[0283] The first approach is to grade sea cucumbers according to their size, for example, dividing them into two grades: mature and juvenile. Or, dividing them into three grades: large, medium, and small.

[0284] For example, sea cucumbers can be graded based on their cross-sectional area R. The larger the cross-sectional area R, the larger the sea cucumber is considered. If the cross-sectional area R is greater than a certain threshold, it is considered mature; otherwise, it is considered juvenile.

[0285] For example, based on volume V hull To classify, volume V hull The larger the value, the larger the sea cucumber. Volume V hull If the size exceeds a certain threshold, it is considered mature; otherwise, it is considered juvenile.

[0286] For example, based on the approximate volume V cap To classify, approximate volume V cap The larger the value, the larger the sea cucumber. Volume V cap If the size exceeds a certain threshold, it is considered mature; otherwise, it is considered juvenile.

[0287] For example, based on the fusion volume To classify and merge volumes The larger the size, the larger the sea cucumber. (Fused volume) If the size exceeds a certain threshold, it is considered mature; otherwise, it is considered juvenile.

[0288] For example, sea cucumbers can be graded based on their length L. The larger the length L, the larger the sea cucumber is considered. If the length L is greater than a certain threshold, it is considered mature; otherwise, it is considered juvenile.

[0289] The second scenario involves grading sea cucumbers based on weight. For example, sea cucumbers could be divided into large, medium, and small grades. The weight obtained in step S606 would then be used as the basis for grading. To classify by weight The larger the size, the larger the sea cucumber.

[0290] weight If the size exceeds a certain threshold, it is considered mature; otherwise, it is considered juvenile.

[0291] The third method involves grading sea cucumbers based on two criteria: size and weight.

[0292] Calculate the effective density ρ eff :

[0293]

[0294] Effective density ρ eff The unit is kg / m^3.

[0295] According to weight Length L, equivalent diameter D, projected area A, volume V hull Volume V cap fusion volume Effective density ρ eff To calculate the composite maturity index:

[0296]

[0297] Map S to maturity probability using the Sigmoid function:

[0298]

[0299] when If it is determined to be a mature sea cucumber, it is determined to be a juvenile.

[0300] It is a percentage constant determined by humans based on the actual situation, such as being set to 95%.

[0301] Multi-level grading is possible (example: Grade I / II / III products), with weight categories designed according to market standards.

[0302] And introduce shape kernel If it meets the fishing standards, it is considered to meet the standards.

[0303] The specific steps for grading are as follows:

[0304] Hierarchical goals and symbols:

[0305] Predicted weight The data originates from the aforementioned white-box regression; it also possesses geometric features: length L, equivalent diameter D, projected area A, and volumes Vhull and Vcap. The fused volume...

[0306] Let K = 3 levels (which can be generalized to any K levels), the corresponding weight threshold sequence is: τ0 = 0 < τ1 < τ2 < τ3 = +∞ (unit: g)

[0307] When the company specifies minimum weight requirements for each level, τ1 and τ2 are directly taken as the corresponding thresholds; when no such requirement exists, the company specifies the target shipment ratio, and the operator uses the quantile method: given the ratios π1, π2, π3 (π1 + π2 + π3 = 1), such that... Q p The function () is mapped to Q mentioned in the previous steps. p (), represents the "p percentile".

[0308] Based on market standards or industry needs, weight tiers are designed; here, we take three tiers as an example: The weight candidate tier function is:

[0309]

[0310] Note: 1 → Level I, 2 → Level II, 3 → Level III; In the calibration data, the samples are first divided into intervals [τk, τk+1) according to the weight rank function. To prevent abnormal conditions of sea cucumbers from interfering with weight estimation, an indicator is used as a verification window. The specific data of this indicator is given according to the actual situation. In operation, the default value is...

[0311]

[0312] Define the level quantity

[0313]

[0314] The final level is

[0315] The eighth step is to carry out the sea cucumber harvesting operation.

[0316] Harvest mature sea cucumbers. Or harvest large sea cucumbers. Or harvest both large and medium-sized sea cucumbers. Or harvest according to grade.

[0317] It is evident that automated, efficient, and intelligent harvesting operations have been achieved, along with precise grading of sea cucumbers.

[0318] The instruction manual states that for step S101, which creates an underwater image dataset for training the target detection model, a public network database can be used instead of the robot's depth camera to directly measure the actual working conditions, collect image data, and then restore the collected image data using a lightweight Visual Transformer model. The restored sample images can then be used to train the target detection model.

[0319] The instruction manual indicates that for step six, the incremental method can be omitted, and the Quickhull algorithm can be used instead.

Claims

1. A sea cucumber harvesting robot based on machine vision, characterized in that, The system includes an unmanned underwater vehicle (UUV), a first depth camera, a parallel robotic arm, a hose, and a suction tube. The parallel robotic arm is connected to the bottom of the UUV's hull, and the first depth camera is connected to the front of the UUV. The parallel robotic arm has a top connecting plate with a hose connection hole. The upper part of the hose is fixedly connected to the hose connection hole, and the lower end of the hose is fixedly connected to the end of the parallel robotic arm. The suction tube is fixedly connected to the end of the robotic arm and communicates with the lower end of the hose. The UUV has a hull, and the upper part of the hose passes through the hose connection hole and then passes upward through the bottom of the hull. The bottom of the UUV's hull is connected to a collection chamber and a pump. The collection chamber has an inlet and an outlet. The upper end of the hose is connected to the inlet of the collection chamber, and the outlet of the collection chamber is connected to the inlet of the pump via a pipe. The drain pipe is connected to the outlet of the pump, and a screen is connected inside the collection chamber, near the outlet of the collection chamber.

2. The sea cucumber harvesting robot based on machine vision according to claim 1, characterized in that, The machine vision-based sea cucumber harvesting robot also includes a second depth camera, which is connected to the end of the parallel robotic arm.

3. A sea cucumber harvesting method using the machine vision-based sea cucumber harvesting robot as described in claim 1, characterized in that, Includes the following steps: The first step is to establish a sea cucumber target detection model based on convolutional neural networks; The second step involves the first depth camera acquiring images of the seabed environment, obtaining color and depth images, and then inputting the color and depth images into the controller. The third step is for the controller to preprocess the color image; The fourth step is to input the preprocessed color image into the sea cucumber target detection model to identify the sea cucumber target area and obtain the detection result image. The fifth step is to perform pixel-level registration between the detection result image and the depth image obtained in the second step, based on the sea cucumber detection box in the detection result image, and extract the three-dimensional point cloud data corresponding to the target area. Step 6: Estimation of quantitative indicators based on 3D convex hull; The seventh step is to grade the sea cucumbers according to quantitative indicators; The eighth step is to harvest the sea cucumbers.

4. The fishing method according to claim 3, characterized in that, The sea cucumber target detection model based on convolutional neural networks is an improved YOLOv8 target detection model that includes a CABM module.

5. The fishing method according to claim 4, characterized in that, The improved YOLOv8 object detection model also includes a Mask detection module.

6. The fishing method according to claim 4 or 5, characterized in that, The improved YOLOv8 object detection model is established in the following way: Step (1): Create an underwater image dataset for training the target detection model; Step 1): Obtain the GroundTruth dataset of real underwater images from a public online database; Step 2): For all images in the GroundTruth dataset, underwater imaging characteristics simulation processing is performed. After spectral absorption, backscattering, forward scattering blurring, and camera image signal processing pipeline, the sample image Isim is obtained. Step 3): Restore the simulated sample images using a lightweight Visual Transformer model; Step 4): Select images from the GroundTruth dataset according to a certain ratio, and together with the restored sample images, form the underwater image dataset Train for training the target detection model; Step (2) Semantic segmentation and annotation of sea cucumber targets in the underwater image dataset Train are performed, sea cucumber edge features are extracted, and a semantic segmentation training set is constructed. Step (3) train the improved YOLOv8 object detection model using the semantic segmentation training set.

7. The fishing method according to claim 3, characterized in that, The sixth step: For point cloud data P in the target area f ={P i ∈R 3 A 3D convex hull calculation is performed to obtain the 3D convex hull structure of the sea cucumber. The surface of the 3D convex hull structure is a closed triangular mesh T with uniform normals. The triangular mesh T contains several triangular faces, and the vertices of each triangular face f∈T are (v0,v1,v2), with an outward normal of n. f The plane offset is d f The centroid of the triangular mesh T is P0; Calculate volume V hull Using the centroid P0 of the mesh as a reference point, and combining each triangular facet (v0, v1, v2) of the triangular mesh T to form a tetrahedron (P0, v0, v1, v2), volume accumulation is performed: Calculate the approximate volume V cap : For point cloud data P in the target area f ={P i ∈R 3 To perform principal component analysis, calculate the covariance ∑ using the following formula: In the formula, μ is the centroid; Perform eigenvalue decomposition on ∑, obtaining three orthogonal principal axes e1, e2, and e3 in descending order of eigenvalues. e1 is the length axis, e2 is the principal diameter axis, and e3 is the principal diameter axis. Then, for each three-dimensional point P... i Perform a one-dimensional projection on the three principal axes e1, e2, and e3: Thus {t 1，i } is a series of scalar coordinates along the length axis; {t 2，i }, {t 3，i } represents scalar coordinates on two cross-sectional axes, and quantile extrema are used to reduce noise. L=Q 98 (t1)-Q2(t1) D1=Q 98 (t2)-Q2(t2) D2=Q 98 (t3)-Q2(t3); Let D = (D1 + D2) / 2 as the equivalent diameter D; Approximate volume V cap : Calculate the fusion volume In the formula, ω is a coefficient, ω∈[0,1]; Calculate the projected area A: The sea cucumber target region is extracted from the detection result image. A small local image is cropped from the color image according to the detection box. Then, a small depth block is cropped from the depth image based on the same pixel coordinates. Non-sea cucumber pixels within the local image are set to zero or removed to obtain a clean mask M(u,v). The projected micro-area of ​​each pixel (u,v) within the mask at the corresponding depth Z(u,v) is calculated. Then, by summing up, we obtain the projected area A: Calculate the cross-sectional area R of the sea cucumber: Where D is the equivalent diameter.

8. A control method for the machine vision-based sea cucumber harvesting robot described in claim 1, characterized in that, The control method described is a sliding mode predictive control (SMPC) method that integrates MPC and sliding mode control. The SMPC method has a sliding mode prediction model, which utilizes the future prediction information of sliding variables and combines the rolling optimization mechanism in MPC to obtain the optimal control input by solving an optimization problem with constraints.

9. The control method according to claim 8, characterized in that, The sliding mode predictive control (SMPC) method includes the following steps: Step 1: First, based on the pose error information obtained from the vision system, the desired acceleration and angular velocity inputs are determined using the SMPC algorithm; Step 2: Solve the control input using the robot hydrodynamic model and motor power feedback information to obtain the actual motor output; Step 3: Use IMU measurement data to estimate and optimize the inertial motion state, further improving the accuracy of pose calculation; Step 4: Finally, the attitude angle information is combined to complete the real-time coordinate transformation, thereby achieving accurate tracking of the target trajectory.

10. A method for grading sea cucumbers, characterized in that, Includes the following steps: The first step is to establish a sea cucumber target detection model based on convolutional neural networks; The second step involves using a depth camera to capture images of the seabed environment, obtaining color and depth images. The third step is to preprocess the color image; The fourth step is to input the preprocessed color image into the sea cucumber target detection model to identify the sea cucumber target area and obtain the detection result image. The fifth step is to perform pixel-level registration between the detection result image and the depth image obtained in the second step, based on the sea cucumber detection box in the detection result image, and extract the three-dimensional point cloud data corresponding to the target area. Step 6: Estimation of quantitative indicators based on 3D convex hull; The seventh step is to grade the sea cucumbers according to quantitative indicators.

Images