Damage detection method for underwater nets of cement cages based on machine vision

Abstract

The present invention provides a machine vision-based method for detecting damaged mesh in underwater cement cage nets, which relates to the technical field of mesh damage detection. The method comprises: controlling an ROV to move to a detection position, capturing an underwater image of the cement cage based on an RGB color space, and transmitting the underwater image to a control computer; the control computer converting the underwater image based on the RGB color space into an L*a*b* color space; identifying meshes of the cement cage net in the underwater image; identifying damaged meshes based on the area and central moment of each mesh in the underwater image; marking the damaged meshes in the underwater image, and outputting the marked underwater image. The present invention can accurately segment the mesh structure in complex underwater environments, offset perspective distortion caused by water flow and mesh topology changes caused by algae adhesion, increase the recognition rate of damaged meshes, achieve real-time continuous tracking of damaged meshes, and accurately locate damaged meshes, achieving high efficiency, low cost, and low risk.

Description

Technical Field

[0001] The present invention relates to the technical field of net damage detection, and in particular to a method for detecting damage of underwater cement cage nets based on machine vision. Background Art

[0002] Cement-based aquaculture cage net damage is primarily caused by a combination of environmental factors (waves, currents, and fouling), net-specific factors (material aging, design, and quality issues), aquaculture management factors (improper operation and inadequate maintenance), and facility and equipment factors (problems with the frame and ancillary facilities, and impacts of supporting equipment). Therefore, regular damage inspection is necessary to promptly detect and repair damaged areas to ensure the proper operation of the cages and the safe growth of aquatic organisms. Existing solutions generally acquire color images of deep-sea cage nets and perform preprocessing steps such as denoising, grayscale conversion, contrast enhancement, and position correction. Feature extraction algorithms, region segmentation, and selective search are then used for target detection to determine the location and extent of the cage nets. Edge information is then extracted and segmented. Finally, this segmented information is analyzed to detect and visualize overall and local breaks and defects in the nets. However, these approaches lack adaptability to aquatic environments. In the complex deep-sea environment, noise generated by light reflection and scattering is difficult to completely remove, impacting subsequent image processing. 2. Limited target detection accuracy. Although multiple algorithms are used to extract key features and perform target detection, detection accuracy still needs to be improved given the diverse variations of cage nets in complex scenarios. 3. Edge extraction accuracy issues. During edge extraction, steps such as threshold selection and gradient calculation may be affected by image quality, resulting in inaccurate extracted edge information and affecting the determination of net damage. 4. Limitations of image processing algorithms. When processing captured images, they may be affected by the underwater environment and image quality, resulting in poor image processing results and, in turn, affecting the accuracy of damage detection. Summary of the Invention

[0003] The purpose of the present invention is to solve the problems existing in the existing methods for detecting damage to underwater aquaculture cage nets: 1) the complex underwater environment makes it impossible for the traditional global threshold method to accurately segment the net structure; 2) the net undergoes perspective deformation due to water flow and interference from surface attachments, making the traditional methods relying on geometric transformation ineffective and having a high false alarm rate; 3) there is a lack of an effective mechanism to filter out false alarms of candidate mesh damage caused by water flow jitter, and real-time continuous tracking cannot be achieved; 4) it relies on manual diving detection, which is inefficient, costly, risky, and cannot accurately locate damage.

[0004] To achieve the above object, the present invention provides a method for detecting damage to underwater nets of cement cages based on machine vision, the method comprising:

[0005] Control the ROV to move to the inspection position, collect underwater images of the cement cage based on RGB color space, and transmit the underwater images to the control computer;

[0006] Control the computer to convert the underwater image based on RGB color space into L*a*b* color space;

[0007] Identify the mesh of cement cage nets in underwater images;

[0008] Identify damaged meshes based on the area and central moment of each mesh in the underwater image;

[0009] The damaged meshes in the underwater image are marked, and the marked underwater image is output.

[0010] Furthermore, the brightness level in the L* channel of the underwater image is compared with a preset brightness threshold to perform binary segmentation, separating the mesh from the background and obtaining a binary image;

[0011] The control computer inverts the binary image to identify the mesh.

[0012] Furthermore, damaged meshes are identified based on the area and central moment of each mesh in the underwater image, specifically including:

[0013] Remove meshes with incomplete edges in underwater images and mark meshes with complete edges in underwater images as a type of mesh;

[0014] Based on the Z-Score detection method, the first-class meshes are analyzed to determine the potential damaged meshes, and the potential damaged meshes are marked as the second-class meshes;

[0015] The damaged meshes are identified according to the area and central moment of the second type meshes and marked.

[0016] Furthermore, a type of mesh is analyzed based on the Z-Score detection method to identify potential damaged meshes, including:

[0017] Calculate the mesh size of each type Z The score is calculated as follows:

[0018]

[0019] In the above formula, Z express Z Fraction, x is the area of ​​the mesh type under analysis, represents the average mesh area, represents the standard deviation of the mesh area;

[0020] Determine whether the Z score of each Class I mesh meets the screening conditions, and use the Class I mesh that meets the screening conditions as potential damaged meshes. The screening conditions are expressed as:

[0021]

[0022] In the above formula, is a measure that defines the change in threshold, Indicates the number of meshes of a type.

[0023] Furthermore, damaged meshes are identified based on the area and central moment of the second type of meshes, specifically including:

[0024] Calculate the Euclidean distance between the central moment of the second type mesh and the central moment of other second type meshes to determine the adjacent mesh closest to the second type mesh;

[0025] The areas of the second-class mesh and its nearest adjacent meshes are compared. If the area difference between the second-class mesh and its adjacent meshes is greater than a preset difference threshold, the corresponding second-class mesh is marked as a damaged mesh.

[0026] Furthermore, when the second-class mesh is identified as a damaged mesh, the damaged mesh is confirmed, specifically including:

[0027] Underwater video of cement cages was collected using ROV;

[0028] Perform spatiotemporal analysis of the damaged mesh based on the underwater video to determine whether the damaged mesh appears in X consecutive frames in the underwater video. If so, proceed to the next step.

[0029] The displacement of the central moment of the damaged mesh in consecutive frames is calculated to determine whether the displacement of the central moment is less than a preset displacement threshold. If so, the existence of the damaged mesh is confirmed.

[0030] Furthermore, when identifying damaged meshes based on the area and central moment of each mesh in the underwater image, the YOLOv8 network is used to identify potential damaged meshes in a class of meshes.

[0031] Furthermore, the damaged meshes in the underwater image are marked, including marking the position, area size and ID of the damaged meshes.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The method provided by this invention combines L*a*b* color space conversion and adaptive threshold segmentation to dynamically adapt to changes in underwater lighting. It automatically inverts binary images based on pixel ratios, is compatible with both natural and artificial light sources, and significantly improves segmentation robustness. It can accurately segment net structures and solve problems such as uneven underwater lighting and turbid water.

[0034] 2. This invention uses the Z-Score detection method instead of traditional rigid models such as Hough transform. It dynamically determines mesh damage based on the number and area difference of adjacent meshes, offsetting mesh topology changes caused by water flow perspective deformation and algae adhesion, and increasing the damage recognition rate.

[0035] 3. This invention collects underwater video and performs continuous frame displacement threshold tracking on damaged meshes based on the underwater video to confirm the existence of damaged meshes, reduce the false alarm rate of candidate mesh damage caused by water flow jitter, and achieve real-time continuous tracking;

[0036] 4. The present invention uses ROV to automatically collect underwater images of cement cages, and controls the computer to perform a series of automated processing on the underwater images, such as mesh segmentation and damage marking, and finally outputs the marked underwater images, which can achieve accurate positioning of damaged meshes, with high efficiency, low cost and low risk. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 The present invention provides a method for detecting underwater mesh damage in a cement cage based on machine vision.

[0039] Figure 2 This is a schematic diagram of the mesh identification process provided by an embodiment of the present invention.

[0040] Figure 3 It is a schematic diagram of the damaged mesh identification process provided by an embodiment of the present invention.

[0041] Figure 4 3 is a schematic diagram of a potential damaged mesh identification process based on a Z-Score detection method provided by an embodiment of the present invention.

[0042] Figure 5 It is a schematic diagram of the damaged mesh identification process provided by an embodiment of the present invention.

[0043] Figure 6 It is a schematic diagram of a damaged mesh confirmation process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The principles and features of the present invention are described below with reference to the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.

[0045] Reference Figure 1 This embodiment provides a method for detecting damage to underwater nets of cement cages based on machine vision, the method comprising:

[0046] S101, control the ROV to move to the detection position, collect underwater images of the cement cage based on the RGB color space, and transmit the underwater images to the control computer.

[0047] S102 , controlling the computer to convert the underwater image based on the RGB color space into the L*a*b* color space.

[0048] For example, converting an underwater image based on the RGB color space to the L*a*b* color space can be performed using the cvtcolor function and the split function. In the L*a*b* color space, L* represents perceived brightness, a* represents the perceived change from red to green, and b* represents the perceived change from yellow to blue.

[0049] S103: Identify the meshes of the cement cage net in the underwater image.

[0050] S104: Identify damaged meshes according to the area and central moment of each mesh in the underwater image.

[0051] S105: Mark the damaged meshes in the underwater image, and output the marked underwater image.

[0052] As a possible implementation method, refer to Figure 2 , to identify the mesh in the underwater image, specifically including the following operations:

[0053] S201 : Compare the brightness level in the L* channel of the underwater image with a preset brightness threshold to perform binary segmentation, separate the mesh from the background, and obtain a binary image.

[0054] In this step, if the brightness level of a pixel in the underwater image exceeds a preset brightness threshold, the pixel is transformed into white; otherwise, it is transformed into black. Specifically, white represents the web, and black represents the background. For example, the adaptiveThreshold function can be used to compare the brightness level in the underwater image's L* channel with a preset brightness threshold to perform binary segmentation. The connectedComponentsWithStats function is also used to remove unnecessary elements from the segmented image. It is expected that the connected component with the largest number of pixels will correspond to the web (noise removed).

[0055] S202: Control the computer to flip the binary image to identify the mesh.

[0056] For example, after receiving the binary segmentation image with the filtered representation of the mesh structure through the connectedComponentsWithStats function and the morphologyEx function, the network segmentation mask is inverted. In this case, black represents the mesh and white represents the mesh.

[0057] In other implementations, other color spaces such as YUV, HSI, etc. may also be used to implement segmentation, but these methods may still be insufficient in adaptability to complex scenes.

[0058] Reference Figure 3 , identifying damaged meshes based on the area and central moment of each mesh in the underwater image, specifically including the following operations:

[0059] S301: remove meshes with incomplete edges in the underwater image, and mark meshes with complete edges in the underwater image as a type of mesh.

[0060] For example, by identifying all white pixel regions surrounded by closed segments of black pixels, all meshes whose pixels are located at the image boundary but not completely visible are discarded, and the remaining meshes with completed edges are marked as a type of mesh.

[0061] S302: Analyze the first-class meshes based on the Z-Score detection method to determine potential damaged meshes, and mark the potential damaged meshes as second-class meshes.

[0062] S303: Identify damaged meshes according to the area and central moment of the second-type meshes, and mark the damaged meshes.

[0063] In this embodiment, the central moment is calculated based on the pixel area and grayscale value after image segmentation, and is used to describe the distribution of the mesh.

[0064] Specifically, refer to Figure 4 , based on the Z-Score detection method, a type of mesh is analyzed to determine the potential damaged mesh, including:

[0065] S401, calculate each type of mesh Z The score is calculated as follows:

[0066]

[0067] In the above formula, Z express Z Fraction, x is the area of ​​the mesh type under analysis, represents the average mesh area, Represents the standard deviation of the mesh area.

[0068] S402: Determine whether the Z score of each Class I mesh satisfies a screening condition, and classify the meshes that meet the screening condition as potential damaged meshes. The screening condition is expressed as:

[0069]

[0070] In the above formula, It is a measure that defines the change in threshold and is a value that requires empirical calibration. Indicates the number of meshes of a type.

[0071] This embodiment identifies candidate damaged areas by analyzing the area of ​​the mesh regions, using a Z score to determine the deviation of an observation from other observations in the dataset, and applying a variable threshold based on the number of meshes detected to obtain the criteria for finding candidates.

[0072] In other embodiments, when identifying damaged meshes based on the area and central moment of each mesh in an underwater image, a YOLOv8 network can be used to identify potentially damaged meshes within a class of meshes. Compared to the Z-Score detection method, identifying potentially damaged meshes using the YOLOv8 network requires a training dataset and is more dependent on data quality.

[0073] Based on the above implementation, Figure 5 , identifying damaged meshes based on the area and central moment of the second type of meshes, specifically including the following operations:

[0074] S501 : Calculate the Euclidean distance between the central moment of the second type mesh and the central moments of other second type meshes to determine the adjacent mesh closest to the second type mesh.

[0075] S502: Compare the areas of the second-class mesh and its nearest adjacent meshes. If the area difference between the second-class mesh and its adjacent meshes is greater than a preset difference threshold, mark the corresponding second-class mesh as a damaged mesh.

[0076] This embodiment performs a local analysis of candidate values ​​by comparing the areas of adjacent cells of a Class II cell that is a damaged candidate cell. If the difference between the area of ​​a Class II cell and its adjacent cells is greater than a preset difference threshold, the corresponding Class II cell is considered a damaged cell and marked as such.

[0077] As a further optional embodiment, according to Figure 6 When the second-class mesh is identified as a damaged mesh, the damaged mesh is confirmed, including the following operations:

[0078] S601. Collect underwater video of the cement cage using an ROV.

[0079] S602: Perform spatiotemporal analysis on the damaged mesh based on the underwater video to determine whether the damaged mesh appears in X consecutive frames in the underwater video. If so, proceed to the next step; otherwise, terminate the process.

[0080] S603: Calculate the displacement of the central moment of the damaged mesh in consecutive frames and determine whether the displacement of the central moment is less than a preset displacement threshold. If so, the existence of the damaged mesh is confirmed; otherwise, the process is terminated.

[0081] In this embodiment, the preset displacement threshold can be determined based on the maximum detection speed and known size of the mesh. For example, when a damaged mesh appears in six consecutive frames of the underwater video and its displacement meets the conditions, it is determined that the damaged mesh exists, i.e., the cage net is damaged at that location.

[0082] In this embodiment, damaged meshes in underwater images are marked, including their location, size, and ID. The ID is a unique identifier for the damaged mesh. By marking the location, size, and other related information in the underwater image, maintenance personnel can locate the damaged meshes and promptly repair the damaged areas, ensuring the normal operation of the cage.

[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting damage to underwater nets of cement cages based on machine vision, characterized in that: The method comprises: Control the ROV to move to the inspection position, collect underwater images of the cement cage based on RGB color space, and transmit the underwater images to the control computer; Control the computer to convert the underwater image based on RGB color space into L*a*b* color space; Identify the mesh of cement cage nets in underwater images; Identify damaged meshes based on the area and central moment of each mesh in the underwater image; Marking the damaged meshes in the underwater image and outputting the marked underwater image; Identify meshes in underwater images, including: Comparing the brightness level in the L* channel of the underwater image with a preset brightness threshold to perform binary segmentation, separate the mesh from the background, and obtain a binary image; The control computer inverts the binary image to identify the mesh; Identify damaged meshes based on the area and central moment of each mesh in the underwater image, including: Remove meshes with incomplete edges in underwater images and mark meshes with complete edges in underwater images as a type of mesh; Based on the Z-Score detection method, the first-class meshes are analyzed to determine the potential damaged meshes, and the potential damaged meshes are marked as the second-class meshes; Identify damaged meshes based on the area and central moment of the second-class meshes and mark the damaged meshes; Analyze a type of mesh based on the Z-Score detection method to identify potential damaged meshes, including: Calculate the mesh size of each type Z The score is calculated as follows: In the above formula, Z express Z Fraction, x is the area of ​​the mesh type under analysis, represents the average mesh area, represents the standard deviation of the mesh area; Determine whether the Z score of each Class I mesh meets the screening conditions, and use the Class I mesh that meets the screening conditions as potential damaged meshes. The screening conditions are expressed as: In the above formula, is a measure that defines the change in threshold, Indicates the number of meshes of a type; Identify damaged meshes based on the area and central moment of the second-class meshes, including: Calculate the Euclidean distance between the central moment of the second type mesh and the central moment of other second type meshes to determine the adjacent mesh closest to the second type mesh; The areas of the second-class mesh and its nearest adjacent meshes are compared. If the area difference between the second-class mesh and its adjacent meshes is greater than a preset difference threshold, the corresponding second-class mesh is marked as a damaged mesh.

2. The method for detecting damage of underwater nets of cement cages based on machine vision according to claim 1, characterized in that: When the second-class mesh is identified as a damaged mesh, the damaged mesh is confirmed, specifically including: Underwater video of cement cages was collected using ROV; Perform spatiotemporal analysis of the damaged mesh based on the underwater video to determine whether the damaged mesh appears in X consecutive frames in the underwater video. If so, proceed to the next step. The displacement of the central moment of the damaged mesh in consecutive frames is calculated to determine whether the displacement of the central moment is less than a preset displacement threshold. If so, the existence of the damaged mesh is confirmed.

3. The method for detecting damage of underwater nets of cement cages based on machine vision according to claim 1, characterized in that: When identifying damaged meshes based on the area and central moment of each mesh in the underwater image, the YOLOv8 network is used to identify potential damaged meshes in a class of meshes.

4. The method for detecting damage of underwater nets of cement cages based on machine vision according to claim 1, characterized in that: Mark the damaged mesh in the underwater image, including marking the location, area size and ID of the damaged mesh.

Images