A seabed garbage classification method and system based on visual analysis
By employing a visual analysis-based approach, utilizing the Sobel operator and mean-shift clustering algorithm, the problem of insufficient accuracy in seabed debris classification was solved, achieving efficient debris classification and statistics, which is applicable to marine environmental protection and resource recycling.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-20
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies lack accuracy in seabed debris classification and image segmentation and recognition. In particular, when classifying images of multiple categories and multiple types of debris, it is difficult to handle entanglement, accumulation, and overlap, resulting in large classification and recognition errors and making it difficult to improve classification accuracy.
A visual analysis-based approach is adopted, which extracts grayscale contour features using the Sobel operator, combines color histograms and mean-shift clustering algorithms to cluster contour data, sets multiple weight ratios, selects the best contour data, and combines a preset classification model to perform garbage classification and statistics.
It improves the accuracy and efficiency of image segmentation for underwater debris sorting, enabling precise debris sorting that is applicable to marine environmental protection and resource recycling.
Smart Images
Figure CN122435306A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of marine conservation and visual analysis, and more specifically, to a method and system for classifying seabed debris based on visual analysis. Background Technology
[0002] Marine debris pollution has become a global ecological crisis. Due to its wide distribution, complex environment, and the difficulty in cleaning, identifying, and classifying it, seabed debris poses a serious threat to marine habitats, shipping safety, and the balance of ecosystems. Seabed debris includes plastics, metals, fishing nets, and organic matter, among others. The degradation characteristics, recycling value, and environmental hazards of different types of debris vary significantly, making accurate classification a prerequisite for efficient cleanup and resource recovery.
[0003] Existing methods for classifying and segmenting marine debris often yield poor results, particularly for tasks involving multiple categories and types of debris. The lack of a fusion analysis process for various contour features leads to inaccurate contour extraction and significant classification errors. Furthermore, marine debris is prone to entanglement, accumulation, and overlap, making it difficult for traditional techniques to accurately and quickly classify images in such situations. This results in errors in the final statistical analysis, and the classification accuracy remains difficult to improve and evaluate. Therefore, there is an urgent need for a visual analysis-based method and system for marine debris classification. Summary of the Invention
[0004] This invention overcomes the shortcomings of existing technologies and proposes a method and system for classifying marine debris based on visual analysis.
[0005] The first aspect of this invention provides a visual analysis-based method for classifying marine debris, comprising: Step 1: In the preset sea area, record the seabed debris image information collected at each station and seal the seabed debris; Step 2: Clean, dry, and pre-dry the seabed debris, and record images of the debris to obtain an image set; Step 3: Perform image preprocessing on the image set, set an N×N moving window, extract contour features from the image set using the Sobel operator to obtain grayscale contour features, introduce a color histogram and moving window, analyze the color differences of the windows in the image set, and extract color contour features. Step 4: In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced to cluster the contour data. During the clustering process, the differences between contour data are reflected by the weighted differences of the two contour features. Multiple weight ratios are set to obtain multiple clustering results. Step 5: Use the weight ratio corresponding to the result with the most clusters as the gray-level and color contour ratio, and use the contour ratio as the contour reference weight. Reduce the data from the gray-level contour features and color contour features in the image set to obtain the preferred contour data. Step 6: Introduce a preset classification model, combine the image set with the selected contour data to perform garbage classification and statistics, and obtain statistical information.
[0006] In this solution, step 1 specifically includes: Before collecting and cleaning seabed debris, images of seabed debris, attached organisms, and site information are recorded at each site to form a pre-cleaning image set, and the seabed debris is then sealed and stored.
[0007] In this solution, step 2 specifically includes: The seabed debris was cleaned, dried, and pre-treated. Images of the cleaned debris were recorded based on preset image parameters to obtain an image set. The unfolded area of the debris was measured using ImageJ software, and the weight of the debris was recorded.
[0008] In this solution, step 3 specifically includes: Denoising and enhancement preprocessing are performed on the image set; Extract a target image from the image set, perform grayscale processing on the target image, set an N×N moving window, introduce the Sobel operator, place the moving window on the target image to calculate local region edge points, calculate the gradient magnitude of the N×N window in each movement, and filter out edge points based on gradient threshold comparison. The grayscale contour features are obtained from the edge points selected through screening.
[0009] In this solution, step 3 further includes: The moving window is placed in the target image to calculate the edge points of the local region. In each move, the color histogram of the moving window and multiple neighboring windows is calculated. In the color histogram, multiple intervals are divided into the three color dimensions of RGB. The number of pixels corresponding to each interval is counted among all pixels in the window. Based on the number of pixels in each interval, a histogram feature vector is generated. By statistically analyzing the color histogram, the histogram feature vectors of the corresponding moving window and neighboring windows are obtained. The distance between the histogram feature vectors is used to characterize the color feature difference between the moving window and neighboring windows. If the difference value is greater than a preset threshold, the corresponding moving window pixels are marked as color edge points. Analyze all color edge points in the target image to obtain color edge regions, and statistically analyze the histogram feature vectors of the edge regions to obtain color contour features.
[0010] In this scheme, the image set includes multiple images, and each image includes one or more types of marine debris.
[0011] In this solution, step 4 specifically includes: In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced to cluster the contour data. During the clustering process, the differences between contour data are reflected by the weighted difference of the two contour features. Multiple weight ratios are set to obtain multiple clustering results. In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced, and the contour data is used as clustering samples to perform contour difference analysis and clustering on each image. Set multiple weight ratios, each of which includes the proportion of grayscale and color outline; In the clustering process, a weight ratio is introduced to evaluate the similarity between clustered samples by weighted difference between contour data. A preset bandwidth is set as the neighborhood range, and all clustered samples are used as center points. The center point drift is calculated by Gaussian kernel function, and the mean drift of the center points is performed until the preset number of iterations is reached or all center points converge. All data that converge to the same center point are grouped into the same cluster, and the clustering result is obtained. The difference analysis between contour data specifically involves calculating the distance value D1 between gray contour features and the distance value D2 between color contour features in two clustered samples, and then taking a weighted average of D1 and D2 according to the aforementioned weight ratio to obtain the mean difference, which reflects the similarity between clustered samples. Multiple clustering evaluations were performed based on various weighting ratios to obtain multiple clustering results.
[0012] In this solution, step 5 specifically includes: The weight ratio corresponding to the result with the most clusters is used as the gray-scale to color contour ratio and marked as the contour reference weight. Based on the reference weight of the contour, the contour data corresponding to the lower weight is reduced. The amount of data to be reduced is set based on the reference weight, and the optimal contour data is obtained.
[0013] In this solution, step 6 includes: By using a pre-defined classification model, image segmentation and target recognition are performed using optimized contour data, and marine debris statistics are compiled by combining information on the weight, category, size, and source of marine debris.
[0014] A second aspect of the present invention also provides a visual analysis-based seabed debris sorting system, the system comprising: a memory, a processor, and a communication interface, wherein the memory includes a visual analysis-based seabed debris sorting program, and the visual analysis-based seabed debris sorting program, when executed by the processor, performs the following steps: Step 1: In the preset sea area, record the seabed debris image information collected at each station and seal the seabed debris; Step 2: Clean, dry, and pre-dry the seabed debris, and record images of the debris to obtain an image set; Step 3: Perform image preprocessing on the image set, set an N×N moving window, extract contour features from the image set using the Sobel operator to obtain grayscale contour features, introduce a color histogram and moving window, analyze the color differences of the windows in the image set, and extract color contour features. Step 4: In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced to cluster the contour data. During the clustering process, the differences between contour data are reflected by the weighted differences of the two contour features. Multiple weight ratios are set to obtain multiple clustering results. Step 5: Use the weight ratio corresponding to the result with the most clusters as the gray-level and color contour ratio, and use the contour ratio as the contour reference weight. Reduce the data from the gray-level contour features and color contour features in the image set to obtain the preferred contour data. Step 6: Introduce a preset classification model, combine the image set with the selected contour data to perform garbage classification and statistics, and obtain statistical information.
[0015] This invention discloses a visual analysis-based method and system for classifying seabed debris, relating to the fields of marine environmental protection and visual analysis. The method includes: collecting and storing debris images at a predetermined sea area site; performing standardized preprocessing on the debris and constructing an image set; extracting grayscale contour features using a moving window and the Sobel operator, and extracting color contour features using a color histogram; characterizing contour similarity through weighted differences between the two contours, and clustering the contour data of the image set based on the mean-shift clustering algorithm; selecting optimal contour data based on the best clustering results; and finally, completing debris classification and statistics using a predetermined classification model. This invention improves the accuracy and efficiency of image segmentation in seabed debris classification through multi-feature fusion and adaptive weight optimization, effectively enhancing classification precision. It is applicable to fields such as marine environmental protection monitoring and resource recycling, demonstrating significant technological advancement and practical value. Attached Figure Description
[0016] Figure 1 A flowchart of a visual analysis-based method for classifying marine debris according to the present invention is shown; Figure 2 A block diagram of a visual analysis-based seabed debris sorting system according to the present invention is shown. Detailed Implementation
[0017] To better understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. In the embodiments, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0018] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.
[0019] Figure 1 A flowchart of a visual analysis-based method for classifying marine debris according to the present invention is shown.
[0020] like Figure 1 As shown, the first aspect of the present invention provides a method for classifying seabed debris based on visual analysis, comprising: Step 1: In the preset sea area, record the seabed debris image information collected at each station and seal the seabed debris; Step 2: Clean, dry, and pre-dry the seabed debris, and record images of the debris to obtain an image set; Step 3: Perform image preprocessing on the image set, set an N×N moving window, extract contour features from the image set using the Sobel operator to obtain grayscale contour features, introduce a color histogram and moving window, analyze the color differences of the windows in the image set, and extract color contour features. Step 4: In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced to cluster the contour data. During the clustering process, the differences between contour data are reflected by the weighted differences of the two contour features. Multiple weight ratios are set to obtain multiple clustering results. Step 5: Use the weight ratio corresponding to the result with the most clusters as the gray-level and color contour ratio, and use the contour ratio as the contour reference weight. Reduce the data from the gray-level contour features and color contour features in the image set to obtain the preferred contour data. Step 6: Introduce a preset classification model, combine the image set with the selected contour data to perform garbage classification and statistics, and obtain statistical information.
[0021] According to an embodiment of the present invention, step 1 specifically includes: Before collecting and cleaning seabed debris, images of seabed debris, attached organisms, and site information are recorded at each site to form a pre-cleaning image set, and the seabed debris is then sealed and stored.
[0022] Before cleaning the seabed debris, photos are taken of the debris at each site, along with any attached organisms. The information from the sealed bags at each site is then photographed. Because long-term accumulation of seabed debris easily attracts mud and sand, it is generally necessary to wash away the sandy and muddy sediment inside the debris with fresh water before returning the cleaned debris to its original sealed bags. Pre-treatment of the seabed debris also includes natural air drying and freeze-drying. Considering the large quantity of debris and the high proportion of plastic, natural air drying is used first, followed by freeze-drying (using a Haier Biomedical DG-65Z04-10A freeze dryer). The experimental table is wiped clean, absorbent paper is laid out, and the cleaned seabed debris is laid flat, ensuring that the debris from each site is separated to avoid mixing. Furthermore, after the seabed debris has dried, photos are taken of the debris at each site (e.g., using Sony RX1 / RX2 cameras or Canon G1X cameras). For analytical purposes, individual photos of each piece of debris can be taken, along with information from each site. During the photography process, the underwater debris should be laid flat in the photography studio, and a 1 cm × 5 cm scale ruler should be placed inside for ImageJ software to measure the unfolded area.
[0023] According to an embodiment of the present invention, step 2 specifically includes: The seabed debris was cleaned, dried, and pre-treated. Images of the cleaned debris were recorded based on preset image parameters to obtain an image set. The unfolded area of the debris was measured using ImageJ software, and the weight of the debris was recorded.
[0024] In this embodiment, preset image parameters may include setting the image resolution, image size, image aspect ratio, etc.
[0025] According to an embodiment of the present invention, step 3 specifically includes: Denoising and enhancement preprocessing are performed on the image set; Extract a target image from the image set, perform grayscale processing on the target image, set an N×N moving window, introduce the Sobel operator, place the moving window on the target image to calculate local region edge points, calculate the gradient magnitude of the N×N window in each movement, and filter out edge points based on gradient threshold comparison. The grayscale contour features are obtained from the edge points selected through screening.
[0026] In this embodiment, N can be 3, meaning a 3×3 window is used to calculate edge points. The gradient threshold can be set to be greater than or equal to 70. Grayscale contour features can include the location of contour edge points, gradient magnitude, gradient direction, and parameters such as the length, width, and area of the contour. Subsequent feature vector conversion and contour feature difference evaluation can be performed.
[0027] According to an embodiment of the present invention, step 3 further includes: The moving window is placed in the target image to calculate the edge points of the local region. In each move, the color histogram of the moving window and multiple neighboring windows is calculated. In the color histogram, multiple intervals are divided into the three color dimensions of RGB. The number of pixels corresponding to each interval is counted among all pixels in the window. Based on the number of pixels in each interval, a histogram feature vector is generated. By statistically analyzing the color histogram, the histogram feature vectors of the corresponding moving window and neighboring windows are obtained. The distance between the histogram feature vectors is used to characterize the color feature difference between the moving window and neighboring windows. If the difference value is greater than a preset threshold, the corresponding moving window pixels are marked as color edge points. Analyze all color edge points in the target image to obtain color edge regions, and statistically analyze the histogram feature vectors of the edge regions to obtain color contour features.
[0028] In this embodiment, color feature analysis of the target image does not require grayscale conversion. The moving window step size is set to N. The neighboring windows include the 8 windows currently in the vicinity of the moving position (excluding the image boundary windows). Here, the set N×N moving windows are used to slide and scan the garbage image from left to right and from top to bottom, collecting the corresponding grayscale and color (color) contour features. The feature vectors are distanced using cosine distance.
[0029] In a color histogram, the colors of an image can be equally divided into several intervals (e.g., red is divided into 8 intervals, green into 8 intervals, and blue into 8 intervals). The color values between intervals are consistent. The number of pixels in each interval is counted to form histogram data. Based on the data of each dimension (corresponding to the number of each type of interval), the histogram feature vector is obtained and used as a color feature for comparison.
[0030] In the process of calculating the distance between histogram feature vectors to characterize the color feature difference between the moving window and its neighboring windows and to filter color edge points, specifically, in the moving window, the average distance between the corresponding feature vector and the feature vectors of multiple neighboring windows is calculated to obtain the average difference value. Based on the difference value and a threshold, it is determined whether it is an edge point.
[0031] According to an embodiment of the present invention, the image set includes multiple images, each image including one or more types of marine debris.
[0032] According to an embodiment of the present invention, step 4 specifically includes: In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced to cluster the contour data. During the clustering process, the differences between contour data are reflected by the weighted difference of the two contour features. Multiple weight ratios are set to obtain multiple clustering results. In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced, and the contour data is used as clustering samples to perform contour difference analysis and clustering on each image. Set multiple weight ratios, each of which includes the proportion of grayscale and color outline; In the clustering process, a weight ratio is introduced to evaluate the similarity between clustered samples by weighted difference between contour data. A preset bandwidth is set as the neighborhood range, and all clustered samples are used as center points. The center point drift is calculated by Gaussian kernel function, and the mean drift of the center points is performed until the preset number of iterations is reached or all center points converge. All data that converge to the same center point are grouped into the same cluster, and the clustering result is obtained. The difference analysis between contour data specifically involves calculating the distance value D1 between gray contour features and the distance value D2 between color contour features in two clustered samples, and then taking a weighted average of D1 and D2 according to the aforementioned weight ratio to obtain the mean difference, which reflects the similarity between clustered samples. Multiple clustering evaluations were performed based on various weighting ratios to obtain multiple clustering results.
[0033] In this embodiment, the weighting of grayscale and color contours can be set in multiple ways (multiple weight ratios), and can gradually change based on the weighting of grayscale contours. For example, the weighting can be set to 9:1, 8:2, 7:3, etc. Finally, among the various preset weight ratios, the optimal weighting is selected as the weighting parameter for the current image segmentation task. Furthermore, certain filtering and deletion processes are performed on the contour features to obtain accurate contour features. The preset bandwidth can be set as the mean of the maximum and minimum distances between all clustered samples (contour data) to match the clustering process. During the clustering process, the clustered samples are contour data, and each clustered sample corresponds to the feature data obtained from one image, including two dimensions of features: grayscale contour and color contour.
[0034] This section introduces the mean-shift clustering algorithm to perform cluster analysis on the contour dataset. The core of the mean-shift clustering algorithm is to iteratively find the density peaks of data points, grouping data points with similar densities into the same cluster. This algorithm does not require a preset number of clusters and has the advantages of strong noise resistance and stable clustering results. Clustering evaluation is performed on contour features in two dimensions to effectively assess the effectiveness of contour feature extraction within a single site. In an image set collected from a single site, a higher number of clusters indicates better segmentation results and higher effectiveness of contour feature extraction. Furthermore, a feature reference weight can be set for color features and Sobel grayscale features. Based on this weight, a contrast threshold is set to delete some contour features, retaining only effective contour features as segmentation criteria for garbage identification, thereby improving recognition accuracy and reducing interference from redundant contour features.
[0035] The presence of redundant contour data can lead to inaccurate segmentation of object images, potentially resulting in the same object being identified multiple times or being classified as multiple objects, thus reducing classification accuracy and increasing statistical error.
[0036] It is worth mentioning that during the process of collecting images of seabed debris at a site, the processing is prone to issues such as incomplete drying, the presence of debris fragments, and entanglement of different types of objects. While traditional analysis and recognition processes can achieve accurate identification, the image segmentation accuracy is difficult to assess, and the segmentation process is difficult to adjust, resulting in certain statistical errors.
[0037] When different types of objects are entangled, manual sorting would significantly increase manpower and material resources, making it difficult to achieve automated and intelligent classification, identification, and statistics. Therefore, this invention analyzes grayscale and color contour features, performs clustering evaluation on image sets, and matches the optimal contour reference weight to achieve effective contour extraction and segmentation accuracy. This enables automated adjustment of the classification process and improves recognition efficiency in various complex waste classification tasks, thereby improving the statistical accuracy of marine debris. It has excellent practicality for multi-site waste collection, recording, and classification.
[0038] In complex waste sorting tasks, contour features should include multiple types and have high dispersion based on different images. If the number of clusters is small, it means that the contour features of the two dimensions of segmentation and recognition are highly consistent, and the corresponding recognition accuracy is also low. Marine debris with similar features is easily classified into different groups, affecting the final statistical results. Moreover, automated image recognition tasks are difficult to detect when their classification accuracy declines, and often require manual intervention to evaluate the accuracy of the recognition results, which has low practicality.
[0039] According to an embodiment of the present invention, step 5 specifically includes: The weight ratio corresponding to the result with the most clusters is used as the gray-scale to color contour ratio and marked as the contour reference weight. Based on the reference weight of the contour, the contour data corresponding to the lower weight is reduced. The amount of data to be reduced is set based on the reference weight, and the optimal contour data is obtained.
[0040] Here, features are reduced based on their proportion in the contour reference section. The specific matching is based on the proportion; for example, a preferred ratio of 9:1 indicates higher importance for grayscale contours and higher contour discrimination compared to color features. Therefore, some color features can be appropriately reduced to decrease the interference of numerous color contour features on the recognition results. The reduction amount is set to 50%, randomly reducing 50% of the color contour features to ensure contour discrimination. Besides random reduction, a preset threshold can be appropriately increased based on color contour analysis to reduce the number of corresponding color contours until the preset reduction percentage is reached. Further, if the preferred ratio is 7:3, the color contour reduction ratio can be set to 30%, which can be adjusted appropriately based on the ratio. If the preferred ratio is 1:1, all contour features are retained.
[0041] According to an embodiment of the present invention, step 6 includes: By using a pre-defined classification model, image segmentation and target recognition are performed using optimized contour data, and marine debris statistics are compiled by combining information on the weight, category, size, and source of marine debris.
[0042] Here, the preset classification models can include object detection models such as the YOLOv5 model and the lightweight Debris-YOLO model.
[0043] The collection of marine debris includes weighing, sorting, size, and source information. During the weighing (balance, accuracy 0.01 g), sorting, size measurement (surface area of marine debris is measured with vernier calipers in spring and with Image J in autumn and winter), and counting processes, if several pieces of marine debris at a single station originate from the same item, they should be counted as one item. For example, a glass bottle broken into two pieces is counted as one item. If two or more pieces of debris are severely entangled, each entanglement should be counted as a separate piece of debris. The mass of each piece of marine debris is weighed using a balance; debris weighing less than 0.01 g should be recorded as 0.01 g.
[0044] In terms of classification, the waste is mainly classified in accordance with the International Bottom Trawl Surveys (IBTS) protocol, the "Guidance on Monitoring of Marine Litter in European Seas," and the "Technical Guidelines for Monitoring and Evaluation of Marine Litter (Trial Implementation)." Marine debris is divided into eight categories: plastics (plastic bags, plastic boxes, plastic fragments, plastic packaging, food packaging bags, and plastic woven bags, etc.), metals (knives, aluminum beverage cans, etc.), glass, paper, rubber, wood, textiles (clothing, towels, cloth shoes, and gloves, etc.), and others (difficult-to-identify marine debris, or marine debris with complex and diverse compositions). If a piece of marine debris is composed of different materials, it is classified according to the dominant material. The specific name and color of the marine debris are recorded (e.g., plastic bags - red).
[0045] In terms of size, for marine debris like steel wool, the original shape should be preserved during measurement. Marine debris should be categorized by size as <25 cm. 2 25-<100 cm 2 100-<400 cm 2 400-<2500 cm 2 2500-<10000 cm 2 ≥10000 cm 2 Six size categories.
[0046] In terms of origin, marine debris is divided into four categories: consumer goods (mainly plastic bags and packaging bags, plastic sheets, snack packaging bags, beverage cans, disposable food containers, plastics, etc.), marine and waterway imports (mainly fishery-related debris, such as fishing lines, fishing nets, etc.), industrial or household (shoes, aprons, cable ties, gloves, etc.), and others (unidentifiable marine debris).
[0047] According to an embodiment of the present invention, it further includes: Collect seabed debris at a single site over multiple periods and perform image set analysis based on one period; Using Sobel and color histogram, contour features are extracted from the image set before cleaning, and the first contour feature and the second contour feature are obtained based on the grayscale and color dimensions, respectively. For each image in the image set before cleaning and the image set for recognition, the difference between the corresponding first contour feature and the gray contour feature is calculated, and multiple difference values are averaged to obtain the gray contour difference value. The difference between the corresponding second contour feature and the color contour feature is calculated and averaged to obtain the color contour difference value. Multiple grayscale contour difference values and multiple color contour difference values are obtained based on multiple cycles; Based on the time dimension, it is evaluated whether the difference values of multiple gray-scale contours are preset linear changes, and whether the difference values of multiple color contours are preset linear changes. The image segmentation accuracy is evaluated based on the judgment results, and the garbage collection and processing scheme is adjusted accordingly.
[0048] In this embodiment, the image set is the image set after cleaning, air drying, and pre-processing in S2. It is used to compare the degree of change in contour features before cleaning. For the analysis of changes in multi-dimensional contour extraction accuracy, this invention introduces a comparison of the consistency of contour features between images before and after cleaning, calculating the linear growth of their differences. For multiple waste collection cycles, if the difference in corresponding contours between images before and after cleaning (pre-processing) is low, it indicates low segmentation accuracy and poor contour determination. Therefore, if the corresponding contour differences show a certain linear growth, it indicates that the contour extraction accuracy is gradually improving, and the segmentation accuracy is also improving. This effectively evaluates the effect of contour screening in steps 1-6 on practical applications, and allows for the analysis of the changes in segmentation accuracy for the two types of contours, determining whether the identification and classification are within expectations. If the corresponding contour difference decreases linearly, it indicates that the contour extraction accuracy is decreasing. In this case, image inspection and preprocessing evaluation can be performed to determine whether there are defects in the image acquisition and processing process (such as whether the image preprocessing is qualified, whether there is too much image noise, whether there is incomplete cleaning or drying, which makes the image contour difficult to identify, etc.). This leads to a decrease in object classification accuracy and achieves effective evaluation in practical applications, improving the adaptability and practicality of the waste classification model, while reducing the involvement of human experience.
[0049] The preset linear change can be set to increase or remain stable (if the difference between grayscale contour and color contour is high in the first cycle).
[0050] Figure 2 A block diagram of a visual analysis-based seabed debris sorting system according to the present invention is shown.
[0051] A second aspect of the present invention also provides a visual analysis-based seabed debris sorting system, the system comprising: a memory, a processor, and a communication interface, wherein the memory includes a visual analysis-based seabed debris sorting program, and the visual analysis-based seabed debris sorting program, when executed by the processor, performs the following steps: Step 1: In the preset sea area, record the seabed debris image information collected at each station and seal the seabed debris; Step 2: Clean, dry, and pre-dry the seabed debris, and record images of the debris to obtain an image set; Step 3: Perform image preprocessing on the image set, set an N×N moving window, extract contour features from the image set using the Sobel operator to obtain grayscale contour features, introduce a color histogram and moving window, analyze the color differences of the windows in the image set, and extract color contour features. Step 4: In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced to cluster the contour data. During the clustering process, the differences between contour data are reflected by the weighted differences of the two contour features. Multiple weight ratios are set to obtain multiple clustering results. Step 5: Use the weight ratio corresponding to the result with the most clusters as the gray-level and color contour ratio, and use the contour ratio as the contour reference weight. Reduce the data from the gray-level contour features and color contour features in the image set to obtain the preferred contour data. Step 6: Introduce a preset classification model, combine the image set with the selected contour data to perform garbage classification and statistics, and obtain statistical information.
[0052] The communication interface is used to receive and transmit image data and analysis results.
[0053] A third aspect of the present invention also provides a computer-readable storage medium comprising a visual analysis-based seabed debris sorting program, wherein when executed by a processor, the visual analysis-based seabed debris sorting program implements the steps of the visual analysis-based seabed debris sorting method as described in any of the preceding claims.
[0054] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.
[0055] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.
[0056] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0057] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0058] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.
[0059] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for classifying marine debris based on visual analysis, characterized in that, include: Step 1: In the preset sea area, record the seabed debris image information collected at each station and seal the seabed debris; Step 2: Clean, dry, and pre-dry the seabed debris, and record images of the debris to obtain an image set; Step 3: Perform image preprocessing on the image set, set an N×N moving window, extract contour features from the image set using the Sobel operator to obtain grayscale contour features, introduce a color histogram and moving window, analyze the color differences of the windows in the image set, and extract color contour features. Step 4: In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced to cluster the contour data. During the clustering process, the differences between contour data are reflected by the weighted differences of the two contour features. Multiple weight ratios are set to obtain multiple clustering results. Step 5: Use the weight ratio corresponding to the result with the most clusters as the gray-level and color contour ratio, and use the contour ratio as the contour reference weight. Reduce the data from the gray-level contour features and color contour features in the image set to obtain the preferred contour data. Step 6: Introduce a preset classification model, combine the image set with the selected contour data to perform garbage classification and statistics, and obtain statistical information.
2. The method for classifying marine debris based on visual analysis according to claim 1, characterized in that, Step 1 specifically includes: Before collecting and cleaning seabed debris, images of seabed debris, attached organisms, and site information are recorded at each site to form a pre-cleaning image set, and the seabed debris is then sealed and stored.
3. The method for classifying marine debris based on visual analysis according to claim 1, characterized in that, Step 2 specifically includes: The seabed debris was cleaned, dried, and pre-treated. Images of the cleaned debris were recorded based on preset image parameters to obtain an image set. The unfolded area of the debris was measured using ImageJ software, and the weight of the debris was recorded.
4. The method for classifying marine debris based on visual analysis according to claim 1, characterized in that, Step 3 specifically includes: Denoising and enhancement preprocessing are performed on the image set; Extract a target image from the image set, perform grayscale processing on the target image, set an N×N moving window, introduce the Sobel operator, place the moving window on the target image to calculate local region edge points, calculate the gradient magnitude of the N×N window in each movement, and filter out edge points based on gradient threshold comparison. The grayscale contour features are obtained from the edge points selected through screening.
5. The method for classifying marine debris based on visual analysis according to claim 4, characterized in that, Step 3 further includes: The moving window is placed in the target image to calculate the edge points of the local region. In each move, the color histogram of the moving window and multiple neighboring windows is calculated. In the color histogram, multiple intervals are divided into the three color dimensions of RGB. The number of pixels corresponding to each interval is counted among all pixels in the window. Based on the number of pixels in each interval, a histogram feature vector is generated. By statistically analyzing the color histogram, the histogram feature vectors of the corresponding moving window and neighboring windows are obtained. The distance between the histogram feature vectors is used to characterize the color feature difference between the moving window and neighboring windows. If the difference value is greater than a preset threshold, the corresponding moving window pixels are marked as color edge points. Analyze all color edge points in the target image to obtain color edge regions, and statistically analyze the histogram feature vectors of the edge regions to obtain color contour features.
6. The method for classifying marine debris based on visual analysis according to claim 1, characterized in that, The image set includes multiple images, each containing one or more types of marine debris.
7. The method for classifying marine debris based on visual analysis according to claim 1, characterized in that, Step 4 specifically includes: In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced to cluster the contour data. During the clustering process, the differences between contour data are reflected by the weighted difference of the two contour features. Multiple weight ratios are set to obtain multiple clustering results. In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced, and the contour data is used as clustering samples to perform contour difference analysis and clustering on each image. Set multiple weight ratios, each of which includes the proportion of grayscale and color outline; In the clustering process, a weight ratio is introduced to evaluate the similarity between clustered samples by weighted difference between contour data. A preset bandwidth is set as the neighborhood range, and all clustered samples are used as center points. The center point drift is calculated by Gaussian kernel function, and the mean drift of the center points is performed until the preset number of iterations is reached or all center points converge. All data that converge to the same center point are grouped into the same cluster, and the clustering result is obtained. The difference analysis between contour data specifically involves calculating the distance value D1 between gray contour features and the distance value D2 between color contour features in two clustered samples, and then taking a weighted average of D1 and D2 according to the aforementioned weight ratio to obtain the mean difference, which reflects the similarity between clustered samples. Multiple clustering evaluations were performed based on various weighting ratios to obtain multiple clustering results.
8. The method for classifying marine debris based on visual analysis according to claim 1, characterized in that, Step 5 specifically involves: The weight ratio corresponding to the result with the most clusters is used as the gray-scale to color contour ratio and marked as the contour reference weight. Based on the reference weight of the contour, the contour data corresponding to the lower weight is reduced. The amount of data to be reduced is set based on the reference weight, and the optimal contour data is obtained.
9. The method for classifying marine debris based on visual analysis according to claim 1, characterized in that, Step 6 includes: By using a pre-defined classification model, image segmentation and target recognition are performed using optimized contour data, and marine debris statistics are compiled by combining information on the weight, category, size, and source of marine debris.
10. A visual analysis-based system for sorting marine debris, characterized in that, The system includes: a memory, a processor, and a communication interface. The memory includes a visual analysis-based seabed debris sorting program. When the processor executes the visual analysis-based seabed debris sorting program, it performs the following steps: Step 1: In the preset sea area, record the seabed debris image information collected at each station and seal the seabed debris; Step 2: Clean, dry, and pre-dry the seabed debris, and record images of the debris to obtain an image set; Step 3: Perform image preprocessing on the image set, set an N×N moving window, extract contour features from the image set using the Sobel operator to obtain grayscale contour features, introduce a color histogram and moving window, analyze the color differences of the windows in the image set, and extract color contour features. Step 4: In the image set, the grayscale contour features and color contour features of each image are used as contour data. The mean-shift clustering algorithm is introduced to cluster the contour data. During the clustering process, the differences between contour data are reflected by the weighted differences of the two contour features. Multiple weight ratios are set to obtain multiple clustering results. Step 5: Use the weight ratio corresponding to the result with the most clusters as the gray-level and color contour ratio, and use the contour ratio as the contour reference weight. Reduce the data from the gray-level contour features and color contour features in the image set to obtain the preferred contour data. Step 6: Introduce a preset classification model, combine the image set with the selected contour data to perform garbage classification and statistics, and obtain statistical information.